JP5020920B2 - Pedestrian detection device and program - Google Patents

Description

The present invention relates to a pedestrian detecting device, and a program, in particular, relates to a pedestrian detecting device and a program for detecting a pedestrian from the captured image.

  2. Description of the Related Art In recent years, an increasing number of vehicles are equipped with an object detection device that performs image processing on an image around a vehicle imaged by an in-vehicle camera, detects an object such as a pedestrian, and presents a detection result to a driver.

  As a method for detecting an object, a learning model dictionary in which an object pattern is learned in advance is generated, and it is determined whether or not an input image is close to data registered in the learning model dictionary. The method of detecting is common. Specifically, for example, there are various techniques such as the technique of Viola & Jones.

For example, a difference in luminance value between two pixels of a set of pixels of an input image is obtained as a feature amount, and the set of pixels is detected by a plurality of discriminating means previously determined by learning based on the obtained feature amount. An object detection device that detects a detection target object of an input image by determining whether or not it is a contour part of the input image has been proposed (Patent Document 1).
JP 2006-318341 A

  However, in the object detection device of Patent Document 1, for example, as illustrated in FIG. 9, an image extracted for detecting a person includes (a) a plurality of persons overlapping, and (b) a person obliquely. Inclined or carrying a large piece of luggage on one side, (c) The outline of the person is unclear due to light from the background light, etc. (d) The person overlaps the background with white lines or signs, etc. If the outline of the figure is unclear, or (e) a person is riding a bicycle or the like (especially in a landscape orientation), the person may not be detected accurately.

  The present invention has been proposed to solve the above-described problems, and an object thereof is to provide a person detection apparatus and program capable of improving the accuracy of person detection.

In order to achieve the above object, a pedestrian detection apparatus according to the present invention includes an image dividing unit that divides an extracted image extracted from an input image into left and right regions, an image of a left region divided by the image dividing unit, and the left side. Comparison with a learning model for detecting a person corresponding to an image in the area, comparison between an image in the right area divided by the image dividing means and a learning model for detecting a person corresponding to the image in the right area And a person is detected in the extracted image when the person is detected by one of the comparison between the whole extracted image and a learning model for detecting a person corresponding to the whole extracted image. Determining means for performing the above-described operation.

According to the pedestrian detection device of the present invention, the image dividing unit divides the extracted image extracted from the input image into left and right regions, and the determining unit includes the left region image and the left region divided by the image dividing unit. Comparison with a learning model for detecting a person corresponding to an image, comparison between a right region image divided by the image dividing means and a learning model for detecting a person corresponding to the right region image, and an extracted image When the person is detected by any one of the comparisons with the learning model for detecting the person corresponding to the whole of the extracted image and the person corresponding to the whole of the extracted image, it is determined that there is a person in the extracted image.

  In this way, since the image of the left region and the image of the right region are detected as such, the person can be accurately detected even if unnecessary information is included in either the left region or the right region of the extracted image. Can be detected.

The pedestrian detection apparatus of the present invention includes an image dividing unit that divides an extracted image extracted from an input image into left and right regions, and an image in a left region and an image in a right region that are divided by the image dividing unit. Image reversing means for reversing the left and right, and the division so that the leftmost pixel of the image in the left area reversed by the image reversing means is adjacent to the rightmost pixel of the image in the left area divided by the image dividing means An image obtained by combining the image of the left area and the image of the inverted left area, and an image of the right area inverted by the image inversion means to the leftmost pixel of the image of the right area divided by the image dividing means of as the right edge of the pixel are adjacent, either of the whole of the divided right area image and the image obtained by synthesizing the image of the inverted right area, and the extracted image, walking Pedestrians in comparison to training models for detecting the if it is detected, is configured to include a, a determining means for determining that there is a pedestrian on the extracted image.

According to the pedestrian detection means of the present invention, the image inversion means inverts the left and right of the left area image and the right area image divided by the image dividing means, and the determination means is divided by the image dividing means. as the leftmost pixel of the image of the left region which is inverted by the image inverting means to the right end of the pixels of the image of the left area are adjacent divided image and inverted left area of the left region image and the synthesized image The right-side divided image and the image of the right-side area divided by the image-dividing means are adjacent to the right- side pixel of the right- side area image inverted by the image inverting means . determination image obtained by synthesizing the image of the right area, and either the whole extracted image, when a pedestrian in comparison with a learning model for detecting pedestrian has been detected, the pedestrian extraction image there and That.

  In this way, an image obtained by synthesizing the image of the left area divided by the image dividing means and the image of the left area inverted by the image inverting means, and the image of the right area divided by the image dividing means and the image inverted by the image inverting means. By using an image that is synthesized with the image of the right area, the person can be detected accurately even if unnecessary information is included in either the left area or the right area of the extracted image, as described above. can do. In addition, it is not necessary to prepare a learning model corresponding to the left region image, the right region image, and the entire extracted image, and the learning model corresponding to the entire extracted image can be used in common.

Moreover, the said determination means of the pedestrian detection apparatus of this invention can be compared with a learning model using the feature-value of an image. Therefore, it is preferable to use a feature amount corresponding to a technique used in comparison with the learning model, for example, a technique such as Viola & Jones or SVM (Support Vector Machine).

The pedestrian detection program according to the present invention includes a computer that divides an extracted image extracted from an input image into left and right areas, an image of a left area divided by the image dividing means, and an image of the left area. Comparison with a learning model for detecting a pedestrian corresponding to an image, Comparison between an image in the right area divided by the image dividing means and a learning model for detecting a pedestrian corresponding to the image in the right area and if the whole with a pedestrian in any one of the comparison of the comparison of the learning model for detecting pedestrians corresponding to the entire of the extracted image of the extracted image is detected, the pedestrian on the extracted image It is for functioning as a determination means for determining presence.

The pedestrian detection program according to the present invention includes a computer that divides an extracted image extracted from an input image into left and right regions, an image of a left region and an image of a right region divided by the image dividing unit. Image reversing means for reversing the left and right of each of the above, an image obtained by combining the image of the left area divided by the image dividing means and the image of the left area reversed by the image reversing means, divided by the image dividing means The pedestrian is compared with a learning model for detecting a pedestrian and either the image obtained by combining the image of the right region and the image of the right region inverted by the image inverting means or the entire extracted image. When it is detected, it serves as a determination means for determining that there is a pedestrian in the extracted image.

As described above, according to the present invention, it is possible to improve the detection accuracy of pedestrians .

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)

As illustrated in FIG. 1, the person detection device 10 according to the first embodiment includes an imaging device 12 that captures a range including a region to be identified, and a person (walking) based on a captured image output from the imaging device 12. A computer 16 for executing a person detection processing routine for detecting a person ) and a display device 18 for displaying a processing result in the computer 16.

  The imaging device 12 captures an area including an identification target region and generates an image signal (not shown), and A / D conversion that converts an analog image signal generated by the imaging unit into a digital signal And an image memory (not shown) for temporarily storing the A / D converted image signal.

  The computer 16 includes a CPU that controls the entire person detection apparatus 10, a ROM as a storage medium that stores a person detection processing program to be described later, a RAM that temporarily stores data as a work area, and a bus that connects these. It is configured. If the computer 16 is described with functional blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 1, an image input unit 20 for inputting a captured image output from the imaging device 12. A window image extraction unit 22 that extracts a predetermined area from the captured image input by the image input unit 20, and a division that generates a divided image for determining the presence or absence of a person from the image extracted by the window image extraction unit 22 Based on the image generation unit 24, the learning model DB 26 storing a learning model generated in advance by a learning image, the image generated by the divided image generation unit 24, and the learning model stored in the learning model DB 26. The determination unit 28 for determining the presence or absence and the determination result by the determination unit 28 are superimposed on the captured image captured by the imaging device 12. Is allowed can be represented by the structure that includes a display control unit 30 for controlling so as to be displayed on the display device 18.

  When the window image extraction unit 22 detects a person from the input image input from the image input unit 20, a window having a predetermined size (referred to as a search window) from the input image is determined in advance for each step. The image is cut while being moved by the movement amount (referred to as a search pitch), and a person is detected from the cut image. Here, the cut image is referred to as a window image, and the size of the window image (that is, the size of the search window) is referred to as a window size. A plurality of types of window sizes are set in order to detect persons of various sizes, and the window image extraction unit 22 extracts window images using search windows of all the set window sizes. In addition, the window image extraction unit 22 converts the extracted window image into an image having a preset number of pixels (for example, an image of 16 horizontal x 32 vertical pixels).

  The divided image generation unit 24 normalizes the luminance value of the extracted window image so as to be able to compare the learning model described later and the extracted window image so as to eliminate the influence of the luminance distribution that is different for each image. Divide into left and right regions. Here, the window image is divided at the center of the window image so that the number of pixels of the left region image and the right region image is equal (for example, 8 × 32 pixels).

  The learning model DB 26 is a built-in or external storage unit such as a hard disk drive (HDD) or a CD-ROM, and is configured of a medium that can store the learning model. In the learning model generation unit 32, the learning model is generated by a learning model generation processing routine described later, the learning model corresponding to the image in the left region of the extracted image, the learning model corresponding to the right region, and the learning corresponding to the entire extracted image. Each of the models is generated. The learning model stored in the learning model DB 26 is used for detection of a person by the determination unit 28 of the person detection device 10.

  The learning model generation unit 32 can be configured by a computer including a CPU, a ROM, a RAM, a built-in HDD, and the like. In the case of such a configuration, a program for realizing the function of each component is stored in a storage medium such as a ROM or HDD, and each function is realized by executing the program by the CPU. To. In addition, the learning model generation unit 32 may be configured by a microcomputer independent of the person detection apparatus 10 or may be configured on the same computer.

  The determination unit 28 includes an image of the left region, an image of the right region, and an entire image of the extracted image generated by the divided image generation unit 24, and the corresponding learning model stored in the learning model DB 26. By comparison, it is determined whether the image of the left region, the image of the right region, and the entire extracted image is a person. If any one of the images is determined to be a person, the window image Is determined to be a person.

  Here, a learning model generation processing routine stored in the learning model 26 will be described with reference to FIG. Here, a case where SVM is used as a learning method will be described. As an object image of an image used for learning (learning image), an image in which a person (one with different orientation, background, clothes, and belongings) is taken, and a non-object image of a learning image other than a person ( A predetermined number of images (for example, 1000 images each) in which a telephone pole, a signboard, or the like, in particular, an object similar to a person) is taken is prepared.

  In step 100, one learning image is input to the learning model generation unit 32. Next, in step 102, an area including a person or a target corresponding to a person is included in the input learning image. The image is cut with a predetermined aspect ratio (for example, horizontal 1: vertical 2), and then, in step 104, the cut image is normalized to a predetermined size (for example, 16 × 32 pixels) by a bilinear method or the like.

  Next, in step 106, the image normalized to a predetermined size (entire image) is divided into an image in the left region and an image in the right region. Here, the image in the left area and the image in the right area are divided at the center of the entire image so that they have the same size (for example, 8 × 32 pixels). For example, in the entire image normalized to 16 × 32 pixels as shown in FIG. 3A, a represents a line indicating the center of the entire image (boundary line between the eighth pixel and the ninth pixel from the left end). If so, the entire image is divided by line a to generate a left half image shown in (B) and a right half image shown in (C). The generated image and the entire image are temporarily stored in the RAM in the learning model generation unit 32.

  Next, in step 108, it is determined whether or not the processing for generating the left half image and the right half image has been completed for all prepared learning images. If completed, the process proceeds to step 110. If not completed, the process returns to step 100 to input the next learning image and repeat the process of generating the left half image and the right half image.

  In step 110, the left half image stored in the RAM in the learning model generation unit 32 is read out, and the luminance is normalized using, for example, the standard deviation, and the four-directional surface features are extracted as image feature amounts. Then, a learning model for the left half image is generated using the SVM method. The right half image and the entire image are processed in the same manner, and a learning model for the right half image and a learning model for the entire image are generated. The generated learning model is stored in the learning model DB 26.

  Next, a person detection processing routine in the first embodiment will be described with reference to FIG.

  In step 200, the image captured by the imaging device 12 is input to the window image extraction unit 22 as an image to be detected by the image input unit 20.

  Next, in step 202, the window image extraction unit 22 sets a search window of, for example, 16 × 32 pixels for the input image as a predetermined area (for example, a left corner area) of the input image, and sets the set search window. And a 16 × 32 pixel window image is extracted from the input image.

  Next, in step 204, the divided image generation unit 24 normalizes the luminance of the window image extracted by the window image extraction unit 22 using the standard deviation. As a method for normalizing the luminance, it is possible to use various methods such as correcting the luminance value so that the luminance value becomes the threshold value for pixels whose luminance value exceeds the threshold value. Although it is not limited, the same method as that used when generating the learning model is used.

  Next, in step 206, the divided image generation unit 24 divides the window image into a left region image and a right region image. Here, the image in the left area and the image in the right area are divided at the center of the entire image so that they have the same size (for example, 8 × 32 pixels). For example, in the window image as shown in FIG. 5A, when b represents a line indicating the center of the window image (the boundary line of the eighth and ninth pixels from the left end), the window image is represented by the line b. The image is divided to generate the left half image shown in FIG.

  Next, in step 208, the determination unit 28 calculates the four-direction plane feature as the feature amount of the left half image, and in the next step 210, compares it with the learning model for the left half image stored in the learning model DB 26. Then, the probability of the person is calculated as an evaluation value using the SVM method.

  Next, in step 212, the determination unit 28 determines whether the left half image is a person based on the calculated evaluation value. If it is determined that the person is a person, the process proceeds to step 228. If it is determined that the person is not a person, the process proceeds to step 214.

  In step 214, the right half image shown in FIG. 5C is generated by the same processing as in step 206. In step 206, when the left half image is generated, the right half image is also generated and this step can be omitted.

  Next, in step 216 to step 220, as in the case of the left half image in step 208 to step 212, the right half image is also compared with the learning model for the right half image stored in the learning model DB 26. Then, it is determined whether or not the right half image is a person. If it is determined that the person is a person, the process proceeds to step 228. If it is determined that the person is not a person, the process proceeds to step 222.

  In step 222 to step 226, as in the case of the left half image in step 208 to step 212, the window image is also compared with the learning model for the entire image stored in the learning model DB 26. It is determined whether or not it is a person. If it is determined that the person is a person, the process proceeds to step 228. If it is determined that the person is not a person, the process proceeds to step 230.

  In step 228, the determination unit 28 stores information such as the position and size of the search window when each image is determined to be a person in steps 212, 220, and 226 as a list in the RAM.

  Next, in step 230, the determination unit 28 scans the search window for the entire input image and determines whether the search has ended. If completed, the process proceeds to step 234. If not completed, the process proceeds to step 232.

  In step 232, the window image extraction unit 22 moves the position of the search window by a predetermined search pitch, returns to step 202, and repeats the processing from step 202 to step 230. When the search window finishes searching the entire image, the process proceeds to step 234.

  In step 234, the determination unit 28 determines whether or not the search in the search windows of all sizes has been completed. Here, the search window is used as a frame for extracting a window image for detecting a person. However, if the size of the search window is different, persons of various sizes can be detected. In the present embodiment, search windows of various sizes are prepared in advance, and it is necessary to search the entire image in each search window. If completed, the process proceeds to step 238. If not completed, the process proceeds to step 236.

  In step 236, the window image extraction unit 22 enlarges the size of the search window by one step (for example, the search window size is 1.2 times), returns to step 202, and repeats the processing from step 202 to step 234. . When the set search window is a window having a size exceeding 16 × 32 pixels, the extracted window image is converted to 16 × 32 pixels. When the search is completed for all search window sizes, the process proceeds to step 238.

  In step 238, the determination unit 28 supplies the information stored in the list to the display control unit 30. Based on the information in the list, the display control unit 30 controls the display device 18 so that the detected person is displayed surrounded by a window with respect to the input image.

  As described above, in the person detection apparatus 10 according to the first embodiment, the presence / absence of a person is determined using the left half image, the right half image, and the entire image. For example, FIG. As shown in the left half image shown in FIG. 3B, unnecessary information is removed even if it is difficult to determine that the person is a person as a whole due to the influence of large luggage on one side of the person or background lighting. Then, since it is easy to determine that it is a person, the accuracy of detection of the person can be improved.

(Second Embodiment)

  In the first embodiment, a case has been described in which a left half image and a right half image are generated from a window image (extracted image) and a person is detected by comparison with a learning model. In the embodiment, a case will be described in which a left inverted image and a right inverted image are generated and a person is detected by comparison with a learning model. Note that, in the configuration of the person detection device of the second embodiment, the same configuration and the same processing as those of the human detection device 10 of the first embodiment are denoted by the same reference numerals, and the description thereof will be given. Is omitted.

  As shown in FIG. 6, the person detection apparatus 10 according to the second embodiment determines the presence / absence of a person from an image input unit 20, a window image extraction unit 22, and an image extracted by the window image extraction unit 22. The reverse image generation part 34 which produces | generates the reverse image for this, The learning model DB26, the determination part 28, and the display control part 30 are provided.

  The inverted image generation unit 34 normalizes the brightness value of the extracted window image so that the learning model can be compared with the extracted window image so as to eliminate the influence of different brightness distributions for each image, and the window image is converted into the left region. And divide it into a right region. Here, the window image is divided at the center of the window image so that the number of pixels of the left region image and the right region image is equal (for example, 8 × 32 pixels). Further, a left inverted image is generated by synthesizing an image obtained by horizontally inverting the left side image with the divided left side image. Further, an image obtained by horizontally inverting the image in the right region is combined with the divided image in the right region to generate a right inverted image.

  Next, a learning model generation processing routine in the second embodiment will be described. In the second embodiment, the aspect ratio of the left inverted image and the right inverted image is the same as the aspect ratio of the entire image (for example, horizontal 1 × vertical 2). Therefore, as in the first embodiment, There is no need to generate a learning model corresponding to each of the left inverted image, the right inverted image, and the entire image. Accordingly, in the learning model generation processing routine in the first embodiment shown in FIG. 2, the learning model generation processing for the left half image and the right half image in step 106 and step 110 can be omitted.

  Next, a person detection processing routine in the second embodiment will be described with reference to FIG.

  In step 200, an image is input to the window image extraction unit 22. In step 202, the window image extraction unit 22 extracts a window image. In step 204, the inverted image generation unit 34 normalizes the luminance of the window image. .

  Next, in step 300, the reverse image generation unit 34 divides the window image into a left region image and a right region image. Here, the left side image and the right side image are divided at the center of the entire image so that the image of the right side is the same size (for example, 8 × 32 pixels), and an image obtained by horizontally inverting the divided left side image is generated. Then, a left inverted image is generated by combining with the image in the left region. For example, in a window image as shown in FIG. 8A, when c represents a line indicating the center of the window image (a boundary line between the eighth pixel and the ninth pixel from the left end), the window image is represented by a line c. The left half image is divided to generate a left-right inverted image of the left half image, and the left half image and the left half image are inverted so that the left end pixel of the inverted image is adjacent to the right end pixel of the left half image. Are combined to generate a left inverted image shown in FIG.

  Next, in step 300, the determination unit 28 extracts the feature amount of the left inverted image, and in the next step 304, the determination unit 28 compares the learning model stored in the learning model DB 26 with the SVM technique. Is calculated as an evaluation value.

  Next, in step 212, the determination unit 28 determines whether the window image is a person. If it is determined that the person is a person, the process proceeds to step 228. If it is determined that the person is not a person, the process proceeds to step 306.

  In step 306, the right inverted image shown in FIG. In step 300, when generating a left inverted image, it is also possible to generate a right inverted image and omit this step.

  Next, in Step 308, Step 310, and Step 220, as in the case of the left inverted image in Step 208 to Step 212, the right inverted image is also compared with the learning model stored in the learning model DB 26. It is determined whether or not the window image is a person. Here, the learning model used for comparison is different from the case of the first embodiment using the learning model for the left half image, the right half image, and the whole image, and the left inverted image, the right inverted image, and A common learning model can be used for the entire image. If it is determined that the person is a person, the process proceeds to step 228. If it is determined that the person is not a person, the process proceeds to step 222.

  As described above, in the person detection apparatus 10 according to the second embodiment, the presence or absence of a person is determined using the left inverted image, the right inverted image, and the entire image. For example, FIG. Even if it is difficult to determine that the person is a person as a whole due to the influence of large luggage on one side of the person or background lighting, the left inverted image shown in FIG. Since it is easy to determine that the person is a person, it is possible to improve the accuracy of detection of the person as in the case of the first embodiment, and the left inverted image and the right inverted image generated as the determination image are extracted. Since it is the same size as the image, it is not necessary to provide separate learning models for the left inverted image, the right inverted image, and the entire image, and the generation of the learning model can be simplified.

  In the first embodiment and the second embodiment, the case where the SVM method is used for generating the learning model and detecting the person has been described. However, the Haar-like feature is calculated as the feature amount, The method of Viola & Jones may be used.

It is a block diagram which shows the structure of the person detection apparatus 10 of 1st Embodiment. It is a flowchart which shows the production | generation process routine of the learning model of 1st Embodiment. It is a figure explaining learning model generation in a 1st embodiment. It is a flowchart which shows the person detection process routine of 1st Embodiment. It is a figure explaining the division | segmentation image generation in 1st Embodiment. It is a block diagram which shows the structure of the person detection apparatus 10 of 2nd Embodiment. It is a flowchart which shows the person detection processing routine of 2nd Embodiment. It is a figure explaining the reverse image generation in 2nd Embodiment. It is a figure which shows an example of an extraction image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Person detection apparatus 22 Window image extraction part 24 Division | segmentation image generation part 26 Learning model DB
28 judgment part

Claims (3)

Image dividing means for dividing the extracted image extracted from the input image into left and right regions;
Image reversing means for reversing the left and right of the image of the left area and the image of the right area divided by the image dividing means;
The divided left side image and the inversion so that the left end pixel of the left region image inverted by the image inversion unit is adjacent to the right end pixel of the left region image divided by the image dividing unit. An image obtained by combining the image of the left area, and the right end pixel of the right area image inverted by the image inversion means is adjacent to the left end pixel of the right area image divided by the image dividing means. The pedestrian is compared with a learning model for detecting a pedestrian with either the image obtained by combining the image of the divided right region and the image of the inverted right region, or the entire extracted image. A determination means for determining that there is a pedestrian in the extracted image when detected,
A pedestrian detection device. The pedestrian detection apparatus according to claim 1 , wherein the determination unit compares a feature amount of an image with the learning model. Computer
Image dividing means for dividing the extracted image extracted from the input image into left and right regions;
Image reversing means for reversing the left and right of the image of the left area and the image of the right area divided by the image dividing means;
The divided left side image and the inversion so that the left end pixel of the left region image inverted by the image inversion unit is adjacent to the right end pixel of the left region image divided by the image dividing unit. An image obtained by combining the image of the left area, and the right end pixel of the right area image inverted by the image inversion means is adjacent to the left end pixel of the right area image divided by the image dividing means. The pedestrian is compared with a learning model for detecting a pedestrian with either the image obtained by combining the image of the divided right region and the image of the inverted right region, or the entire extracted image. A determination means for determining that there is a pedestrian in the extracted image when detected,
Pedestrian detection program to make it function.