JP5399887B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device that recognizes a monitoring object around a vehicle from an image captured by a camera attached to the vehicle.

  Conventionally, a vehicle periphery monitoring device has been proposed in which a monitoring object such as a pedestrian that may come into contact with the host vehicle is detected from an image around the host vehicle captured by a camera and is notified to the driver. (For example, refer to Patent Document 1).

  In the vehicle periphery monitoring device described in Patent Document 1, an image part similar to the shape of the pedestrian's head is extracted from a binary image obtained by binarizing a grayscale image captured by an infrared camera, A pedestrian determination process is performed on this image portion.

Japanese Patent No. 4283266

  When a pedestrian is imaged with an infrared camera, the pedestrian's head has a higher temperature than the surrounding area, and thus becomes a high-luminance region in a grayscale image. Therefore, normally, an image portion having a head feature can be extracted relatively easily from a binary image obtained by binarizing the grayscale image.

  However, when the pedestrian is wearing glasses, the temperature of the glasses is lower than the pedestrian's head, the reflectivity / emissivity is different from the pedestrian, the infrared rays from the pedestrian's head are blocked, etc. Due to the influence, there are cases where the image portion of the head which becomes black (low luminance region) and becomes white (high luminance region) in the binary image is divided into upper and lower parts by the spectacle portion. In addition, a part of the image portion of the head may be largely lost due to the spectacle portion. In these cases, the image portion of the pedestrian's head does not have a head feature, and the pedestrian cannot be determined.

  Similarly, when the pedestrian is wearing a hat (sun visor or the like), the image portion of the pedestrian's head may be divided or lost.

  Therefore, an object of the present invention is to provide a vehicle periphery monitoring device that suppresses the determination that a pedestrian wearing glasses or a pedestrian wearing a hat cannot be determined to be a pedestrian.

  The present invention has been made in order to achieve the above object, and is a vehicle periphery monitoring device that recognizes a monitoring object around a host vehicle based on an image captured by an in-vehicle infrared camera. Binary image candidate extraction means for extracting a processing target image portion from a binary image generated by binarizing the scale image, and whether or not the processing target image portion has the characteristics of a pedestrian's head It is related with the vehicle periphery monitoring apparatus provided with the pedestrian identification means which judges whether the object on the real sky corresponding to the said process target image part is a pedestrian.

And the 1st aspect of this invention WHEREIN: The said pedestrian identification means determines whether the two image parts which faced the perpendicular direction with the space | interval in a predetermined range were extracted as the said process target image part. And when determining that two image portions facing each other in the vertical direction with an interval within a predetermined range have been extracted by the determination unit, the two image portions are integrated by filling the interval portion, Determining whether the converted image portion has the characteristics of the pedestrian's head and identifying whether the object in the real space corresponding to the two image portions is a pedestrian or not. To do.

  According to the present invention, since the pedestrian is wearing glasses or the pedestrian is wearing a hat, the binary image of the head is divided vertically by an image portion such as the brim of the glasses or the hat. When the binary image candidate extracting means extracts two image parts that are vertically opposed with an interval within the predetermined range as the processing target image part, the pedestrian identifying means The two image parts are integrated by filling the part. And the said pedestrian identification means judges the presence or absence of a pedestrian's characteristic part about this integrated image part.

  Therefore, the image part of the pedestrian's head is divided up and down by image parts such as eyeglasses or hat collars, and each divided image part has head characteristics (height, width, aspect ratio, etc.). Even if it is no longer performed, it can be suppressed that the pedestrian cannot be identified by determining the presence or absence of the characteristics of the head of the pedestrian in the integrated image portion.

Next, according to a second aspect of the present invention, the pedestrian identification means has a horizontally long missing portion surrounded by three high-intensity areas of a binary image at the substantially central portion as the processing target image portion. image portion having comprising a determining means for determining Taka not been extracted, sometimes it is determined that the image portion having a long defective part in the horizontal direction is extracted at a substantially central portion by said judging means, fills the該欠loss portion Determining whether or not the image portion has the characteristics of the head of the pedestrian, and identifying whether or not the object in the real space corresponding to the image portion having the missing portion is a pedestrian And

  According to the present invention, since the pedestrian is wearing spectacles or the pedestrian is wearing a hat, the image portion of the spectacles or the brim of the cap is placed at the approximate center of the binary image of the head. When a processing target image portion having a horizontally long missing portion is extracted by the image portion, the pedestrian identification unit fills in the missing portion. And the said pedestrian identification means judges the presence or absence of the characteristic of a pedestrian about the image part by which the defect | deletion part was filled in this way.

  For this reason, there is a case where the image portion of the pedestrian's head has a missing portion due to the brim portion of glasses or a hat and the image portion does not have the characteristics of the head (area, sufficiency, etc.). However, it is possible to prevent the pedestrian from being able to be identified by determining the presence or absence of the characteristics of the pedestrian's head for the image portion in which the missing portion is filled.

The block diagram of the vehicle periphery monitoring apparatus of this invention. Explanatory drawing of the attachment aspect to the vehicle of the vehicle periphery monitoring apparatus shown in FIG. 3 is a flowchart showing a processing procedure by the image processing unit shown in FIG. 1. The flowchart of a pedestrian identification process. Explanatory drawing of the process which unifies two process target image parts. Explanatory drawing of the process which fills the defect | deletion part of a process target image part.

  An embodiment of the present invention will be described with reference to FIGS. Referring to FIG. 1, a vehicle periphery monitoring apparatus according to the present invention includes infrared cameras 2R and 2L that can detect far infrared rays, a yaw rate sensor 3 that detects the yaw rate of the vehicle, and a vehicle speed sensor 4 that detects the traveling speed of the vehicle. And a brake sensor 5 that detects the amount of brake operation by the driver, and an object to be monitored in front of the vehicle (such as a pedestrian) from an image obtained by the infrared camera 2, and the object to be monitored can come into contact with the vehicle Image processing unit 1 that outputs an alarm when there is a high probability, a speaker 6 for outputting an alarm by sound, and an image obtained by the infrared cameras 2R and 2L, and a monitoring target with a high possibility of contact And a display device 7 for performing display for causing the driver to visually recognize the object.

  Referring to FIG. 2, infrared cameras 2 </ b> R and 2 </ b> L are disposed at the front portion of vehicle 10 at a position that is substantially symmetrical with respect to the center portion of vehicle 10 in the vehicle width direction. The two infrared cameras 2R and 2L are fixed with their optical axes parallel to each other and with the same height from the road surface. The infrared cameras 2R and 2L have a characteristic that the output level increases (the luminance increases) as the temperature of the imaged object increases. Further, the display device 7 is provided so that the screen 7 a is displayed at a front position on the driver side of the front window of the vehicle 10.

  Referring to FIG. 1, an image processing unit 1 is an electronic unit composed of a microcomputer (not shown) and converts analog video signals output from the infrared cameras 2R and 2L into digital data. The microcomputer has a function of fetching it into an image memory (not shown) and performing various arithmetic processes by the microcomputer with respect to the image ahead of the vehicle taken into the image memory.

  The image processing unit 1 functions as the processing target image extraction unit 20 and the pedestrian identification unit 21 by causing the microcomputer to execute a vehicle periphery monitoring program.

  Next, according to the flowchart shown in FIG. 3, the monitoring target object detection and alerting processing by the image processing unit 1 will be described.

  The image processing unit 1 first inputs an analog signal of an infrared image output from the infrared cameras 2R and 2L in STEP1, and then stores a grayscale image obtained by digitizing the analog signal by A / D conversion in an image memory in STEP2. To do. In STEP1 to STEP2, a grayscale image (hereinafter referred to as a right image) obtained by the infrared camera 2R and a grayscale image (hereinafter referred to as a left image) obtained by the infrared camera 2L are acquired. Since the right image and the left image have a shift (parallax) in the horizontal position of the image portion of the same object, the distance from the vehicle 10 to the object in real space can be calculated based on this parallax. it can.

  In subsequent STEP 3, the image processing unit 1 binarizes the right image as a reference image (pixels with luminance equal to or higher than a threshold value are set to “1 (white)”, and pixels smaller than the threshold value are set to “0 (black)”. To generate a binary image.

  The next STEP 4 to STEP 6 are processes by the processing target image extracting means 20. In STEP 4, the processing target image extracting unit 20 converts the image portion of each white area included in the binary image to run length data (data of white pixel lines continuous in the x (horizontal) direction of the binary image). . Further, the processing target image extracting unit 20 labels a line having a portion overlapping in the y (vertical) direction of the binary image as one image portion in STEP 5, and uses the labeled image portion as a processing target image portion in STEP 6. Extract.

  In the next STEP 7, the image processing unit 1 calculates the center of gravity G, the area S, and the aspect ratio ASPECT of the circumscribed square of each processing target image portion. The specific calculation method is described in detail in, for example, the above-mentioned Japanese Patent No. 4283266, and thus the description thereof is omitted here. Then, the image processing unit 1 executes the following STEP 8 to STEP 9 and STEP 20 to STEP 22 in parallel.

  In STEP 8, the image processing unit 1 determines the identity of the image of the target object extracted from the binary image based on the images captured by the infrared cameras 2R and 2L at every predetermined sampling period. The time-series data of the position (center of gravity position) of the image determined to be an image is stored in the memory (time tracking). In STEP 9, the image processing unit 1 reads the vehicle speed VCAR detected by the vehicle speed sensor 4 and the yaw rate YR detected by the yaw rate sensor 3, and calculates the turning angle θr of the vehicle 10 by time integration of the yaw rate YR. To do.

  On the other hand, in STEP 20, the image processing unit 1 selects one of the target image candidates tracked by the binary image of the reference image (right image), and searches the search image R1 from the grayscale image of the right image. (An image of the entire region surrounded by a circumscribed rectangle of the selected candidate image) is extracted. In subsequent STEP 21, the image processing unit 1 sets a search area for searching for an image corresponding to the search image R1 (hereinafter referred to as the corresponding image R1 ′) from the grayscale image of the left image, and performs an inter-phase calculation with the search image R1. The corresponding image R1 ′ is extracted by executing. In STEP 22, the image processing unit 1 calculates the difference between the centroid position of the search image R 1 and the centroid position of the corresponding image R 1 ′ as a parallax amount Δd (number of pixels), and proceeds to STEP 10.

  In STEP 10, the image processing unit 1 calculates the distance z between the vehicle 10 and the object based on the parallax amount Δd, and calculates the coordinates (x, y) and the distance z of the search image as real space coordinates (X, Y, Z) to calculate the coordinates of the real space position corresponding to the search image. As shown in FIG. 2, the real space coordinates (X, Y, Z) are set such that the midpoint position of the attachment positions of the infrared cameras 2R, 2L is the origin 0, X is the vehicle width direction of the vehicle 10, and Y Is set in the vertical direction, and Z is set in the forward direction of the vehicle 10. The image coordinates are set such that the center of the image is the origin, the horizontal direction is x, and the vertical direction is y.

  In subsequent STEP 11, the image processing unit 1 performs a turning angle correction for correcting a positional deviation on the image due to the turning of the vehicle 10. The subsequent STEP 12 is a process performed by the movement vector calculation means 21. The movement vector calculation means 21 is the real space position of the same object after turning angle correction obtained from a plurality of images captured within a predetermined monitoring period. The relative movement vector between the object and the vehicle 10 is calculated from the time series data.

  The specific calculation method of the real space coordinates (X, Y, Z) and the movement vector is described in detail in the above-mentioned Japanese Patent No. 4283266, and the description thereof is omitted here.

  Next, in STEP 13, the image processing unit 1 determines a possibility of contact between the vehicle 10 and the detected object, and performs “attention determination processing” for determining whether or not it is necessary to perform attention. Run. When it is determined by the “attention call determination process” that it is necessary to call attention, the process branches to STEP 30 to output a warning sound by the buzzer 6 and display a warning on the display device 7. On the other hand, when it is determined by the “attention calling determination process” that it is not necessary to call attention, the process returns to STEP 1 and the image processing unit 1 does not call attention.

  Next, the process by the pedestrian identification means 21 performed in the “attention calling determination process” will be described with reference to the flowchart shown in FIG. The pedestrian identification unit 21 determines whether each processing target image portion extracted by the processing target image extraction unit 20 has a pedestrian's head feature.

  The pedestrian identification means 21 corresponds to the pedestrian's head, in which the size of the processing target image portion is set based on the distance between the real space position corresponding to the processing target image portion and the vehicle 10 in STEP 50 of FIG. It is judged whether it is the magnitude | size. Then, when the size of the processing target image portion is a size corresponding to the head of the pedestrian, the process proceeds to STEP 51, and when it is not the size corresponding to the head, the process branches to STEP 60.

  Here, as shown in FIG. 5A, when the grayscale image 70 of the head part of the pedestrian wearing glasses is binarized, the image part of the head in the part of the glasses having a low temperature becomes As shown in (b), it is divided into two upper and lower parts 71, 71, and as shown in (c), it may be extracted as two processing target image parts 75, 76. In this case, since the processing target image portions 75 and 76 do not have the characteristics (size, center of gravity, aspect ratio, etc.) of the pedestrian's head, they are not extracted as the image portion of the pedestrian's head. become.

  Accordingly, in STEP 60, the pedestrian identification means 21 searches for another processing target image portion that is opposed to the processing target image portion 71 with an interval within a predetermined range. When no other processing target image portion is detected, the process proceeds from STEP 61 to STEP 70, and the pedestrian identification unit 21 determines that the processing target image portion 71 is not an image portion of the pedestrian's head. Then, the process proceeds to STEP 55 and the process is terminated (if there is another processing target image portion, the processing according to the flowchart of FIG. 4 is executed again for the processing target image portion).

  On the other hand, as shown in FIG. 5C, when another processing target image portion 72 is detected in STEP 60, the process proceeds from STEP 61 to STEP 62, and the pedestrian identification means 21 It is determined whether the size of the combined portion of the upper processing target image portion 75 and the lower processing target image portion 76 corresponds to the size of the upper portion of the pedestrian's head. To do.

  The size of the upper processing target image portion 75 and the combined portion of the upper processing target image portion 75 and the lower processing target image portion 76 is the size of the upper portion of the pedestrian's head. If it is equivalent, the process proceeds to STEP 63, and the pedestrian identification means 21 determines the interval between the upper processing target image portion 75 and the lower processing target image portion 76 as shown in FIG. It is embedded (pixels in the interval are set to “1 (white)”) and integrated.

  Then, proceeding to STEP 54, the pedestrian identification means 21 determines whether or not the integrated processing target image portion 80 has the characteristics of the pedestrian's head, and in the real space corresponding to this image portion. It is determined whether or not the object is a pedestrian. Thereby, even if it is a case where the image part of a pedestrian's head is divided | segmented up and down by the image part of spectacles, the image part of a pedestrian's head can be extracted.

  On the other hand, if the size of the combined portion of the upper processing target image portion 75 and the lower processing target image portion 76 is not equivalent to the size of the upper portion of the pedestrian's head, the process proceeds to STEP 70. After branching, the pedestrian identification means 21 determines that the upper processing target image portion 75 is not the pedestrian image portion, proceeds to STEP 7, and ends the processing.

  Next, in STEP 51, the pedestrian identification means 21, in the processing target image portion, in the processing target image portion 80, as shown in FIG. It is determined whether or not there is a black region surrounded on all sides. If there is a missing part, the process proceeds to STEP 52, and if there is no missing part, the process branches to STEP 54.

  In STEP 52, the pedestrian identification means 21 determines whether or not the size and position of the missing portion in the processing target image portion 80 is a size and position equivalent to glasses. When the size and position of the missing portion in the processing target image portion 80 are the size and position equivalent to glasses, the process proceeds to STEP 53, and the pedestrian identification means 21 fills the missing portion (the pixel of the missing portion is set to “1”). (White) "). For example, in the case of FIG. 6B, the defect portion 91 is filled and the state of FIG.

  Then, proceeding to STEP 54, the pedestrian identification means 21 determines whether or not the image portion having the missing portion has the characteristics of the head of the pedestrian, and in the real space corresponding to this image portion. It is determined whether or not the object is a pedestrian. As a result, the image portion of the glasses causes a missing portion in the image of the pedestrian's head, and in the determination based on the area and the fullness rate, it can be recognized that the processing target image portion is the image of the pedestrian's head. Even if it is lost, the image portion of the pedestrian's head can be extracted.

  In the present embodiment, an image of a pedestrian wearing glasses has been described. However, a similar process is performed on a pedestrian image wearing a cap, so that the image portion of the cap may be divided or missing by performing the same processing. It is possible to determine whether or not the pedestrian is an pedestrian by judging the characteristics of the head of the generated image portion of the pedestrian.

  Further, in the present embodiment, the case where there is a defect at a position above the central portion of the processing target image portion has been described as an example, but the top of the head is visible depending on the human face shape, the angle at the time of imaging, or the hair. Due to difficulties and the like, the position of the defect is not necessarily a location above the central part, and a defect may occur in the lower part of the central part.

  DESCRIPTION OF SYMBOLS 1 ... Image processing unit, 2R, 2L ... Infrared camera, 20 ... Processing object image part extraction means, 21 ... Pedestrian identification means, 70 ... Processing object image part, 71, 72 ... Divided to the upper limit by the image part of glasses Image portion of the head, 75, 76... Target image portion facing vertically in a predetermined interval, 80... Processing target image portion in which the interval portion is buried and integrated, 90. .

Claims (2)

A vehicle periphery monitoring device that recognizes an object to be monitored around the host vehicle based on an image captured by an in-vehicle infrared camera,
Binary image candidate extraction means for extracting a processing target image portion from a binary image generated by binarizing the grayscale image of the captured image;
Pedestrian identification for determining whether or not the processing target image portion has the characteristics of a pedestrian's head and identifying whether or not an object in the real sky corresponding to the processing target image portion is a pedestrian A vehicle periphery monitoring device comprising means,
The pedestrian identification unit includes a determination unit that determines whether or not two image portions facing each other in the vertical direction with an interval within a predetermined range are extracted as the processing target image portion , and within the predetermined range by the determination unit When it is determined that two image portions that are vertically opposed to each other are extracted, the two image portions are integrated by filling the interval portion, and the integrated image portion is the pedestrian's head. A vehicle periphery monitoring device that determines whether or not an object in real space corresponding to the two image portions is a pedestrian by determining whether or not it has a feature. A vehicle periphery monitoring device that recognizes an object to be monitored around the host vehicle based on an image captured by an in-vehicle infrared camera,
Binary image candidate extraction means for extracting a processing target image portion from a binary image generated by binarizing the grayscale image of the captured image;
Pedestrian identification for determining whether or not the processing target image portion has the characteristics of a pedestrian's head and identifying whether or not an object in the real sky corresponding to the processing target image portion is a pedestrian A vehicle periphery monitoring device comprising means,
The pedestrian identification means determines whether or not an image portion having a horizontally long missing portion surrounded by three high-intensity areas of a binary image at a substantially central portion is extracted as the processing target image portion. A determination unit, and when it is determined by the determination unit that an image portion having a horizontally long defect portion is extracted at a substantially central portion, the image portion in which the defect portion is filled A vehicle periphery monitoring device that determines whether or not the object in the real space corresponding to the image portion having the deficient portion is a pedestrian.