JP2007058805A - Forward environment recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately perform determination of a pedestrian even with change of the pedestrian's state or various environmental conditions. <P>SOLUTION: A control unit 3 performs detection of pedestrian by performing pattern matching processing to images taken by a pair of infrared cameras 4 using a plurality of preliminarily prepared model images. Specifically, when it is raining, or when an atmospheric temperature of a threshold or lower is determined, an object is determined to be the pedestrian if matching with the model image of pattern A is successful. When it is not raining, the atmospheric temperature is higher than the threshold, and the current time is out of a set time zone, the object is determined to be the pedestrian if matching with any one of the patterns A, C and E is successful. Further, when it is not raining, the outside air temperature is higher than the threshold, and the current time is within the set time zone, the object is determined to be the pedestrian if matching with any one of the patterns B, D and F is successful. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention particularly relates to a front environment recognition apparatus that recognizes a pedestrian by pattern matching processing from an image taken by a far-infrared camera or the like.

  In recent years, a driving support device has been developed and put into practical use, which captures the driving environment ahead using an on-board camera, etc., recognizes the preceding vehicle or pedestrian according to the driving environment status, and can perform warning and brake control for the driver. ing. In particular, in pedestrian recognition, a system has been proposed in which heat generated by a pedestrian is detected using a far-infrared camera and pedestrian recognition is performed with high accuracy by pattern matching processing.

For example, in Japanese Patent Application Laid-Open No. 2004-135034, from an image taken by a far-infrared camera, the center of gravity, area, distance from the host vehicle, aspect ratio of the circumscribed rectangle, height and width, and center of gravity of the object The binarized object shape feature amount in the real space is calculated using the coordinate value. Next, if the weather around the vehicle is not rainy, calculate the height of the object on the grayscale image, set multiple mask areas in the grayscale object, and average the brightness of each mask. Calculate the value and the change in brightness. Then, it is determined whether the height, width, existence height, luminance average value, luminance dispersion, etc. of the object are within the appropriate range for the pedestrian, and any of them is not appropriate for the pedestrian. In such a case, a technique for determining that the object is not a pedestrian is disclosed.
JP 2004-135034 A

  However, the shape of the heat generation area of the camera pedestrian captured by the far-infrared camera is various, and as in Patent Document 1 described above, walking is performed with the height, width, existence height, average luminance value, and luminance dispersion of the object. When it is determined whether or not the person is a person, there is a problem that the accuracy of the determination decreases. Moreover, although the method of pedestrian determination is changed in the case of rainy weather and other cases in Patent Document 1 described above, there is a problem that pedestrian determination is affected by factors other than rainy weather.

  The present invention has been made in view of the above circumstances, and provides a forward environment recognition device capable of accurately determining a pedestrian even if there are changes in the state of the pedestrian and various environmental conditions. Objective.

  The present invention detects a pedestrian by performing pattern matching processing using a heat source detection means for detecting a heat source in the front environment and a model image set in advance for the shape of the heat source detected by the heat source detection means. In the forward environment recognition device comprising pedestrian recognition means, the pedestrian recognition means presets the shape of the pedestrian itself and the shape of the pedestrian clothes as the model image. Yes.

  The forward environment recognition device according to the present invention can accurately determine a pedestrian even if the pedestrian state and various environmental conditions change.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1 to 6 show an embodiment of the present invention, FIG. 1 is a schematic configuration diagram of a driving support apparatus mounted on a vehicle, FIG. 2 is a flowchart of a model image selection program used for pattern matching processing, and FIG. FIG. 4 is an explanatory diagram showing an example of a model image of a pedestrian himself, FIG. 5 is an explanatory diagram of an example of a model image of a pedestrian's clothes, and FIG. It is explanatory drawing of an example of the model image of the clothes of a pedestrian different from FIG.

  In FIG. 1, reference numeral 1 denotes a vehicle (host vehicle) such as an automobile, and a vehicle driving support device 2 is mounted on the vehicle 1.

  The vehicle driving support apparatus 2 includes a control unit 3 that includes a pair of far-infrared cameras 4, a vehicle speed sensor 5 that detects vehicle speed, a raindrop sensor that detects whether the vehicle is raining, or a wiper switch 6 and a current time. A clock 7 for detection and an outside air temperature sensor 8 for detecting outside air temperature are connected. Note that the raindrop sensor or wiper switch 6, the clock 7, and the outside air temperature sensor 8 are provided as environmental information detection means.

  The vehicle driving support device 2 recognizes the front environment and extracts a three-dimensional object such as a pedestrian or a preceding vehicle, and a driver through a speaker 9 based on information obtained by the front environment recognition function. And an alarm function for voice warning.

  The pair of far-infrared cameras 4 are provided as heat source detection means, and are mounted as a stereo optical system at a certain distance in front of the ceiling in the vehicle interior, and take a stereo image of the front environment from different viewpoints and photograph the object outside the vehicle. The obtained image is output to the control unit 3. The far-infrared camera 4 has a characteristic that the output signal level becomes higher (the luminance increases) as the temperature of the subject to be photographed is higher.

  The processing of images from the pair of far infrared cameras 4 in the control unit 3 is performed as follows, for example. First, for a pair of stereo images of the environment in the traveling direction of the host vehicle 1 captured by a pair of far-infrared cameras 4, a process for obtaining distance information over the entire image from the corresponding positional deviation amount by the principle of triangulation. In line, a distance image representing a three-dimensional distance distribution is generated. Based on this data, a well-known grouping process or a pre-stored three-dimensional model image (road shape data, three-dimensional object data, etc.) is compared and a known pattern matching process is performed. Data, sidewall data such as guardrails and curbs that exist along the road, and three-dimensional object data such as pedestrians and vehicles are extracted.

  For a three-dimensional object, for example, as shown in FIG. 3, a circumscribed rectangle is formed on the grouped images. When a pedestrian is recognized, a circumscribed rectangle having a preset size range and a shape (aspect ratio, etc.) of the circumscribed rectangle is within a set range, A pedestrian is extracted from a three-dimensional object by comparing with a model image stored in advance and performing pattern matching processing.

  A plurality of model images used for the pattern matching process for extracting the pedestrian are prepared according to the front environment. For example, when the raindrop is detected by the raindrop sensor or the wiper switch 6 is turned on, or the outside air temperature by the outside air temperature sensor 8 is below a threshold (for example, 34 ° C. at a temperature close to body temperature). In this case, it is not raining, the outside temperature is larger than the threshold, and the current time measured by the clock 7 is a set time zone (for example, 6:00 to 18:00 in the daytime time zone). In addition, when it can measure until the date, it may be variably set every month and day))) In other cases, the pattern A is formed with high pedestrian brightness as shown in FIG. Model images are used.

  Also, when it is not in a rainy state, the outside air temperature is larger than the threshold value, and the current time measured by the clock 7 is within the set time zone, a pedestrian as shown in FIG. A model image of pattern B formed with low brightness is used.

  Further, when it is not raining, the outside air temperature is larger than the threshold value, and the current time measured by the clock 7 is outside the set time zone, a pedestrian as shown in FIG. A model image of the pattern C formed with high brightness of the clothes is used. Also, under this condition, a model image of a pattern E formed as shown in FIG. 6A, which is different from that in FIG.

  And when it is not in a rainy state, the outside air temperature is larger than the threshold value, and the current time measured by the clock 7 is outside the set time zone, it matches any of the patterns A, C, and E. If there is, it is determined to be a pedestrian.

  On the other hand, when it is not raining, the outside air temperature is larger than the threshold value, and the current time measured by the clock 7 is within the set time zone, a pedestrian as shown in FIG. The model image of the pattern D formed with the low brightness of the clothes is used. Also, under this condition, a model image of a pattern F formed as shown in FIG. 6B and having a low luminosity of pedestrian clothing different from that in FIG. 5B is used.

  And when it is not in a rainy state, the outside air temperature is larger than the threshold value, and the current time measured by the clock 7 is within the set time zone, it matches with any of the patterns B, D, and F. If there is, it is determined to be a pedestrian.

  Thus, the front environment recognition function of the control unit 3 has a function as a pedestrian recognition means. In this embodiment, the two images shown in FIGS. 5 and 6 are prepared as model images relating to the clothes of the pedestrian. However, only one of them may be used, and a model image of clothes depending on the season. May be used properly. Further, a plurality of model images may be prepared.

  When the pedestrian is recognized as described above, the alarm function in the control unit 3 is an audio alarm such as “Please beware of the pedestrian in front” when the pedestrian is present in a preset area in front of the host vehicle. Is emitted from the speaker 9. This area is variably set according to the vehicle speed, for example, and a larger area is set forward as the vehicle speed increases.

Next, a model image selection program used for the pattern matching process executed by the control unit 3 will be described with reference to the flowchart of FIG.
First, in step (hereinafter abbreviated as “S”) 101, it is determined whether or not there is rain with reference to a signal from the raindrop sensor or the wiper switch 6. If it is determined that there is rain as a result of this determination, the process proceeds to S102, and it is determined to apply a model image of the pattern A formed with high brightness of the pedestrian as shown in FIG. , Exit the program. That is, since the temperature during rainfall is generally lower than the body temperature of the person, the brightness of the person is higher than the brightness of the background. Therefore, when it is determined that it is raining, the model image of pattern A is selected as the model image to be applied, and pattern matching processing is performed.

  If it is determined in S101 that there is no rain, the process proceeds to S103, and it is determined whether or not the outside air temperature TA by the outside air temperature sensor 8 is equal to or lower than a threshold Tc (for example, 34 ° C. at a temperature close to body temperature). As a result of this determination, when the outside air temperature TA is equal to or lower than the threshold value Tc, the reason is the same as described above, that is, the luminance of the person is higher than the luminance of the background, so that the process proceeds to S102, as shown in FIG. It decides to apply the model image of the pattern A formed with high brightness of the pedestrian himself, and exits the program.

  On the other hand, if the outside air temperature TA is higher than the threshold value Tc as a result of the determination in S103, the process proceeds to S104, where the current time measured by the clock 7 is a set time zone (for example, 6:00:00 in the daytime time zone). 18:00).

  As a result of the determination in S104, if the current time is within the set time zone, the process proceeds to S105, and pattern matching processing is performed using the patterns B, D, and F as model images, and the pattern B, D, and F are matched. If there is, it is determined that the person is a pedestrian and the program is exited. That is, in this case, since the brightness of the background and the brightness of the person are almost the same, it is difficult for the person to recognize the model image only with the patterns A and B. Therefore, since clothes do not generate heat like humans and are difficult to store heat, the brightness tends to be lower than that of people and backgrounds. Also, in the daytime hours of 6:00 to 18:00, the road surface and the wall surface of the building are likely to be warmed by the solar heat, and the background luminance is likely to increase. Therefore, patterns B, D, and F are used as model images. If a matching process is performed and it matches with any of pattern B, D, and F, it will determine with it being a pedestrian.

  On the other hand, as a result of the determination in S104 described above, if the current time is outside the set time zone, the process proceeds to S106, and pattern matching processing is performed using the patterns A, C, and E as model images, and any of the patterns A, C, and E is performed. If they match, it is determined that the person is a pedestrian and the program is exited. That is, in this situation, contrary to the reason described in S105 above, the background luminance tends to decrease, so that pattern matching processing is performed using the patterns A, C, and E as model images, and the patterns A, C, and E are performed. If it matches any of these, it will determine with it being a pedestrian.

  As described above, according to the embodiment of the present invention, not only a model image of the pedestrian itself but also a model image of the shape of the pedestrian's clothes is prepared. Pedestrian determination is performed by properly using according to time and performing pattern matching processing. Therefore, even if there is a change in the state of the pedestrian and various environmental conditions, the pedestrian can be accurately determined.

  In the present embodiment, the control unit 3 of the vehicle driving support apparatus 2 is described as an example having an alarm function. However, a collision avoidance function for automatically applying brakes to avoid a collision, etc. It may have.

  Further, the position where the far-infrared camera 4 is disposed is not limited to the present embodiment, and may be disposed, for example, in a vehicle bumper.

  Further, in the present embodiment, the distance to the three-dimensional object is obtained by the pair of far infrared cameras 4, but the distance is obtained by a normal stereo camera (CCD camera), and only the temperature distribution is obtained by the far infrared rays. Needless to say, the present invention can also be applied to a system in which images are synthesized by a camera.

Schematic configuration diagram of a driving support device mounted on a vehicle Flowchart of model image selection program used for pattern matching processing Explanatory drawing showing an example of an image taken with a far-infrared camera Illustration of an example of a pedestrian's own model image Illustration of an example of a model image of a pedestrian's clothes Explanatory drawing of an example of the model image of the clothes of the pedestrian different from FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Own vehicle 2 Driving support apparatus for vehicles 3 Control unit (pedestrian recognition means)
4 Far-infrared camera (heat source detection means)
6 Raindrop sensor or wiper switch (environmental information detection means)
7 Clock (environmental information detection means)
8 Outside air temperature sensor (environmental information detection means)

Claims (5)

Heat source detection means for detecting a heat source in the front environment;
In the front environment recognition device including a pedestrian recognition unit that performs pattern matching processing using a model image set in advance with respect to the shape of the heat source detected by the heat source detection unit, and detects a pedestrian,
The pedestrian recognizing device is characterized in that the shape of the pedestrian and the shape of the clothes of the pedestrian are set in advance as the model image. Environmental information detection means for detecting environmental information of the front environment,
The front environment recognition apparatus according to claim 1, wherein the model image used by the pedestrian recognition means uses a different model image according to the environment information. The environmental information detection means detects at least the outside air temperature,
The front environment recognition apparatus according to claim 2, wherein the model image used by the pedestrian recognition means uses a model image that differs according to at least the outside air temperature. The environmental information detection means detects at least the presence or absence of rainfall,
4. The front environment recognition device according to claim 2, wherein the model image used by the pedestrian recognition means uses a model image that differs depending on at least the presence or absence of the rain. The environmental information detection means detects at least the current time,
The front environment recognition apparatus according to any one of claims 2 to 4, wherein the model image used by the pedestrian recognition means uses a model image that differs according to at least the current time.

Images