JP2007133644A - Pedestrian recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To decide necessity of alarm issuing with a high reliability degree when recognizing a pedestrian near one's own vehicle and issuing an alarm to a driver. <P>SOLUTION: The alarm issuing to the driver is controlled on the basis of an estimation result of a course estimation means and detection results of a view line direction detection means and a pedestrian detection means. For example, when detecting the pedestrian and when a detection position thereof accords with a course of the vehicle and discords with a view line direction, an alarm level is set as a maximum level. When detecting the pedestrian and when the detection position accords with the course of the vehicle and accords with the view line direction, when the detection position discords with the course of the vehicle and discords with the view line direction, or when the detection position discords with the course of the vehicle and accords with the view line direction, the alarm level is set as a level lower than the maximum level. When not detecting the pedestrian or when the own vehicle stops, the alarm is not performed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention is used for protecting non-occupants (pedestrians and the like) of a vehicle. It is particularly suitable for use in large vehicles such as trucks and buses.

  It goes without saying that the driver of the vehicle must always pay attention to the surrounding situation and try to drive safely. However, there are various factors that hinder the driver's field of view, such as nighttime and rainy weather, and it is undeniable that there is a limit to the complete understanding of the surrounding situation by human ability alone.

  In order to compensate for such human limitations, proposals have been made to borrow the power of electronic equipment. For example, in Patent Document 1, a pedestrian in front of the vehicle is detected using an infrared sensor.

JP 11-353593 A Japanese Patent Laid-Open No. 6-319701

  By detecting a pedestrian or the like with the help of an electronic device, the detection accuracy of a pedestrian can be improved as compared with a case where only a human ability is used. However, on the other hand, the detected pedestrian has not yet made a determination as to whether or not he / she really needs attention when traveling the vehicle.

  For this reason, even when a pedestrian does not need to pay attention to the traveling of the host vehicle, warnings are frequently issued, leading to a decrease in reliability with respect to driver warnings. As a result, it is expected that the driver will ignore the alarm or cause the alarm device to be turned off in advance.

  The present invention has been made under such a background, and an object thereof is to provide a pedestrian recognition device capable of determining the necessity of issuing an alarm with high accuracy.

  The present invention provides pedestrian detection means, eye direction detection means for detecting the direction of the driver's eyes, course estimation means for estimating the course of the vehicle, detection results of the pedestrian detection means and the eye direction detection means. And a pedestrian recognition device comprising warning control means for controlling the issuing of a warning to the driver based on the estimation result of the course estimation means.

  That is, the pedestrian recognition device of the present invention does not issue an alarm simply because a pedestrian is detected, but recognizes the relationship between the detected pedestrian and the host vehicle, and there is a need for an alarm. It is characterized by issuing an alarm only in the case.

  In the present invention, alarm control is performed based on the position of the pedestrian, the course of the own vehicle, and the direction of the driver's line of sight. For example, even if the position of the pedestrian matches the course of the own vehicle, When the direction of the line of sight is toward the pedestrian, it is possible to refrain from issuing a strong warning, and it is possible to avoid discomforting the driver. Thereby, the reliability with respect to a driver | operator's warning can be improved, and the effect of a warning can be heightened by extension.

  More specifically, the alarm control means detects the pedestrian by the pedestrian detection means, the detected position coincides with the course of the vehicle estimated by the course estimation means, and is detected by the eye direction detection means. When the sight line direction does not match, the warning level is set to the maximum level, the pedestrian detection means detects the pedestrian, the detection position coincides with the course of the vehicle estimated by the course estimation means, and the sight line direction When the eye direction coincides with the eye direction detected by the detecting means, or the pedestrian detecting means detects a pedestrian, and the detected position is inconsistent with the course of the vehicle estimated by the course estimating means, and the eye direction When it does not coincide with the eye direction detected by the detecting means, or the pedestrian detecting means detects a pedestrian, and the detected position is estimated by the course estimating means. When both the paths are inconsistent and coincides with the line-of-sight direction detected by the line-of-sight direction detection means, the alarm level is set lower than the maximum level, and the pedestrian detection means does not detect a pedestrian, or When the host vehicle is stopped, it is possible to provide means for controlling the issuance of an alarm not to perform an alarm.

  The detection position of the pedestrian here is not only a fixed position at the moment of detection, but also the position where the pedestrian is estimated to move from now on by detecting the moving speed and moving direction of the pedestrian. It can also be a dynamic position including.

  The pedestrian detection means includes an infrared camera device that captures an infrared image, a thermograph camera device that detects a temperature distribution of a subject, and a pedestrian from images captured by the infrared camera device and the thermograph camera device. A pedestrian shape recognition means for recognizing the shape of the pedestrian, a temperature distribution detection means for detecting a temperature distribution in an image estimated as a pedestrian recognized by the pedestrian shape recognition means, and a temperature distribution detection means When the temperature distribution does not match the temperature distribution of a person, a pedestrian image determination unit that determines that there is no pedestrian can be provided.

  According to this, a human-shaped object and a real human can be distinguished. For example, when a billboard in the shape of a person is placed on the shoulder of a store, it can be distinguished from a person and the driver's unnecessary stress can be reduced.

  Actually, if an object that is not a person is present at a position that obstructs the course of the host vehicle, it is also necessary to warn of this, but this is a function different from the function provided by the present invention. Since such a function has been widely used in the past, the alarm function for an object other than a person is excluded here.

  The pedestrian image determining means determines that there is a pedestrian when the pedestrian shape recognizing means recognizes the shape of a pedestrian that is not photographed by the infrared camera device but is photographed only by the thermograph camera device. Means may be provided.

  According to this, since it is possible to detect a person who is hidden behind an object such as implantation and is difficult to be photographed by the infrared camera device, the pedestrian is simply compared with the case where the pedestrian is detected by the infrared camera device. Detection accuracy can be improved.

  An infrared sensor can be used as the pedestrian detection means. According to this, although the figure of a pedestrian cannot be projected on the screen of the display part of a driver's seat, the pedestrian recognition device of the present invention can be realized with an inexpensive and simple configuration.

  According to the present invention, since it is possible to determine the necessity of issuing an alarm with high accuracy, it is possible to improve the reliability of the driver's alarm and thus to increase the effect of the alarm. In addition, it is possible to reduce stress on the driver due to unnecessary warnings.

  A pedestrian recognition apparatus according to an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram of the pedestrian recognition device of this embodiment. FIG. 2 is an overall configuration diagram of the pedestrian recognition device of the present embodiment. FIG. 3 is a diagram showing the relationship between the position of the pedestrian and the direction of the line of sight and the arrangement of the infrared camera device, the thermograph camera device, and the projector. FIG. 4 is a flowchart showing an overall operation procedure of the pedestrian recognition apparatus of the present embodiment. FIG. 5 is a flowchart showing a pedestrian recognition procedure of the pedestrian recognition apparatus of the present embodiment.

  As shown in FIGS. 1 and 2, the pedestrian recognition apparatus according to the present embodiment includes infrared camera apparatuses 1-1, 1-2, 1-3 that take infrared images to detect pedestrians, and the temperature of a subject. Thermograph camera devices 2-1, 2-2, 2-3 for detecting the distribution, a pedestrian detection unit 10 mounted in the alarm control unit 6, a line-of-sight camera device 3 for detecting the direction of the driver's line of sight, and Detection of the eye direction detection unit 11 implemented in the alarm control unit 6, the course estimation unit 12 implemented in the alarm control unit 6 for estimating the course of the vehicle, the detection of the pedestrian detection unit 10 and the eye direction detection unit 11 And a warning control unit 6 that controls a warning issued to the driver based on the result and the estimation result of the course estimation unit 12.

  The projectors 8-1, 8-2, and 8-3 are infrared projectors, and are mainly used at night. The eye camera device 3 uses, for example, a line-of-sight recognition device disclosed in Patent Document 2. The line-of-sight recognition device disclosed in Patent Document 2 is an apparatus that can recognize a line of sight (line of sight) stably and accurately without mounting a device on the head.

  The alarm control unit 6 detects the pedestrian by the pedestrian detection unit 10, and the detected position coincides with the course of the vehicle estimated by the course estimation unit 12, and does not coincide with the eye direction detected by the eye direction detection unit 11. Is set to the maximum level, the pedestrian detection unit 10 detects the pedestrian, the detected position coincides with the course of the vehicle estimated by the course estimation unit 12, and is detected by the eye direction detection unit 11. Or when the pedestrian detection unit 10 detects a pedestrian and the detected position does not match the vehicle path estimated by the path estimation unit 12, and is detected by the line-of-sight direction detection unit 11. Or the pedestrian detection unit 10 detects a pedestrian, and the detected position is inconsistent with the course of the vehicle estimated by the course estimation unit 12, and the eye direction detection unit 1 When the direction of the line of sight is detected, the alarm level is set to a level lower than the maximum level, and when the pedestrian detection unit 10 does not detect a pedestrian, or based on the vehicle speed information from the vehicle speed sensor 5, When it can be determined that the vehicle is stopped, a means for controlling the issuing of an alarm is provided so that no alarm is issued.

  In addition, the pedestrian detection unit 10 determines the shape of the pedestrian from the images taken by the infrared camera devices 1-1, 1-2, 1-2 and the thermograph camera devices 2-1, 2-2, 2-3. A pedestrian shape recognition unit 20 that recognizes, a temperature distribution detection unit 21 that detects a temperature distribution in an image estimated as a pedestrian recognized by the pedestrian shape recognition unit 20, and a temperature distribution detection unit 21 that detects the temperature distribution. A pedestrian image determination unit 22 that determines that there is no pedestrian when the temperature distribution does not match the temperature distribution of a person.

  Further, the pedestrian image determination unit 22 is not captured by the infrared camera devices 1-1, 1-2, 1-3 and the thermograph camera devices 2-1, 2-2, 2-2. When the shape of the pedestrian captured only in 3 is recognized, a means for determining that there is a pedestrian is provided.

  Next, the operation of the pedestrian recognition device of this embodiment will be described with reference to the flowcharts of FIGS. FIG. 4 is an overall procedure for alarm control, and FIG. 5 is a procedure for detecting the presence / absence of a pedestrian and generating angle and distance information for the pedestrian.

  As shown in FIG. 4, the alarm control unit 6 executes a pedestrian detection process, an estimated course calculation process, and a line-of-sight direction calculation process almost simultaneously in parallel. In the pedestrian detection process, the process shown in FIG. 5 (steps S11 to S18) is executed as step S1.

  As shown in FIG. 5, the pedestrian detection unit 10 inputs images from the infrared camera devices 1-1, 1-2, 1-3 and the thermograph camera devices 2-1, 2-2, 2-3 ( S11, S12), the pedestrian shape is recognized (S13, S14). A pedestrian shape pattern stored in advance is used for the recognition of the pedestrian shape. Since such a shape recognition technique is already widely used, a detailed description thereof will be omitted.

  Subsequently, a pedestrian presence determination (S15) is performed. (1) When both the infrared camera devices 1-1, 1-2, 1-3 and the thermographic camera devices 2-1, 2-2, 2-3 photograph a subject that can be recognized as a pedestrian, It is determined that there are pedestrians. (2) Although the infrared camera devices 1-1, 1-2, and 1-3 are shooting subjects that can be recognized as pedestrians, the thermograph camera devices 2-1, 2-2, and 2-3 are recognized as pedestrians. If the subject that can be taken is not photographed, the subject photographed by the infrared camera devices 1-1, 1-2, and 1-3 is an object resembling a human figure and is not a pedestrian. Judge that there is no. (3) Although the infrared camera devices 1-1, 1-2, and 1-3 do not photograph a subject that can be recognized as a pedestrian, the thermograph camera devices 2-1, 2-2, and 2-3 recognize as a pedestrian. When a photographable subject is being photographed, it is considered that a pedestrian is hidden behind the object, so it is determined that there is a pedestrian. (4) When both the infrared camera devices 1-1, 1-2, and 1-3 and the thermographic camera devices 2-1, 2-2, and 2-3 have not photographed a subject that can be recognized as a pedestrian. Determines that there is no pedestrian.

  When it is recognized that there is a pedestrian based on the determination result in step S15 (S16), information on the angle and distance of the host vehicle with respect to the pedestrian is generated (S17). The information of the angle of the own vehicle with respect to the pedestrian is based on the position of the pedestrian in the field of view of the infrared camera device 1-1, 1-2, 1-3 or the thermograph camera device 2-1, 2-2, 2-3. It can be generated by recognition.

  Further, the information on the distance between the pedestrian and the own vehicle may be generated by separately providing a distance measuring device and measuring the distance between the pedestrian and the own vehicle, or an infrared camera. The distance between the pedestrian and the host vehicle can be obtained by making the device 1-1, 1-2, 1-3 or the thermograph camera device 2-1, 2-2, 2-3 as a camera device capable of taking a stereo image. , And generation of distance information can be realized. Since various methods have already been widely used for the distance measurement method, an appropriate conventional method is used, and detailed description thereof is omitted.

  FIG. 6 illustrates a state in which a pedestrian and a human-shaped sign are displayed separately. In the example of FIG. 6, the pedestrian and the sign in the shape of a person are distinguished by the procedure (2) of step S15, and only the pedestrian is displayed surrounded by a square bright line, and the driver is informed of the presence of the pedestrian. Can be recognized.

  Based on the angle and distance information of the host vehicle with respect to the pedestrian generated in step S17, the presence / absence of the pedestrian and the pedestrian position are calculated in step S2 of FIG. As described above, the position of the pedestrian is not only the fixed position at the moment when the pedestrian is detected, but also the movement speed and direction of the pedestrian, and the pedestrian is estimated to move from now on. It can also be a dynamic position including the position. FIG. 7 shows how pedestrian movement prediction information is displayed. In the example of FIG. 7, the pedestrian travel direction and the predicted movement distance are displayed by arrows. Such movement prediction can be realized by performing the procedure of FIG. 5 twice or more with a time difference.

  The course estimation unit 12 takes in the steering information from the steering sensor 4 or the yaw rate sensor 9 and the vehicle speed information from the vehicle speed sensor 5 (S3), and calculates the estimated course (S4).

  The line-of-sight detection unit 11 captures the driver image (S5) and calculates the direction of the driver's line of sight (S6).

  Subsequently, an alarm level calculation (S7) is performed. That is, (a) if there is a pedestrian and the estimated course of the vehicle coincides with the direction of the pedestrian, but the driver's eye direction also coincides with the direction of the pedestrian, the driver walks It seems that they are aware of the presence of the person, so a weak warning is given. The weak alarm is, for example, only displaying a bright line surrounding the pedestrian as shown in FIG. 6 or 7 on the screen displaying the pedestrian image, and no alarm sound or voice alarm is given.

  (B) If there is a pedestrian and the driver's line of sight is not facing the direction of the pedestrian even though the estimated course of the vehicle matches the direction of the pedestrian, Because it is not recognized and the vehicle is heading towards a pedestrian, it is considered a very dangerous situation, so a strong warning is given. The strong alarm is, for example, on the screen displaying the pedestrian image, flashing the bright line surrounding the pedestrian as shown in FIG. 6 or FIG. If an automatic braking control device is installed, it is activated and automatically stopped.

  (C) Although there are pedestrians, the estimated course of the vehicle is different from the direction of the pedestrians. The driver ’s line of sight does not point toward the pedestrian, and the driver may not recognize the presence of the pedestrian, but it is dangerous because the estimated course of the vehicle is not toward the pedestrian. It is thought that there are few. Therefore, a weak alarm is given.

  (D) Although there are pedestrians, the estimated course of the vehicle is different from the direction of the pedestrians. Further, the driver's line of sight faces the direction of the pedestrian, and it is considered that the driver recognizes the presence of the pedestrian, and there is little danger. Therefore, a weak alarm is given.

  (E) Since there is no pedestrian, the estimated course calculation and the eye direction calculation are not performed, and no warning is given.

  (F) Since the vehicle is stopped, no warning is given regardless of the presence or absence of pedestrians.

  FIG. 3 shows the relationship between the position of the pedestrian and the direction of the line of sight, infrared camera devices 1-1, 1-2, 1-3, thermographic camera devices 2-1, 2-2, 2-3, and projector 8-1. 8-2 and 8-3 are illustrated. The infrared camera device 1-1 and the thermograph camera device 2-1 are in front of the vehicle. The infrared camera device 1-2 and the thermograph camera device 2-2 show the rear left side of the vehicle. The infrared camera device 1-3 and the thermograph camera device 2-3 show the right rear of the vehicle.

  Thereby, it is possible to detect pedestrians in the front and left and right rear of the vehicle. Since the visibility of the left and right sides of the driver's seat is good, the pedestrian is recognized only by the driver's visual observation. Of course, it goes without saying that the left and right sides of the driver's seat can be set as the monitoring target area of the pedestrian recognition device of the present invention by increasing the number of infrared camera devices, thermographic camera devices, and projectors.

  The driver directly looks at the pedestrians # 1 to # 3, but the driver looks at the pedestrians # 4 and # 5 through the fender mirror 15. Therefore, whether or not the driver recognizes the pedestrians # 4 and # 5 is determined based on whether or not the direction of the line of sight faces the fender mirror 15.

(Other examples)
Another embodiment will be described with reference to FIG. FIG. 8 is a diagram showing an arrangement state of infrared sensors in other embodiments. As shown in FIG. 8, the pedestrian recognition apparatus that can determine the necessity of issuing an alarm with high accuracy is realized by using the infrared sensors 13-1 to 13-7 as in the above-described embodiment. it can.

  That is, the present embodiment has a configuration in which the infrared camera devices 1-1 to 1-3 in the above-described embodiment are replaced with infrared sensors 13-1 to 13-7, and the projectors 8-1 to 8-3 and the thermograph. The camera devices 2-1 to 2-3 are not used. Therefore, as shown in FIG. 6 or FIG. 7, a pedestrian cannot be projected on the screen of the driver's seat display unit, but the walking for judging the necessity of issuing an alarm which is the object of the present invention with high accuracy. The person recognition device can be realized with an inexpensive and simple configuration as compared with the above-described embodiment.

  That is, as shown in FIG. 8, the infrared sensor 13-1 is responsible for detection of a pedestrian in the area where the pedestrian # 1 exists (the vehicle left front), and the infrared sensor 13-2 indicates the presence of the pedestrian # 2. It is responsible for pedestrian detection in the area (front of the vehicle center), and is responsible for detection of pedestrians in the area where the pedestrian # 3 exists (right front of the vehicle) by the infrared sensor 13-3, and by infrared sensors 13-4 and 13-6. It is responsible for pedestrian detection in the area where pedestrian # 4 exists (left side of the vehicle), and is responsible for detection of pedestrian in the area where pedestrian # 5 exists (right side of the vehicle) by infrared sensors 13-5 and 13-7.

  In addition, when the directivity which the infrared sensors 13-1 to 13-7 originally has is dull, the desired directivity can be obtained by arranging the light shielding plate 14. If the detection distances of the infrared sensors 13-4 and 13-5 are sufficiently long and pedestrian detection on the side of the vehicle can be performed with only one infrared sensor, the infrared sensors 13-6 and 13-7 are omitted. Also good.

  Thus, by using the infrared sensors 13-1 to 13-7, the presence / absence of a pedestrian and the angle information of the host vehicle with respect to the pedestrian can be generated. Moreover, since the distance that the infrared sensors 13-1 to 13-7 can detect a pedestrian is limited (for example, 5m to 10m), it is not necessary to separately measure the distance between the host vehicle and the pedestrian. Even if the procedure of the flowchart of FIG. 4 is executed only by the angle information of the own vehicle with respect to the pedestrian, there is no problem. In this case, the distance measuring means used in the above-described embodiment can be omitted.

  Accordingly, the procedure of the flowchart of FIG. 4 is also executed in this embodiment except that the method of generating the angle and distance information of the host vehicle with respect to the pedestrian in step S1 of the flowchart shown in FIG. 4 is different from the above-described embodiment. be able to.

  According to the present invention, since it is possible to determine the necessity of issuing an alarm with high accuracy, it is possible to improve the reliability of the driver's alarm and thus to increase the effect of the alarm. In addition, it is possible to reduce stress on the driver due to unnecessary warnings. Thereby, it can contribute to traffic safety.

The block block diagram of the pedestrian recognition apparatus of a present Example. The whole block diagram of the pedestrian recognition device of a present Example. The figure which shows the relationship between the position of a pedestrian and the direction of eyes, and the arrangement | positioning condition of an infrared camera device, a thermograph camera device, and a projector. The flowchart which shows the whole operation | movement procedure of the pedestrian recognition apparatus of a present Example. The flowchart which shows the pedestrian recognition procedure of the pedestrian recognition apparatus of a present Example. The figure which shows a mode that a pedestrian and the signboard in the shape of a person are distinguished and displayed. The figure which shows a mode that the movement prediction information of a pedestrian is displayed. The figure which shows the arrangement | positioning condition of the infrared sensor in another Example.

Explanation of symbols

1-1 to 1-3 Infrared camera device 2-1 to 2-3 Thermograph camera device 3 Eye camera device 4 Steering sensor 5 Vehicle speed sensor 6 Alarm control unit 7 Display unit 8-1 to 8-3 Projector 9 Yaw rate sensor 10 Pedestrian detection unit 11 Eye direction detection unit 12 Path estimation unit 13-1 to 13-7 Infrared sensor 14 Light shielding plate 15 Fender mirror 20 Pedestrian shape recognition unit 21 Temperature distribution detection unit 22 Pedestrian image determination units # 1 to # 5 Pedestrian

Claims (5)

Pedestrian detection means;
Eye direction detection means for detecting the direction of the driver's eye;
Course estimation means for estimating the course of the vehicle;
A pedestrian recognition device comprising: an alarm control unit that controls a warning issued to a driver based on a detection result of the pedestrian detection unit and the eye direction detection unit and an estimation result of the course estimation unit. The alarm control means includes
When the pedestrian detection means detects a pedestrian, and the detected position coincides with the course of the vehicle estimated by the course estimation means and does not coincide with the line-of-sight direction detected by the line-of-sight direction detection means, an alarm level Is the maximum level,
When the pedestrian detection means detects a pedestrian, and the detection position coincides with the course of the vehicle estimated by the course estimation means, and coincides with the eye direction detected by the eye direction detection means, or When the pedestrian detection means detects a pedestrian, and the detection position is inconsistent with the course of the vehicle estimated by the course estimation means, and is inconsistent with the eye direction detected by the eye direction detection means, or When the pedestrian detection means detects a pedestrian and the detected position is inconsistent with the course of the vehicle estimated by the course estimation means and coincides with the eye direction detected by the eye direction detection means, an alarm level Is lower than the maximum level,
The pedestrian recognition device according to claim 1, further comprising means for controlling the issuing of an alarm so as not to issue an alarm when the pedestrian detection means does not detect a pedestrian or when the host vehicle is stopped. The pedestrian detection means includes
An infrared camera device for taking infrared images;
A thermographic camera device for detecting the temperature distribution of the subject;
Pedestrian shape recognition means for recognizing the shape of a pedestrian from images taken by the infrared camera device and the thermograph camera device;
A temperature distribution detecting means for detecting a temperature distribution in an image estimated as a pedestrian recognized by the pedestrian shape recognizing means;
The pedestrian recognition device according to claim 1 or 2, further comprising: a pedestrian image determination unit that determines that there is no pedestrian when the temperature distribution detected by the temperature distribution detection unit does not coincide with a temperature distribution of a person.   The pedestrian image determining means is means for determining that there is a pedestrian when the pedestrian shape recognizing means recognizes the shape of a pedestrian photographed only by the thermographic camera device but not photographed by the infrared camera device. The pedestrian recognition device according to claim 3 provided.   The pedestrian recognition device according to claim 1, wherein the pedestrian detection means is an infrared sensor.

Images