JP4683192B2 - Vehicle blind spot monitoring device and vehicle driving support system - Google Patents

Description

  The present invention relates to a vehicle blind spot monitoring device that monitors a blind spot area outside the vehicle, which is a blind spot from the driver's field of view, and gives a warning that an obstacle is present, and a vehicle that restricts the traveling operation of the vehicle in accordance with the warning. The present invention relates to a driving support system.

  Technology that recognizes obstacles in the blind spot of a vehicle and notifies the driver is attracting attention as an important technology for eliminating traffic accidents. In particular, in the case of a vehicle with a high vehicle height such as a large truck, the driver's blind spot is likely to be in front of the vehicle, and if there is an obstacle such as a person or object, the driver will not notice it and cause a traffic accident. May end up.

JP 2000-185597 A

  In order to eliminate such blind spots, for example, a truck may be provided with a convex mirror that looks just before the vehicle at the same site as the side mirror. However, the scene where a pedestrian or the like enters the driver's blind spot is likely to occur when the vehicle is in a low speed or stopped state. It is easy to neglect the confirmation of the convex mirror, and even if a pedestrian or the like is reflected on the convex mirror, it may be overlooked and cause an accident.

  Patent Document 1 can be cited as a related technique for monitoring the blind spot ahead of the vehicle. Patent Document 1 discloses a technique in which a camera for photographing the front is mounted on a side mirror, and the front image is displayed on a monitor having a larger vertical length in the vehicle than the left and right to notify the driver. However, this technology only displays the image taken by the camera on the monitor, and even if the figure of a pedestrian etc. is displayed on the monitor, it is ultimately the driver that confirms it, so a mirror is provided. If the driver fails to confirm as in the case of accidents, an accident may occur. Moreover, even if the figure of a pedestrian, etc. is displayed on the monitor, the driver may not be able to instantly figure out where it is in relation to the vehicle, which may lead to a delay in driving operation. Therefore, it is not the best way to display blind spot information.

  The present invention has been made in view of the above-described problem, and a vehicle blind spot monitoring device capable of surely letting a driver know an obstacle present in a blind spot area, and a vehicle capable of safer driving. The purpose is to provide a driving support system.

Means for Solving the Problems and Effects of the Invention

In order to solve the above-described problem, a vehicle blind spot monitoring device related to the present invention is based on a vehicle exterior imaging unit that captures a blind spot area immediately before a vehicle outside the vehicle that is a blind spot from the driver's field of view, and an image obtained by the vehicle exterior imaging unit Obstacle recognition means for recognizing an obstacle existing in the blind spot area immediately before the vehicle and its position, driver imaging means for photographing the driver's eyes, and detecting the driver's line of sight from images obtained by the driver imaging means A visual line detection means, a display means for displaying an image on the windshield of the vehicle, and a position near the driver's visual line in the windshield when an obstacle is recognized in the blind spot area by the obstacle recognition means Display control means for displaying a warning to the effect that an obstacle exists on the display means, and the obstacle recognition means and the line-of-sight detection means are configured so that the vehicle is below a predetermined speed. It Ru is assumed to perform obstacle recognition processing and line-of-sight detection processing.

According to such an invention, a blind spot area outside the vehicle, which is a blind spot from the driver's field of view, is captured to recognize the obstacle, and the driver's line of sight is detected to warn that there is an obstacle in the windshield. By displaying in the vicinity of the position hit on the driver's line of sight, the driver can be surely informed of the obstacle present in the blind spot area. Further, the danger that a pedestrian or the like appears in the blind spot area immediately before the vehicle occurs when the vehicle is in a low speed state or a stopped state, and thus the vehicle is below a predetermined speed (for example, 40 km / h or less: 0). If the obstacle recognition process and the line-of-sight detection process are performed, a warning is effectively issued in the scene with the highest risk of moving out of the stop state. Therefore, it is possible to prevent traffic accidents.

In the vehicle blind spot monitoring device related to the present invention includes an obstacle direction calculation means for calculating the relative position of the obstacle relative to the line of sight of luck Utatete, display control means, the relative direction as warnings Can be configured to display. According to this, since the relative direction of the obstacle position relative to the driver's line of sight is displayed in the warning that there is an obstacle, the driver will not only have the obstacle, but also the obstacle in relation to the vehicle. The position of can be grasped instantly.

In the vehicle blind spot monitoring device related to the present invention, the table示制control means may be configured to display an image of the obstacle obtained by the outside imaging unit as a warning. According to this, since the image of the obstacle is displayed in the warning that there is an obstacle, the driver can instantly grasp the obstacle as well as the presence of the obstacle.

In the vehicle blind spot monitoring device related to the present invention, the table示制control means, when the obstacle recognition means recognizes a plurality of obstacles may be configured so as to display a warning corresponding to each of the obstacles. According to this, when a plurality of obstacles exist, a warning corresponding to each obstacle is displayed, so that the driver can appropriately grasp the existence of each obstacle.

In the vehicle blind spot monitoring device related to the present invention includes a gaze behavior measuring means for measuring the movement of the line of sight of luck Utatete, display control means, the frequency facing the gaze direction the line of sight of the driver a predetermined reference In the case where the frequency is lower than the frequency, the warning that the visual recognition is insufficient is displayed on the display means in the vicinity of the position on the driver's line of sight on the windshield. According to this, it is possible to prevent traffic accidents by warning the driver when the visibility is insufficient regardless of the presence or absence of obstacles, and to ensure that the driver does not neglect visual recognition. It will be glazed.

The vehicle blind spot monitoring device according to the present invention includes a driver imaging unit that captures a driver's eyes, a gaze detection unit that detects a driver's gaze from an image obtained by the driver imaging unit, and a movement of the driver's gaze. In the windshield when the frequency of the driver's line of sight facing a predetermined gaze direction is less than the reference frequency. Display control means for displaying on the display means a warning that the visual recognition is insufficient in the vicinity of the position hitting the driver's line of sight, the reference frequency is set according to the vehicle speed, and the vehicle travel information The gaze direction is set according to the travel information of the vehicle, and the gaze direction confirms a location closer to the vehicle body as the vehicle goes to a stop state. Characterized in that it is set to. According to this, the driver is warned when the visual recognition is insufficient. In particular, the gaze direction is set so as to confirm a portion closer to the vehicle body as the vehicle speed is lower, and the number thereof is also set higher. Therefore, when the vehicle in which a danger such as a pedestrian appears in the blind spot area immediately before the vehicle is in a low speed state or a stopped state (the vehicle is below a predetermined speed (for example, 40 km / h or less: 0 is included)), specifically Is effectively warned in the most dangerous scene of moving from a stopped state. Therefore, it is possible to prevent a traffic accident, and the driver can be trained so as not to neglect visual recognition.

In the vehicle blind spot monitoring system of the present invention includes a traveling information acquiring means for acquiring traveling information of vehicles, gaze direction can be configured to be set in accordance with the travel information of the vehicle. According to this, it is possible to make the driver check an appropriate gaze point according to the traveling state of the vehicle.

Next, to solve the above problems, the vehicle driving assist system of the present invention, includes a vehicle dual blind spot monitoring system, and a driving control device for limiting the travel operation of the vehicle when the display control means to display a warning It is characterized by comprising. According to this, since the traveling operation of the vehicle is restricted together with the warning display, it becomes possible to more reliably prevent a traffic accident.

(1) Vehicle blind spot monitoring device Hereinafter, an embodiment of a vehicle blind spot monitoring device according to the present invention will be described with reference to the drawings. As shown in FIGS. 11 and 12, the vehicle blind spot monitoring device according to the present embodiment is attached to a vehicle (for example, a truck) TR having a high vehicle height, and is a blind spot immediately before the vehicle that becomes a blind spot from the view of the driver EY. The area BS is monitored by photographing with the outside camera 3 installed outside the vehicle. On the other hand, a line-of-sight camera 4 for photographing the driver EY is installed in the vehicle, whereby the line of sight LS of the driver EY is grasped. When there is an obstacle K (for example, a short child) K in the blind spot area BS, as shown in the cockpit diagram of FIG. It is displayed on the windshield WS. Since this warning display DA is displayed in the vicinity on the line of sight LS of the driver EY, the driver EY can be surely notified of the presence of the obstacle K. Further, since the warning display DA includes an image KD of the obstacle K taken by the camera 3 outside the vehicle and a directional arrow AR indicating the position of the obstacle K, the driver EY has the obstacle K such as the position. You can instantly get information about.

  Further, the movement of the line of sight LS of the driver EY obtained by the line-of-sight camera 4 is measured, and the frequency (for example, the number of times and time) of facing the predetermined gaze direction is monitored. The gaze direction can be determined as a direction to be confirmed by the driver EY during driving such as the front of the vehicle, the left-right direction, and mirrors. Then, when the frequency at which the line of sight LS faces the gaze direction is lower than the reference frequency, as shown in FIG. 3, a warning display VS indicating that the visual recognition is insufficient is displayed to prompt the driver EY to visually recognize it. . Note that the direction to be confirmed by the driver EY depends on the speed of the vehicle TR, for example, in the vicinity of the vehicle when the vehicle is stopped and ahead of the vehicle at a low speed. .

(1-1) Device Configuration A configuration of a vehicle blind spot monitoring device will be described. FIG. 4 is a block diagram of the vehicle blind spot monitoring device 1. The vehicle blind spot monitoring device 1 includes an external camera 3 (external imaging means) that captures a blind area BS outside the vehicle, a line-of-sight camera 4 (driver imaging means) that captures the eyes of the driver EY, and images on the front windshield WS. A display device 5 (display means) for displaying, a communication I / F (interface) 6 and a control circuit 10 to which these are connected are provided.

  The outside camera 3 and the line-of-sight camera 4 are composed of a CCD camera, and input captured images to the control circuit 10 as video signals. In addition, two outside cameras 3 and two line-of-sight cameras 4 are installed to obtain a three-dimensional image.

  In the display device 5, the display driving circuit 53 drives the liquid crystal panel 51 and the backlight 52 based on the display command output from the control circuit 10. Specifically, as shown in FIG. 7, the liquid crystal panel 51 and the backlight 52 are disposed on an instrument panel 91 that extends from the lower side of the front windshield WS to the inside of the vehicle. The liquid crystal panel 51 is supported by the opening of the instrument panel 91 and is driven by the display drive circuit 53 and displays a display image by receiving light from the backlight 52. The display image is incident on the front windshield WS as display light and reflected toward the eyes of the driver EY seated on the driver's seat, thereby being captured by the driver EY as a virtual image.

  The communication I / F (interface) 6 has an interface function with the in-vehicle LAN, thereby enabling communication with other devices in the vehicle such as an ECU (Electronic Control Unit). The communication I / F 6 receives travel information such as vehicle speed from a control system ECU that controls the operation of the vehicle and inputs the travel information to the control circuit 10, and thus functions as travel information acquisition means of the present invention. Note that when travel information such as vehicle speed is directly input to the control circuit 10 from various sensors 16 (vehicle speed sensor or the like) mounted in the vehicle, the sensor 16 functions as travel information acquisition means.

  The control circuit 10 is configured as a normal computer, and includes a known CPU 11, ROM 12, RAM 13, input / output interface (I / O) 14, and a bus line 15 for connecting these configurations. The CPU 11 controls the program and data stored in the ROM 12 and RAM 13. The ROM 12 has a program storage area 12a and a data storage area 12b. A blind spot monitoring program 12p is stored in the program storage area 12a. The blind spot monitoring program 12p operates on the RAM 13 with the work memory 13w as a work area. On the other hand, gaze direction data 12d is stored in the data storage area 12b. The gaze direction data 12d is configured as data of a direction to be visually recognized by the driver and data of a reference frequency to be visually recognized (these are determined according to the vehicle speed). In addition, various data necessary for the operation of the blind spot monitoring program 12p is stored in the data storage area 12b. These programs and data may be stored in an external memory. With the above configuration, when the blind spot monitoring program 12p is started by the CPU 11, the control circuit 10 functions as an obstacle recognition unit, a line-of-sight detection unit, a display control unit, an obstacle direction calculation unit, and a line-of-sight movement measurement unit of the present invention. To do.

(1-2) Device Operation Next, the operation of the vehicle blind spot monitoring device 1 will be described with reference to FIGS. 5 and 6. Here, FIG. 5 shows a process of the blind spot monitoring program 12p executed by the CPU 11 of the vehicle blind spot monitoring apparatus 1 as a functional block diagram, and FIG. 6 shows the process as a flowchart.

  In the blind spot monitoring program 12p, the two processes of the obstacle recognition system process and the line-of-sight detection system process operate independently (that is, the CPU 11 performs multiple processes). In the obstacle recognition system process, a process of recognizing the obstacle K based on the image data input from the outside camera 3 and displaying the warning display DA is mainly performed. Further, in the line-of-sight detection system processing, mainly the process of detecting and warning the line of sight LS of the driver EY based on the image data input from the line-of-sight camera 4, and the detected line-of-sight data etc. Provide for processing. Hereinafter, a detailed description of these processes will be given.

  First, the CPU 11 receives input of vehicle travel information from the control system ECU via the communication I / F 6 (steps S1, T1), and when the vehicle is in a low speed state or a stopped state (steps S2, T2). The following processing is started. That is, the danger that a pedestrian or the like appears in the blind spot area BS immediately before the vehicle occurs when the vehicle is in a low speed state or a stopped state, and therefore the vehicle is below a predetermined speed (for example, 40 km / h or less: The following obstacle recognition processing and line-of-sight detection processing are performed. With this configuration, a warning is effectively issued in a scene with the highest risk of starting movement from a stopped state.

  Next, the obstacle recognition process (steps S3 and S4, the obstacle recognition unit 71) will be described. The CPU 11 receives input of image data from the two outside cameras 3 (see FIG. 12) for photographing the blind spot area BS, and grasps a three-dimensional object from a three-dimensional image obtained by stereoscopic viewing them (parallax calculation). ). Specifically, the ground is used as a reference plane, and a three-dimensional object therefrom is captured as an obstacle. Further, in this process, the presence of the obstacle K in the blind spot area BS and the position of the obstacle K with respect to the vehicle TR (driver EY) when the obstacle K exists are specified (for example, An obstacle K is located in front of the vehicle TR (immediately before the driver EY).

  Next, the line-of-sight detection process (step T3, line-of-sight detection unit 72) will be described. The line of sight LS of the driver EY is first obtained by analyzing the image data obtained from the line-of-sight camera 4, and the eye direction (or head position) of the driver EY is used as a reference in the direction of the line of sight. It can be obtained by giving a starting point. Specifically, the two line-of-sight cameras 4 (see FIG. 1) photograph the vicinity of the face of the driver EY, and the CPU 11 that receives the input of the image data specifies an eyeball region from the face image. The eyeball shape is stereoscopically viewed and the eyeball center coordinates are obtained from the curvature. On the other hand, a black eye (pupil) region is specified from the eyeball region, and a black eye center coordinate is also obtained. Then, the direction from the eyeball center coordinates to the black eye center coordinates is defined as a line-of-sight direction (line-of-sight vector). A straight line extending from the start point to the line of sight with the position coordinates of the eyeball as the start point is the line of sight LS. Note that the position of the approximate head of the driver sitting in the driver's seat is set in advance as a starting point, and the line of sight LS can be obtained from this. As described above, the line of sight LS of the driver EY is specified. The line of sight LS specified in this way is also provided to the obstacle recognition system processing side.

  Further, after the line-of-sight detection process (step T3, line-of-sight detection unit 72), a warning display position calculation process (step T4, warning display position calculation unit 76) is performed. This determines the display position (position near the line of sight LS) of warning displays DA and VS, which will be described later. The CPU 11 obtains a position where the line of sight LS strikes the front windshield WS (hereinafter also referred to as a butt position), and uses the vicinity as a display position. For example, in the data storage area 12b of the ROM 12, the visual field range (angle range of the line of sight LS) and the position range of the front windshield WS are stored in association with each other, and the line of sight LS is specified. The position of the corresponding front windshield WS can be read out. It should be noted that the display positions of the warning indications DA and VS are preferably displayed in the vicinity of the abutting position because if they are displayed at the abutting position, the driver EY is obscuring what the driver EY is currently viewing. The display positions of the warning indications DA and VS can be any position within the range where the viewing angle is about 20 ° or less, for example, with the contact position as the center. The display position of the warning display thus obtained is also provided to the obstacle recognition system processing side.

  Next, the obstacle direction calculation process (step S5, obstacle direction calculation unit 74) will be described. When the obstacle K in the blind spot area BS is recognized (step S4: Yes), the CPU 11 uses the line-of-sight LS as a reference based on the line-of-sight LS provided from the line-of-sight detection system processing side (step T3, the line-of-sight detection unit 72). The relative direction of the obstacle K to be calculated is calculated. Specifically, a direction in which a perpendicular line from the line of sight LS to the obstacle K is defined as the relative direction. The relative direction obtained here is used for a direction arrow AR included in a warning display DA described later (see FIG. 1).

  After the obstacle direction calculation process (step S5, obstacle direction calculation unit 74), a warning display process (step S6, display control unit 77 (77a)) that an obstacle exists is performed. The CPU 11 displays the display of the display device 5 so as to display the warning display DA indicating that the obstacle K exists at the warning display position provided from the line-of-sight detection system processing side (step T4, warning display position calculation unit 76). A display command is output to the drive circuit 53 (see FIG. 1). In addition, the CPU 11 uses the image of the obstacle K taken by the outside camera 3 as the image KD, and sets the relative direction obtained by the obstacle direction calculation processing (step S5, the obstacle direction calculation unit 74) as the position of the obstacle K. A display command is issued so as to be included in the warning display DA as a direction arrow AR indicating (see FIG. 1).

  In the above-described obstacle recognition processing (step S3, obstacle recognition unit 71), when there are a plurality of obstacles K in the blind spot area BS, the obstacles are individually recognized as obstacles. When a plurality of Ks are recognized, a plurality of warning displays DA corresponding to the respective obstacles K are displayed on the front windshield WS in the warning display process (step S6, display control unit 77 (77a)) (FIG. 2). reference).

  Next, in the line-of-sight detection system process, a process of measuring the movement of the line of sight LS of the driver EY and issuing a warning that the visual recognition is insufficient when the frequency of facing the gaze direction is small will be described. First, in the line-of-sight movement measurement process (step T5, line-of-sight movement measurement unit 75), the CPU 11 forms a line-of-sight movement measurement memory in the RAM 13 and sequentially stores the movement of the line of sight LS detected by the line-of-sight detection unit 72. . The operation of the line of sight LS is specified by, for example, the direction in which the line of sight LS is directed, the number of times, the time, and the like. When a predetermined time elapses after starting the measurement of the line-of-sight movement, the CPU 11 reads the reference frequency from the gaze direction data 12d stored in the data storage area 12b of the ROM 12, and the frequency ( When the number of times, time, etc.) is less than the reference frequency (step T6: Yes), the warning display VS indicating that the visual recognition is insufficient is calculated by the warning display position calculation process (step T4, warning display position calculation unit 76). A display command is output to the display drive circuit 53 of the display device 5 so as to display the warning display position (see FIG. 3) (step T7, display control unit 77 (77b)). Note that the gaze direction is set according to the vehicle speed as described above. The lower the vehicle speed, the closer the vehicle body is to be confirmed, and the greater the number is set.

(2) Vehicle Driving Support System Hereinafter, an embodiment of a vehicle driving support system according to the present invention will be described with reference to the drawings. The vehicle driving support system 100 according to the present embodiment controls the traveling operation of the vehicle using the vehicle blind spot monitoring device 1 described above. Since the description of the vehicle blind spot monitoring device 1 is as described above, the description thereof is omitted by attaching the same numbers in the drawing.

  FIG. 8 shows a block diagram of the vehicle driving support system 100. The vehicle driving support system 100 is configured such that the vehicle blind spot monitoring device 1 and a control system ECU (Electronic Control Unit) 110 that controls the traveling operation of the vehicle can communicate with each other via an in-vehicle LAN. The control system ECU 110 includes an engine ECU 111 that performs various controls related to the engine and a brake ECU that performs various controls related to the brake.

  FIG. 9 shows a function block diagram of the process of the blind spot monitoring program 12p executed by the CPU 11 of the vehicle blind spot monitoring device 1 in the vehicle driving support system 100, and FIG. 10 shows the process as a flowchart. . The processes of steps S1 to S6 related to the obstacle recognition system process and steps T1 to 7 related to the line-of-sight detection system process are the same as the process of the vehicle blind spot monitoring apparatus 1 described in FIG. In the vehicle driving support system 100, the CPU 11 of the vehicle blind spot monitoring device 1 displays a warning display process indicating that there is an obstacle (step S6, display control unit 77 ( 77a)) and a command (output) for restricting the traveling operation of the vehicle to the control system ECU 110 is transmitted via the in-vehicle LAN by the communication I / F 6 (step S7, display control unit 77 (77a)). .

  When the control system ECU 110 receives a travel operation restriction command from the vehicle blind spot monitoring device 1, for example, even if the engine ECU 111 receives an operation input of an accelerator pedal, the control system ECU 110 stops the fuel supply to the engine to start or stop the vehicle TR. The traveling operation of the vehicle TR is limited by operating so as not to accelerate, or by operating the brake ECU 112 to stop or decelerate the vehicle TR by automatically applying a brake. This makes it possible to prevent an accident from occurring when there is an obstacle (for example, a short child) K in the blind spot area BS.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to these, In the range which does not deviate from the summary, it can change suitably and can implement. For example, in the present embodiment, the two processes of the obstacle recognition system process and the line-of-sight detection system process are performed by one control circuit. However, these processes may be performed by separate control circuits.

The figure showing the case where the warning display to the effect that the obstacle exists is displayed on the front windshield by the vehicle blind spot monitoring device of the present invention Modification of FIG. The figure showing the case where the warning display to the effect that visual recognition is insufficient by the vehicle blind spot monitoring device of the present invention is displayed on the front windshield Block diagram of the vehicle blind spot monitoring device of the present invention The functional block diagram showing the process which the blind spot monitoring apparatus for vehicles of this invention performs The flowchart showing the process which the blind spot monitoring apparatus for vehicles of this invention performs. Illustration of display device Block diagram of the vehicle driving support system of the present invention The functional block diagram showing the process which the driving assistance system for vehicles of this invention performs The flowchart showing the process which the driving assistance system for vehicles of this invention performs A diagram that schematically shows the case where an obstacle exists in the blind spot area of the driver Front view of FIG.

Explanation of symbols

1 vehicle blind spot monitoring device 3 vehicle exterior camera (vehicle exterior imaging means)
4 Line-of-sight camera (driver imaging means)
5. Display device (display means)
6 Communication I / F (Driving information acquisition means)
10 Control circuit (obstacle recognition means, line-of-sight detection means, display control means, obstacle direction calculation means, line-of-sight movement measurement means)
DESCRIPTION OF SYMBOLS 100 Vehicle driving assistance system 110 Control system ECU (driving control means)
BS blind spot area DA warning display EY driver LS line of sight WS front windshield

Claims (2)

A driver imaging means for photographing the driver's eyes;
Line-of-sight detection means for detecting the line of sight of the driver from an image obtained by the driver imaging means;
Eye movement measuring means for measuring the movement of the driver's eyes,
Display means for displaying images on the windshield of the vehicle;
When the frequency of the driver's line of sight facing a predetermined gaze direction is less than a reference frequency, there is insufficient visual recognition in the vicinity of the position corresponding to the driver's line of sight in the windshield. Display control means for displaying a warning on the display means,
The reference frequency is set according to the vehicle speed ,
Comprising travel information acquisition means for acquiring travel information of the vehicle, the gaze direction is set according to the travel information of the vehicle;
The blind spot monitoring device for vehicles, wherein the gaze direction is set so as to check a location closer to the vehicle body of the vehicle as the vehicle heads to a stop state . Vehicle, wherein the vehicle blind spot monitoring system according to claim 1, said display control means is configured to include a driving control device for limiting the travel operation of the vehicle when displaying the warning Driving support system.

Images