JP2008071062A - Operation support device and notification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide notification which is just enough and appropriate to a driver of a vehicle. <P>SOLUTION: A circumstance judging part 11 detects an object such as a pedestrian to which the driver of a vehicle should be urged to pay attention by a radar 33 or an outside camera 34 and a night vision camera 35, and a notification control part 14 controls an inside notification system 50 to notify the driver. Afterwards, a driver recognition and estimation part 14a estimates whether or not the driver has recognized the notified object based on a driving operation status or visual field circumstances by the driver and the classification of a monitor means used for object detection, and ends the notification when the object is recognized, and continues notification in the other case. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a driving support apparatus that notifies a driver of information and a notification method thereof, and more particularly, to a driving support apparatus that prevents an excessive or insufficient notification and performs appropriate notification and a notification method thereof.

  In recent years, a technique has been devised that supports driving operations by monitoring the periphery of a vehicle and notifying the driver of the monitoring results.

  Although image recognition and radar are used for surrounding monitoring, if all of the information obtained by this monitoring is notified, it becomes a notification containing a large amount of information unnecessary for the driver, which may hinder driving operation.

  Therefore, it is important to detect an object that should alert the driver from the monitoring result, and select and notify information necessary for the driver. Information necessary for the driver is, for example, the presence of other vehicles or pedestrians. Especially for pedestrians, it is very important for the driver to recognize its presence and avoid collisions.

  Furthermore, it is not necessary to notify all other vehicles and pedestrians in the same way, and the risk of collisions varies depending on the positional relationship with the host vehicle. It is done.

  As a technique for determining the risk level of a pedestrian, for example, Patent Document 1 discloses a technique for determining a risk level in consideration of a driver's condition and road condition in addition to a pedestrian condition. Document 2 discloses a technique for determining the degree of risk according to the pedestrian's line of sight.

JP 2006-163637 A JP 2004-362586 A

  By the way, after detecting and notifying an object that should alert the driver, if the notification is continued even after the driver has recognized the object, the driving operation is hindered. On the other hand, if the notified object cannot be recognized, such as when the visibility is poor or the driver is a beginner, it is necessary to continue the notification and promote recognition.

  That is, it has been an important issue in the past to perform notification without excess or deficiency based on whether or not the driver has recognized the notified object.

  The present invention has been made to solve the above-described problems in the prior art, and prevents excessive or insufficient notification by estimating whether or not the driver has recognized the notified object and controlling the notification. It is an object of the present invention to provide a driving support apparatus and a notification method for performing appropriate notification.

  In order to solve the above-described problems and achieve the object, the driving support apparatus and the notification method according to the present invention detect an object that should alert the driver from the monitoring result by the surrounding monitoring means, and the detected object is detected by the driver. Then, it is estimated whether or not the driver has recognized the notified object, and it is determined whether or not the notification can be continued based on the estimation result.

  According to the present invention, the driving support device and the notification method terminate the notification when the driver recognizes the notified object, and continue the notification when the driver does not recognize the notified object. There is an effect that it is possible to obtain a driving support device and a notification method that perform notification.

  Exemplary embodiments of a driving support apparatus according to the present invention will be described below in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic configuration diagram illustrating a schematic configuration of a vehicle equipped with a driving support apparatus according to the present invention. As shown in the figure, in the vehicle C1, the driving support device 10 includes a communication device 31, a vehicle speed sensor 32, a radar 33, an outside camera 34, a night vision camera 35, a navigation device 36, an in-vehicle camera 37, a notification stop switch 38, a driver. The information acquisition device 39, the operation control system 40, and the in-vehicle notification system 50 are connected.

  The communication device 31 is a communication unit that communicates with another vehicle or a roadside communication device installed on the road. The vehicle speed sensor 32 is detection means for measuring the traveling speed of the vehicle C1, for example, from the wheel speed, and the radar 33 irradiates millimeter waves, lasers, etc., and detects the object around the vehicle from the reflected wave to the detected object. It is the detection means which measures the distance. The vehicle exterior camera 34 is a photographing means for photographing the periphery of the host vehicle in the visible light region. Furthermore, the night vision camera 35 is a photographing unit that photographs the periphery of the host vehicle in the infrared region.

  The navigation device 36 is an in-vehicle device that sets and guides a travel route using the position of the host vehicle specified by communicating with a GPS (Global Positioning System) artificial satellite and map data stored in advance. In addition, the navigation device 36 provides the driving support device 10 with position information of the host vehicle, surrounding map information, information about roads, time information, and the like.

  The in-vehicle camera 35 is a photographing means for photographing the inside of the host vehicle, particularly the state of the driver. The notification stop switch 38 is installed at a position where the driver can operate during driving, and is operated when the driver requests the end of notification.

  The driver information acquisition device 39 is a device that acquires information related to the driver, particularly information related to driving experience and driving technology. The information regarding the driver may be input by the driver himself or may be read out from a recording medium such as a card. In addition, information regarding users who may drive the vehicle C1 may be stored in association with each other, and the driver may be identified and read by image recognition or the like.

  The operation control system 40 is a group of devices that control the operation of the vehicle, and includes an engine control mechanism 41 that controls the operation of the engine based on an accelerator operation, a braking mechanism 42 that brakes the vehicle based on an operation of a brake pedal, and a handle. It includes a steering mechanism 43 that controls the steering angle of the vehicle based on the operation, a horn 44 that emits a warning sound around the vehicle, a wiper 45 that sweeps the windshield to ensure the driver's view.

  The in-vehicle notification system 50 is a group of devices that notify the passengers of the vehicle C1, particularly the driver, and includes a speaker 51 that performs voice notification, a display 52 that performs display notification, and the like. This in-car notification system 50 can be shared by various in-vehicle devices such as the navigation device 36 and an in-vehicle audio device (not shown) in addition to performing output under the control of the driving support device 10.

  The driving support device 10 includes a situation determination unit 11, a weather determination unit 12, a driver state recognition unit 13, and a notification control unit 14 therein. The weather determination unit 12 is a processing unit that determines the weather around the host vehicle, particularly the presence or absence of rain, fog, or snow that adversely affects the field of view. The determination of the weather can be made based on the image recognition of the image taken by the outside camera 34, and can be determined based on, for example, the operation state of the wiper 45 or the lighting state of a fog lamp (not shown).

  The driver state recognition unit 13 is a means for recognizing the state of the driver by image recognition on the driver image taken by the in-vehicle camera 37, and recognizes the face direction, dozing, blinking, conversation with the passenger, and the like. Can do.

  The situation determination unit 11 is a processing unit that performs a process of determining the situation of the host vehicle. Specifically, information received by the communication device 31, detection results of the vehicle speed sensor 32 and the radar 33, image recognition for an image taken by the outside camera 34, image recognition for an image taken by the night vision camera 35, and the navigation device 36 Using the acquired position information, map information, road information, time information, the operation state of the own vehicle acquired from the operation control system 40, the determination result by the weather determination unit 12, and the driver state recognized by the driver state recognition unit 13 Recognize the presence of pedestrians and other vehicles, and determine the possibility of a collision with the host vehicle.

  In addition, the situation determination unit 11 performs notification to the driver and intervention in vehicle operation control based on the situation determination result. Here, the intervention to the vehicle operation control is executed as a control request output to the operation control system 40, and the notification to the driver is executed as a notification request to the notification control unit 14. For example, when the situation determination unit 11 detects the presence of a pedestrian, the notification request for alerting the pedestrian is output to the notification control unit 14.

  The notification control unit 14 is a processing unit that receives the notification request from the situation determination unit 11 and controls the in-car notification system 50 to notify the driver, and the driver recognition estimation unit 14a and the notification continuation determination are included therein. Part 14b.

  The driver recognition estimation unit 14a is the input means that the driving operation state acquired from the operation control system 40, the operation state of the notification stop switch 38, the determination result of the weather determination unit 12, and the situation determination unit 11 used for the situation determination. It is estimated whether or not an object (for example, a pedestrian) that has been cautioned by the notification is recognized based on the type of.

  The notification continuation determination unit 14b determines whether or not to continue the notification based on the estimation result of the driver recognition estimation unit 14a, the recognition result of the driver state recognition unit 13, and the driver information acquired by the driver information acquisition device 39. Determine whether.

  Next, the operation of the driving support device 10 will be described in more detail. First, when the situation determination unit detects the presence of a pedestrian by image processing, radar detection, or the like, a risk level is calculated based on the state of the pedestrian.

  Specifically, as shown in FIG. 2, if the pedestrian is facing backwards or sideways, it is more likely that the pedestrian is not aware of the presence of the host vehicle compared to when the pedestrian is facing the front. Set the degree higher.

  Further, when the position of the pedestrian is within the course of the host vehicle, the degree of danger is set higher than when the pedestrian is outside the course of the host vehicle. Similarly, when the pedestrian is moving in the direction of entering the course of the own vehicle, the degree of danger is set higher than in the case of moving in the direction of going out of the course of the own vehicle. Furthermore, when the moving speed of the pedestrian is slow, the degree of danger is set higher than when the moving speed is fast.

  Also, when the pedestrian has a large luggage, the danger level is set higher because it is harder to move and the possibility of contact is higher than when the luggage is small (no luggage).

  When the clothes of the pedestrian are dark or the same color as the road, the visibility from the driver is lower than when the clothes are light, so the degree of danger is set high. Also, if the pedestrian is a child, there is a possibility that it may suddenly jump out without noticing the car, and if it is an elderly person, there is a possibility that it may not notice the car or it may suddenly fall, so each risk level Set high.

  A risk value based on the state of the pedestrian can be calculated by processing such as assigning and summing appropriate values to the state of the pedestrian. Further, the degree of risk obtained from the pedestrian's own condition is corrected according to the surrounding road condition, weather condition, and driver condition. For example, as shown in FIG. 3, in the situation where the own vehicle C1 has a pedestrian crossing in the traveling direction and there is a pedestrian H1, if the vehicle traffic light S1 is red and the pedestrian traffic light S2 is blue, the host vehicle stops. Although the risk is low, if the vehicle traffic light S1 is blue and the pedestrian traffic light S2 is red, the host vehicle C1 travels beyond the pedestrian crossing, so that the presence of the vehicle in preparation for the pedestrian H1 jumping out It is necessary to make it recognize.

  Therefore, the situation determination unit 11 acquires the road shape in the vicinity of the host vehicle from the navigation device 36, acquires the state of the traffic light through image recognition and communication with the outside of the vehicle by the communication device 31, and corrects the risk of pedestrians. .

  Similarly, the situation determination unit 11 uses the recognition result of the driver state recognition unit 13 to display the front in the case where the driver is talking to the passenger, the patient is asleep, the blink is a lot, The risk level is corrected to be high when it is not suitable.

  In addition, when the driver is listening to music, or when traveling in a place where there is a lot of noise in the surroundings, or when another on-vehicle device such as the navigation device 36 or an air conditioner (not shown) is being operated, the traveling speed is high. In some cases, it is desirable to increase the degree of danger. These states can be acquired not only by image recognition but also by the operation state (operation reception state) of the microphone and other devices, the vehicle speed sensor 32, and the like.

  Furthermore, based on the determination result by the weather determination unit 12, the situation determination unit 11 corrects the degree of danger to be high when the visibility is bad, such as rain, fog, and snow.

  And about the pedestrian who has a certain degree or more (for example, more than predetermined threshold value) danger among the detected pedestrians, a notification request | requirement is performed to the notification control part 14 as an object which should call a driver | operator's attention.

  The notification control unit 14 receives the notification request from the situation determination unit 11 and notifies the driver, and then determines whether or not the driver has recognized the notified target.

  Specifically, as shown in FIG. 4, when the driver returns the accelerator pedal, releases his / her foot from the accelerator pedal, puts his / her foot on the brake pedal, or depresses the brake pedal, he / she performs a downshift operation. In this case, when the horn is sounded by operating the alarm 44, and further when the notification stop switch 38 is operated, the driver recognition estimation unit 14a estimates that the driver has recognized the danger, and the notification continuation determination unit 14b ends the notification.

  Further, when the visibility is poor, such as when the weather is rain, fog, snow, or when the time is night, the driver recognition estimation unit 14a changes the estimation result depending on how the target to be alerted is detected. .

  In other words, when a pedestrian is detected by image recognition using an image captured by the outside camera 34 that captures the visible light region, it is considered that the pedestrian can be confirmed by visual recognition of the driver as well. It is estimated that the driver has recognized a pedestrian. On the other hand, when a pedestrian is detected from the output of the radar 33 or the night vision camera 35, the driver may not be able to visually recognize the pedestrian.

  Therefore, when the visibility is poor, the notification continuation estimation unit 14b continues the notification about the danger detected by other than the image recognition, and ends the notification about the danger detected by the image recognition.

  Next, the processing operation of the driving support device 10 will be further described with reference to the flowchart shown in FIG. The processing operation shown in the figure is a processing flow when the driving support device 10 notifies the presence of a pedestrian, and is repeatedly executed.

  First, the situation determination unit 11 detects pedestrians around the host vehicle from outputs of monitoring means such as the communication device 31, the vehicle speed sensor 32, the radar 33, the outside camera 34, and the night vision camera 35 (step S101).

  As a result, if the presence of a pedestrian is detected (step S102, Yes), the position information acquired from the navigation device 36, map information, road information, the operation state of the host vehicle acquired from the motion control system 40, and the like are further used. Then, the risk level of the pedestrian is determined (step S103).

  Further, after the risk level correction using the driver status recognized by the driver status recognition unit 13 (step S104) and the risk level correction based on the determination result by the weather determination unit 12 (step S105), Notification processing by the notification control unit 14 is executed so as to call the driver's attention about a pedestrian having a high degree (step S106).

  After execution of the notification process, the driver recognition determination unit 14 estimates whether or not the driver has recognized the pedestrian (step S107), and if not recognized (step S107, No), the notification process is continued (step S107). S106). Here, the continued notification process need not be the same as before, and the content of the treatment may be changed, for example, by increasing the volume of the voice notification.

  And a notification process is complete | finished when a driver | operator recognizes a pedestrian (step S107, Yes) (step S108). Here, if the driver is a beginner from the driver information, or if there is a possibility that the driver has overlooked the driver state such as looking aside or falling asleep, the notification process may be continued. .

  After the notification process is completed or when a pedestrian is not recognized (step S102, No), the process flow is finished.

  As described above, the driving support device 10 according to the present embodiment notifies the object to be alerted that is detected by the situation determination unit 11, and then the driving operation state, visibility state, and object detection by the driver. It is estimated whether or not the driver has recognized the notified object based on the type of the monitoring means used in the above, and if recognized, the notification is terminated, and if not recognized, the notification is continued.

  Therefore, it is possible to prevent the continuation of the notification about the danger recognized by the driver and the termination of the notification for the danger not recognized by the driver, and to make an appropriate notification without excess or deficiency. Such appropriate notifications greatly contribute to the safe driving of the vehicle because it prevents the driver from feeling uncomfortable due to excessive notifications and prevents the reliability of the notifications from being reduced, and prevents accidents due to insufficient notifications. be able to.

  As described above, the driving support device and the notification method according to the present invention are useful for information notification in a vehicle, and are particularly suitable for executing appropriate notifications with no excess or deficiency.

1 is a schematic configuration diagram showing a schematic configuration of a vehicle according to an embodiment of the present invention. It is explanatory drawing explaining the relationship between a pedestrian's state and a danger level. It is explanatory drawing explaining the change of the danger level by the state of a signal. It is explanatory drawing explaining the estimation of recognition by a driver | operator. It is a flowchart explaining the processing operation of a driving assistance device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Hand deeper device 11 Situation judgment part 12 Weather judgment part 13 Driver state recognition part 14 Notification control part 14a Driver recognition estimation part 14b Notification continuation judgment part 31 Communication apparatus 32 Vehicle speed sensor 33 Radar 34 Outside camera 35 Night vision camera 36 Navigation device 37 In-car camera 38 Notification stop switch 39 Driver information acquisition device 40 Operation control system 41 Engine control mechanism 42 Braking mechanism 43 Steering mechanism 44 Alarm sounder 45 Wiper 50 In-car notification system 51 Display 52 Speaker C1 Vehicle

Claims (8)

Situation determination means for detecting a situation from the monitoring result by the periphery monitoring means for monitoring the periphery of the host vehicle, and detecting an object that should alert the driver;
Notification control means for notifying the driver of the object detected by the situation determination means;
Driver recognition estimation means for estimating whether or not the driver has recognized the notified object;
Notification continuation determining means for determining whether or not to continue the notification based on the estimation result of the driver recognition estimating means;
A driving support apparatus comprising:   The driving assistance apparatus according to claim 1, wherein the situation determination unit detects a pedestrian as an object to be alerted to the driver.   The said driver recognition estimation means estimates that the said notified object was recognized when the driver | operator performed predetermined | prescribed driving operation after the said notification. Driving assistance device.   A plurality of the surrounding monitoring means are provided, and the driver recognition estimating means determines the object notified to the driver based on which of the plurality of surrounding monitoring means is detected based on the monitoring result. The driving support device according to claim 1, 2 or 3, wherein it is estimated whether or not it is recognized.   As one of the plurality of surrounding monitoring means, a photographing means for photographing the periphery of the host vehicle is provided, and the driver recognition estimating means drives the object detected by the image recognition on the image photographed by the photographing means according to the notification. The driving support device according to claim 4, wherein it is estimated that the person has recognized.   It further comprises weather judgment means for judging the weather around the host vehicle, and the driver recognition estimation means estimates whether or not the driver has recognized the notified object based on the weather. The driving assistance device according to any one of claims 1 to 5.   The notification stop requesting unit operated by a driver is further provided, and the notification continuation determining unit ends the notification when the notification stop requesting unit is operated. The driving assistance device according to one. A periphery monitoring process for monitoring the periphery of the vehicle,
From the monitoring result of the periphery monitoring step, a detection step of detecting an object to alert the driver;
A notification step of notifying the driver of the object detected by the detection step;
A recognition estimation step of estimating whether or not the driver has recognized the notified object;
A continuation determination step of determining whether or not to continue the notification based on an estimation result of the recognition estimation step;
The notification method characterized by including.

Images