JP2017068673A - Driving support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make a driver visually confirm a change in surroundings of a vehicle more reliably, when the surroundings of the vehicle have changed.SOLUTION: A driving support device according to an embodiment comprises: a recognition processing part for recognizing surroundings of a vehicle; a driving support processing part for performing a driving support control in accordance with the surroundings of the vehicle recognized by the recognition processing part; a display processing part for displaying a change in the surroundings of the vehicle on a display part, when the surroundings of the vehicle have changed; a detection processing part for detecting information relating to a driver's visual line of the vehicle; and an output processing part. When a recognition state has changed, the output processing part uses a notification part that is different from the display part to output a first notification for notifying the driver that the surroundings of the vehicle have changed, and after the output of the first notification, on the basis of detection results obtained by the detection processing part, outputs a second notification for re-notifying the driver that the surroundings of the vehicle have changed, using the notification part, when the driver's visual line is not detected to be directed to the display part.SELECTED DRAWING: Figure 7

Description

  Embodiments described herein relate generally to a driving support apparatus.

  2. Description of the Related Art Conventionally, a technique for performing driving support control according to a situation around a vehicle recognized by a sensor or a camera is known. In such a technique, when the situation around the vehicle changes, the change in the situation is visually displayed on a display unit or the like.

JP 2003-34205 A

  In the technology as described above, it is desired to make the driver visually recognize the situation around the vehicle when the situation around the vehicle changes.

  The driving support device according to the embodiment includes a recognition processing unit that recognizes a situation around the vehicle, a driving support processing unit that executes driving support control according to the situation around the vehicle recognized by the recognition processing unit, When the surroundings change, the display processing unit that displays changes in the surroundings of the vehicle on the display unit, the detection processing unit that detects information about the line of sight of the driver of the vehicle, and the surroundings of the vehicle change In such a case, the first notification that notifies the driver that the situation around the vehicle has changed is output using a notification unit that is different from the display unit, and the detection process is performed after the first notification is output. A second notification that uses the notification unit to notify the driver again that the situation around the vehicle has changed when it is not detected that the driver's line of sight is directed to the display unit based on the detection result by the unit. An output processing unit for outputting a notification. With such a configuration, it is possible to make the driver visually recognize changes in the situation around the vehicle using the first notification and the second notification.

  In the driving support device according to the embodiment, the output processing unit repeatedly outputs the second notification until it is detected that the driver's line of sight is facing the display unit. With such a configuration, it is possible to make the driver visually recognize changes in the situation around the vehicle.

  In the driving support device according to the embodiment, the output processing unit outputs the second notification in an output form different from the first notification and outputs the second notification two or more times. The second notification is output in an output form different from the second notification output in the past. With such a configuration, for example, it is possible to make the driver visually recognize the change in the situation around the vehicle by increasing the alerting power of the notification that is output later.

  In the driving support device according to the embodiment, when the second notification is output twice or more, the output processing unit outputs the second notification up to a preset number of times after outputting the first notification for the second time. Output notifications repeatedly. With such a configuration, it is possible to suppress the driver from feeling excessively bothered by the repeated output of the second notification.

  In the driving support device according to the embodiment, when the output processing unit outputs the second notification two or more times, after the first second notification is output, the detection processing unit detects the driver's line of sight. The second notification is repeatedly output until it is detected that it is facing the display unit and that the driver has performed a predetermined active eye operation. With such a configuration, it can be more reliably determined whether or not the driver has actively looked at the display unit.

  In the driving support device according to the embodiment, the notification unit includes a sound output unit capable of outputting sound, and the output processing unit outputs the first notification and the second notification by sound using the sound output unit. To do. With such a configuration, it is possible to more easily notify the driver that the situation around the vehicle has changed using sound.

FIG. 1 is an exemplary perspective view showing a state in which a part of a compartment of a vehicle according to the first embodiment is seen through. FIG. 2 is an exemplary view showing an instrument panel according to the first embodiment. FIG. 3 is an exemplary view showing an example of an image displayed on the display device of the instrument panel according to the first embodiment. FIG. 4 is an exemplary view showing another example of an image displayed on the display device of the instrument panel according to the first embodiment, which is different from FIG. 3. FIG. 5 is an exemplary diagram illustrating an arrangement of the imaging device according to the first embodiment. FIG. 6 is an exemplary block diagram showing the configuration of the driving support system mounted on the vehicle according to the first embodiment. FIG. 7 is an exemplary block diagram showing a functional configuration of the ECU according to the first embodiment. FIG. 8 is an exemplary flowchart showing a process executed by the ECU according to the first embodiment. FIG. 9 is an exemplary flowchart showing processing executed by the ECU according to the second embodiment. FIG. 10 is an exemplary flowchart showing a process executed by the ECU according to the third embodiment. FIG. 11 is an exemplary diagram showing an example of an image displayed on the display device of the navigation system when the recognition state around the vehicle changes in the modified example. FIG. 12 is an exemplary diagram showing an example of an image displayed on the display device of the navigation device before the situation around the vehicle changes in the modified example. FIG. 13 is an exemplary diagram showing an image displayed by switching from the image of FIG. 12 after the situation around the vehicle changes in the modified example. FIG. 14 is an exemplary diagram showing an example different from FIG. 12 of an image displayed on the display device of the navigation device before the situation around the vehicle changes in the modified example. FIG. 15 is an exemplary diagram showing an image displayed by switching from the image of FIG. 14 after the situation around the vehicle changes in the modified example. FIG. 16 is an exemplary diagram showing an example different from FIGS. 12 and 14 of an image displayed on the display device of the navigation device before the situation around the vehicle changes in the modified example. FIG. 17 is an exemplary diagram showing an image displayed by switching from the image of FIG. 16 after the situation around the vehicle changes in the modification. FIG. 18 is an exemplary diagram showing an example different from FIGS. 12, 14, and 16 of an image displayed on the display device of the navigation device before the situation around the vehicle changes in the modified example. FIG. 19 is an exemplary diagram showing an image displayed by switching from the image of FIG. 18 after the situation around the vehicle changes in the modified example.

  Hereinafter, embodiments will be described with reference to the drawings.

(First embodiment)
First, with reference to FIGS. 1-5, the schematic structure of the vehicle 1 by 1st Embodiment is demonstrated.

  The vehicle 1 according to the first embodiment may be, for example, a vehicle (an internal combustion engine automobile) using an internal combustion engine such as an engine as a drive source, or a vehicle (electric vehicle, fuel cell) using an electric motor such as a motor as a drive source. A car). Further, the vehicle 1 may be a vehicle (hybrid vehicle) using both of these as drive sources. In addition, the vehicle 1 can be mounted with various transmissions and various devices (systems, components, etc.) necessary for driving the internal combustion engine and the electric motor. In addition, the method, number, layout, and the like of devices related to driving of the wheels 3 in the vehicle 1 can be variously set.

  As shown in FIG. 1, the vehicle 1 according to the first embodiment includes a vehicle body 2. The vehicle body 2 constitutes a passenger compartment 2a in which an occupant (not shown) gets. In the passenger compartment 2a, a steering unit 4, an instrument panel 25, and the like are provided so as to correspond to a seat 2b of a driver (not shown in FIG. 1) as a passenger. The steering unit 4 is, for example, a steering wheel 4a protruding from the dashboard 12.

  As shown in FIG. 2, the instrument panel 25 is provided with a speedometer 25a, a tachometer 25b, and a display device 25c. The display device 25c is, for example, a liquid crystal display or an organic EL (Electroluminescent) display. In the example of FIG. 2, the display device 25 c is positioned between the speedometer 25 a and the tachometer 25 b at the approximate center of the instrument panel 25.

  In the first embodiment, information related to driving support control executed by the vehicle 1 is displayed on the display device 25c of the instrument panel 25. As an example of the driving support control, for example, follow-up running control that causes the vehicle 1 to automatically follow the preceding vehicle. When the follow-up running control is executed, the display device 25c displays information such as the presence or absence of the preceding vehicle and the distance between the preceding vehicle and the like in the manner shown in FIGS. 3 and 4, for example. Is done. FIG. 3 is an illustrative diagram showing a specific example of an image IM1 displayed on the display device 25c when a preceding vehicle is present. In the image IM1 of FIG. 3, a mark M1 indicating a preceding vehicle is displayed. FIG. 4 is an illustrative diagram showing a specific example of an image IM2 displayed on the display device 25c when there is no preceding vehicle. In the image IM2 in FIG. 4, unlike the image IM1 in FIG. 3, the mark M1 indicating the preceding vehicle is not displayed.

  Returning to FIG. 1, a display device 8 different from the display device 25c and an audio output device 9 are provided in the passenger compartment 2a. The display device 8 is, for example, a liquid crystal display or an organic EL display. Moreover, the audio | voice output apparatus 9 is a speaker, for example. In the example of FIG. 1, the display device 8 and the audio output device 9 are provided in a navigation device 11 that is located at the center (so-called center console) of the dashboard 12 in the vehicle width direction (left-right direction).

  In the first embodiment, information (image) such as the current position of the vehicle 1 and route guidance to the destination is displayed on the display device 8 of the navigation device 11 when the vehicle 1 is traveling. The display device 8 includes an operation input unit 10 such as a touch panel. The occupant of the vehicle 1 can execute various operation inputs (instruction inputs) on the image displayed on the display device 8 by touching the operation input unit 10 with fingers or the like. In addition, the navigation apparatus 11 can also have operation input parts (not shown), such as a switch, a dial, a joystick, and a push button.

  Here, in the first embodiment, as shown in FIG. 5, the imaging device 201 is installed on the steering column 4b of the steering wheel 4a. The imaging device 201 is, for example, a CCD (Charge Coupled Device) camera. The imaging device 201 is adjusted in view angle and posture so that the face of the driver 302 seated on the seat 2b is positioned at the center of the view. The imaging apparatus 201 captures the face of the driver 302 sitting on the seat 2b and sequentially outputs image data obtained by the capturing. In addition, the imaging device 201 should just be installed in the position which can image | photograph the eyes | visual_axis of the driver | operator 302, for example, the center console in which the navigation apparatus 11 is installed, the A pillar by the side of a driver's seat (seat 2b), It may be installed in a room mirror.

  Next, with reference to FIG. 6, the structure of the driving assistance system 100 mounted in the vehicle 1 by 1st Embodiment is demonstrated.

  As shown in FIG. 6, in the driving support system 100, in addition to the ECU 14, the navigation device 11, the steering system 13, the distance measuring units 16 and 17, the brake system 18, the steering angle sensor 19, the accelerator sensor 20, and the shift sensor 21. The wheel speed sensor 22 and the like are electrically connected via an in-vehicle network 23 as an electric communication line. The ECU 14 is an example of a “driving support device”.

  The in-vehicle network 23 is configured, for example, as a CAN (Controller Area Network). The ECU 14 can operate the actuator 13 a of the steering system 13, the actuator 18 a of the brake system 18, and the like by sending a control signal through the in-vehicle network 23. Further, the ECU 14 detects detection results of the torque sensor 13b, the brake sensor 18b, the rudder angle sensor 19, the distance measuring units 16 and 17, the accelerator sensor 20, the shift sensor 21, the wheel speed sensor 22, and the like via the in-vehicle network 23, An operation signal from the operation input unit 10 can be received.

  In the example of FIG. 6, the ECU 14 includes a CPU 14a (Central Processing Unit), a ROM 14b (Read Only Memory), a RAM 14c (Random Access Memory), a display control unit 14d, an audio control unit 14e, and an SSD 14f (Solid State D). Flash memory).

  The CPU 14a controls the entire vehicle 1. The CPU 14a reads a program installed in a non-volatile storage device such as the ROM 14b, and executes arithmetic processing according to the program. The RAM 14c temporarily stores various data used in the calculation of the CPU 14a.

  The display control unit 14d mainly executes image processing using image data obtained by the imaging devices 15 and 201, synthesis of image data displayed by the display devices 8 and 25c, and the like, among arithmetic processing in the ECU 14. . In addition, the voice control unit 14 e mainly executes processing of voice data output from the voice output device 9 among the calculation processes in the ECU 14. The SSD 14f is a rewritable nonvolatile storage device, and can retain data even when the power of the ECU 14 is turned off, for example.

  In the first embodiment, the CPU 14a, the ROM 14b, the RAM 14c, and the like can be integrated in the same package. In the ECU 14, instead of the CPU 14a, other logical operation processors such as a DSP (Digital Signal Processor), a logic circuit, or the like may be used. The ECU 14 may be provided with an HDD (Hard Disk Drive) instead of the SSD 14f (or in addition to the SSD 14f). Note that the SSD 14f and the HDD may be provided separately from the ECU 14.

  The configuration, arrangement, electrical connection form, and the like of the various sensors and actuators described above are examples, and can be set (changed) in various ways.

  Here, the CPU 14a of the ECU 14 according to the first embodiment executes a program stored in the ROM 14b when executing the driving support control, thereby realizing a functional configuration (functional module) as shown in FIG. 7 on the RAM 14c. To do.

  As shown in FIG. 7, the ECU 14 includes a recognition processing unit 401, a driving support processing unit 402, a display processing unit 403, a detection processing unit 404, and an output processing unit 405 as functional configurations.

  The recognition processing unit 401 recognizes the situation around the vehicle 1 based on image data obtained from the imaging device 15 and the like. The driving support processing unit 402 performs driving support control according to the situation around the vehicle 1 recognized by the recognition processing unit 401. For example, when the follow-up running control is executed as the driving support control, the recognition processing unit 401 recognizes the presence / absence of the preceding vehicle, the inter-vehicle distance from the preceding vehicle, and the like. In the following, only the case where the driving support control is the following traveling control will be described as an example. However, the driving assistance control according to the first embodiment is not limited to the following traveling control, and may include a parking assistance control and the like. . In the following, for the sake of simplification, the situation around the vehicle 1 recognized by the recognition processing unit 401 is expressed as a recognition situation.

  The display processing unit 403 controls an image to be displayed on the display device 25 c of the instrument panel 25 according to the recognition status by the recognition processing unit 401. For example, when it is recognized that a preceding vehicle is present, the display processing unit 403 displays an image IM1 (see FIG. 3) including the mark M1 indicating the preceding vehicle on the display device 25c.

  Here, when the recognition status by the recognition processing unit 401 changes, the display processing unit 403 according to the first embodiment displays the change in the recognition status on the display device 25c. The change in the recognition status is a change that triggers switching of the control content of the follow-up traveling control. In the follow-up running control, for example, control is performed so that the vehicle 1 is automatically accelerated when the state where it is recognized that the preceding vehicle is present is changed to the state where it is recognized that the preceding vehicle is not present. The contents are switched automatically. Therefore, the display processing unit 403 according to the first embodiment has changed from a state where it is recognized that the preceding vehicle is present to a state where it is recognized that there is no preceding vehicle when the following traveling control is executed. In this case, the image displayed on the display device 25c is switched from the image IM1 (see FIG. 3) including the mark M1 indicating the preceding vehicle to the image IM2 (see FIG. 4) not including the mark M1 indicating the preceding vehicle.

  The detection processing unit 404 detects information related to the line of sight of the driver 302 of the vehicle 1 (see FIG. 5). For example, the detection processing unit 404 tracks the driver 302's face based on the image data obtained from the imaging device 201 and a three-dimensional face model stored in advance in a storage medium such as the SSD 14f, while driving the driver 302. The direction of the face, the position of the eyes, the direction in which the eyes are facing (line-of-sight direction), the movement of the eyes, and the like are detected. Note that the 3D face model is a statistical face shape model that includes the average 3D face shape of the subject, the position of facial parts such as the subject's eyes, mouth, and nose, and changes in the shape of the subject's eyes. It is.

  By the way, as described above, in the follow-up traveling control, for example, when the state in which the preceding vehicle is recognized is changed to the state in which the preceding vehicle is not present, the vehicle 1 is automatically The control contents are automatically switched so as to accelerate at. However, there are cases where it is inappropriate to automatically accelerate the vehicle 1, for example, when it is recognized that there is no preceding vehicle even though the preceding vehicle actually exists. In such a case, the driver 302 visually recognizes the situation around the vehicle 1 recognized by the recognition processing unit 401 using the display device 25c, and then visually observes the situation around the vehicle 1 to actively view the vehicle 1. Need to operate. Therefore, when the recognition status by the recognition processing unit 401 changes, it is desired to more reliably notify the driver 302 that the change in the recognition status has occurred.

  Therefore, the output processing unit 405 according to the first embodiment, when the recognition status by the recognition processing unit 401 changes, in order to urge the driver 302 to view the display device 25c, is different from the display device 25c. The driver 302 is notified that the recognition status has changed. Then, the output processing unit 405 monitors the detection result of the detection processing unit 404 and determines whether or not the driver 302 has actually viewed the display device 25c.

  Specifically, when the recognition status changes, the output processing unit 405 uses the audio output device 9 of the navigation device 11 to notify the driver 302 that the recognition status has changed using a sound. Output. In addition, the output processing unit 405 monitors the detection result by the detection processing unit 404 after outputting the first notification, and outputs a voice when the line of sight of the driver 302 is not detected facing the display device 25c. Using the device 9, a second notification for notifying the driver 302 again that the recognition status has changed is output with sound. Then, the output processing unit 405 repeatedly outputs the second notification until it is detected that the line of sight of the driver 302 is facing the display device 25c.

  The first notification is a voice message or a beep sound that prompts the driver 302 to view the display device 25c. The second notification is a voice message or a beep sound that prompts the driver 302 to visually recognize the display device 25c. As described above, the output processing unit 405 outputs the second notification in an output form different from that of the first notification. In other words, the second notification is output as a precaution when the first notification fails to direct the driver 302's line of sight toward the display device 25c. If the output is performed in the same output form as the notification, the driver 302 may not be able to attract the attention of the driver 302 sufficiently. Therefore, the output processing unit 405 outputs the second notification, for example, at a volume higher than that of the first notification, or with a voice pattern different from that of the first notification (that is, alerting more than the first notification). (With a strong voice pattern).

  In addition, when the output processing unit 405 outputs the second notification two or more times, the second notification after the second time is referred to as the second notification (for example, the first second notification) output in the past. Output in different output formats. That is, in order to increase the alerting power of the second notification for the second time or later than the second notification for the first time, the output processing unit 405 transmits the second notification for the second time or later, for example, the first notification for the first time. The second notification is output at a volume larger than the second notification, or the second and subsequent second notifications are output with a voice pattern having a higher alerting power than the first second notification.

  Next, with reference to FIG. 8, the process which ECU14 by 1st Embodiment performs is demonstrated.

  In the processing flow of FIG. 8, first, in S1, the recognition processing unit 401 determines whether or not the recognition state around the vehicle 1 has changed. The process of S1 is repeated until it is determined that the recognition status has changed. If it is determined in S1 that the recognition status has changed, the process proceeds to S2.

  In S <b> 2, the display processing unit 403 displays a change in the recognition status on the display device 25 c of the instrument panel 25, and the output processing unit 405 sends a first notification that notifies that the recognition status has changed to the navigation device 11. The sound is output through the sound output device 9. In step S <b> 3, the output processing unit 405 monitors the detection result by the detection processing unit 404.

  In S <b> 4, the output processing unit 405 determines whether or not the second notification that notifies the driver 302 again that the recognition status has changed has already been output.

  In S4, when the second notification has not been output yet, the process proceeds to S5. In S5, the output processing unit 405 determines whether or not it is detected that the line of sight of the driver 302 is facing the display device 25c based on the monitoring result in S3.

  In S5, when it is detected that the line of sight of the driver 302 is facing the display device 25c, it can be estimated that the driver 302 has visually recognized the display device 25c, so that the driver 302 can further visually recognize the display device 25c. There is no need to prompt the user to complete the process. However, if it is not detected in S5 that the driver's 302 is looking toward the display device 25c, the driver 302 needs to be prompted again to view the display device 25c.

  Therefore, in S6, the output processing unit 405 outputs a second notification for notifying the driver 302 again that the recognition status has changed in an output form different from the first notification output in S2, in the navigation device 11. The sound is output through the sound output device 9. For example, the output processing unit 405 outputs the second notification at a volume higher than that of the first notification, or outputs the second notification with an audio pattern different from that of the first notification. Then, the process returns to S3.

  If it is determined in S4 that the second notification has already been output at least once, the process proceeds to S7. In S7, the output processing unit 405 determines whether or not it is detected that the line of sight of the driver 302 is facing the display device 25c based on the monitoring result in S3.

  In S7, when it is detected that the line of sight of the driver 302 is facing the display device 25c, it is not necessary to prompt the driver 302 to view the display device 25c any more, so the processing ends. However, if it is not detected in S7 that the driver's line of sight is facing the display device 25c, it cannot be said that the driver 302 has visually recognized the display device 25c, and therefore the driver 302 visually recognizes the display device 25c. You need to prompt again.

  Therefore, in S <b> 8, the output processing unit 405 outputs the second notification again with sound via the audio output device 9. The output processing unit 405 changes the output form of the second notification output in S8 from the output form of the second notification output in the past (for example, the second notification output in S6). Then, the process returns to S3.

  As described above, when the recognition status by the recognition processing unit 401 changes, the display processing unit 403 according to the first embodiment displays the change in the recognition status on the display device 25c of the instrument panel 25. In addition, when the recognition status changes, the output processing unit 405 uses a notification unit (voice output device 9) different from the display device 25c to notify the driver 302 that the recognition status has changed. Output the notification. Then, after outputting the first notification, the output processing unit 405 monitors the detection result by the detection processing unit 404, and when it is not detected that the line of sight of the driver 302 is facing the display device 25c, an audio output is performed. Using the device 9, a second notification for notifying the driver 302 again that the recognition status has changed is output. Thereby, the driver 302 can be made to visually recognize the change in the recognition state more reliably using the first notification and the second notification.

  Further, the output processing unit 405 according to the first embodiment repeatedly outputs the second notification until it is detected that the line of sight of the driver 302 is facing the display device 25c. Thereby, it is possible to make the driver 302 visually recognize the change in the recognition status.

  In addition, the output processing unit 405 according to the first embodiment outputs the second notification in an output form different from the first notification, and outputs the second notification two or more times. The second notification is output in a different output form from the second notification output in the past. Thereby, for example, the driver 302 can be made to recognize the change in the recognition state more reliably by increasing the alerting power of the notification that is output later.

(Second Embodiment)
Next, the configuration of the ECU 214 according to the second embodiment will be described with reference to FIG. In the second embodiment, the condition for stopping the output of the second notification after the second time is only one condition that the line of sight of the driver 302 faces the display device 25c. Differently, the conditions for stopping the output of the second notification after the second time are that the line of sight of the driver 302 is directed to the display device 25c and the number of times of output of the second notification is more than the preset number. There are two things:

  In other words, when the output processing unit 2405 according to the second embodiment outputs the second notification two or more times, the output processing unit 2405 outputs the second number of times after the first second notification is output. Until the second notification is repeatedly output. More specifically, when the output processing unit 2405 outputs the second notification more than once, after the first second notification is output, the line of sight of the driver 302 faces the display device 25c. Even if this is not detected, when the number of times of output of the second notification is equal to or greater than the preset number, output of the second notification is stopped from the viewpoint of annoyance. Note that the number of times to be the output stop condition of the second notification may be set in advance as an experimentally or ergonomically determined number, or may be arbitrarily set by the driver 302 himself. .

  In addition, since the other structure of 2nd Embodiment is the same as that of 1st Embodiment mentioned above, description is abbreviate | omitted here.

  Next, a process executed by the ECU 214 according to the second embodiment will be described with reference to FIG. In 2nd Embodiment, the process of S1-S8 similar to 1st Embodiment mentioned above (refer FIG. 8) is performed. In the second embodiment, the criterion for determining whether or not to output the second notification again is a criterion other than the criterion of whether or not the driver's 302 is looking toward the display device 25c (S7). It differs from the first embodiment in that it exists.

  That is, in the processing flow according to the second embodiment (see FIG. 9), if it is determined in S7 that the driver's 302 eyes are not facing the display device 25c, the processing proceeds to S7a. In step S <b> 7 a, the output processing unit 2405 determines whether or not the number of times of output of the second notification that has already been output is greater than or equal to a preset number. The preset number of times is the number of times that becomes a branch point of whether or not the driver 302 feels excessive annoyance, and may be an experimentally or ergonomically determined number. 302 may be an arbitrarily set number.

  In S7a, when it is determined that the number of times of output of the second notification is less than a preset number of times, even if the second notification is notified again, the driver 302 does not feel excessive annoyance. . Therefore, in this case, the second notification is output again in S8, and the process returns to S4.

  On the other hand, if it is determined in S7a that the number of times of output of the second notification is equal to or greater than the preset number of times, if the second notification is output again, the driver 302 will feel excessive annoyance. . Therefore, in this case, the process ends without outputting the second notification again.

  As described above, in the second embodiment, when the second notification is output twice or more, the driver's 302's line of sight faces the display device 25c after the first second notification is output. If the number of times the second notification is output is less than a preset number, the second notification is output again. As a result, it is possible to suppress the driver 302 from feeling excessive annoyance due to the repeated output of the second notification.

  In addition, since the other effect of 2nd Embodiment is the same as that of 1st Embodiment, description is abbreviate | omitted here.

(Third embodiment)
Next, the configuration of the ECU 314 according to the third embodiment will be described with reference to FIG. In the third embodiment, the second condition is that there is a condition other than the line of sight of the driver 302 facing the display device 25c as a condition for stopping the output of the second notification after the second time. Although it is the same as that of embodiment, the content of the said other conditions differs from 2nd Embodiment.

  Specifically, when the output processing unit 3405 according to the third embodiment outputs the second notification more than once, after the first second notification is output, the driver 302's line of sight indicates the display device 25c. Until the driver 302 detects that the driver 302 has performed a predetermined active eye operation, and repeatedly outputs the second notification. The predetermined active eye operation is, for example, blinking actively performed at a predetermined number of times and at predetermined intervals. In other words, in the third embodiment, in order to avoid the second notification being output twice or more, the driver 302 looks at the display device 25c and then performs a predetermined eye operation such as blinking. Must be actively performed.

  In addition, since the other structure of 3rd Embodiment is the same as that of 2nd Embodiment mentioned above, description is abbreviate | omitted here.

  Next, a process executed by the ECU 314 according to the third embodiment will be described with reference to FIG. Also in the third embodiment, similarly to the second embodiment, the same processes of S1 to S6 and S8 as in the first embodiment (see FIG. 8) are executed. However, in the third embodiment, when the second notification is output twice or more (in the case of Yes in S4), the second notification is output again (S8) on the basis different from the second embodiment. Determine whether to do it.

  That is, in the processing flow according to the third embodiment (see FIG. 10), when it is determined in S4 that the second notification has already been output at least once, the processing proceeds to S7b. In S7b, the output processing unit 3405 detects that the driver's line of sight is facing the display device 25c, and that the driver 302 has performed a predetermined active eye operation. It is determined whether or not it has been detected. The predetermined active eye operation is, for example, blinking actively performed at a predetermined number of times and at predetermined intervals.

  In S7b, when it is detected that the line of sight of the driver 302 is facing the display device 25c, and it is detected that the driver 302 has performed a predetermined active eye operation, the driver 302 is no more. Since it is no longer necessary to prompt the user to visually recognize the display device 25c, the processing ends. However, if it is not detected in S7 that the driver's line of sight is directed to the display device 25c, or even if it is detected that the driver's line of sight is directed to the display device 25c, the driver 302 If it is not detected that a predetermined active eye operation has been performed, it cannot be said that the driver 302 has actively and reliably visually recognized the display device 25c. Therefore, in this case, in S8, the output processing unit 3405 outputs the second notification again to prompt the driver 302 to visually recognize the display device 25c, and the process returns to S4.

  As described above, the output processing unit 3405 according to the third embodiment displays the line of sight of the driver 302 after the second notification is output for the first time when the second notification is output twice or more. The second notification is repeatedly output until it is detected that it is facing the device 25c and it is detected that the driver 302 has performed a predetermined active eye operation. The predetermined active eye operation is, for example, blinking actively performed at a predetermined number of times and at predetermined intervals. As a result, it is possible to more reliably determine whether the driver 302 has actively looked at the display device 25c.

  The remaining effects of the third embodiment are the same as those of the second embodiment, and a description thereof will be omitted here.

(Modification)
In the embodiment described above, an example in which each module in FIG. 7 is realized by cooperation of software (computer program) and hardware (CPU 14a) has been described. However, each module in FIG. 7 may be realized only by hardware. That is, in the embodiment, dedicated hardware (circuit) corresponding to each module in FIG. 7 may be provided.

  In the above-described embodiment, an example is shown in which control is executed based on the line of sight of the driver 302, but the face orientation may be used instead of the line of sight.

  In the above-described embodiment, the example in which the first notification and the second notification for notifying the driver 302 that the recognition state has changed is output from the voice output device 9 as voice. However, the first notification and the second notification may be output as light or vibration. In addition, the first notification and the second notification may be output from a speaker device other than the audio output device 9 provided in the navigation device 11. In addition, the first notification and the second notification may be output as an image IM3 illustrated in FIG. This image IM3 may be output simultaneously with the sound for notifying the driver 302 that the recognition status has changed.

  Here, the image IM3 in FIG. 11 is an image output to the display device 8 of the navigation device 11 and includes a message indicating that a change has occurred in the recognition status. In the above-described embodiment, an example in which the repeated output of the second notification is stopped on the condition that it is detected that the line of sight of the driver 302 faces the display device 25c of the instrument panel 25 is one of the conditions. Indicated. However, when the image IM3 illustrated in FIG. 11 is output as the first notification and the second notification, it is detected that the line of sight of the driver 302 is facing the display device 8 of the navigation device 11. With this as one of the conditions, the repeated output of the second notification may be stopped.

  In the above-described embodiment, the processing in the case where the follow-up running control as an example of the driving support control is executed has been mainly described. However, as will be described below, even when parking assistance control as another example of driving assistance control is executed, there is a situation in which the driver 302 wants to confirm a change in the recognition situation. For example, if a cautionary object that was not recognized at the start of parking is newly recognized in the middle, or conversely, a cautionary object that has already been recognized at the start of parking is no longer recognized in the middle It is necessary for the driver 302 to check the change in the situation around the vehicle. Hereinafter, with reference to FIGS. 12-19, the change of the recognition condition in parking assistance control and the process performed when the said change generate | occur | produces are demonstrated more concretely.

  FIGS. 12 to 15 show parking assistance as an example of a parking assistance image displayed on the display device 8 of the navigation device 11 when the parking assistance control is executed. An image is shown. FIGS. 16 to 19 show parking assistance images in which the situation around the vehicle 1 (the vicinity of the parking place) is viewed from the rear view of the vehicle 1 as another example of the parking assistance image. In addition, the mark M2 corresponding to the vehicle 1 is displayed in the overhead parking assistance image illustrated in FIGS.

  FIG. 12 and FIG. 13 show specific examples of changes in the overhead parking assistance image when a cautionary object that has not been recognized at the start of parking is newly recognized on the way. In this specific example, when another vehicle as the attention object is newly recognized, the image IM4 (see FIG. 12) at the start of parking includes an image M3 indicating the newly recognized other vehicle. It is switched to IM5 (see FIG. 13).

  14 and 15 show a specific example of a change in the overhead parking assistance image when the attention object that has already been recognized at the start of parking is no longer recognized in the middle. In this specific example, when a person who has already been recognized as a target object is no longer recognized in the middle, the image IM6 (see FIG. 12) at the start of parking including the mark M4 indicating the recognized person is displayed with the mark M4. It is switched to the image IM7 (see FIG. 13) that is not included.

  FIGS. 16 and 17 show specific examples of changes in the parking assistance image in the rear visual field when a cautionary object that has not been recognized at the start of parking is newly recognized in the middle. In this specific example, when a person as an attention object is newly recognized, the image IM8 (see FIG. 16) at the start of parking includes an image IM9 (FIG. 17) including a mark M5 indicating the newly recognized person. To see).

  FIG. 18 and FIG. 19 show specific examples of changes in the parking assistance image in the rear view when the attention object that has already been recognized at the start of parking is no longer recognized in the middle. In this specific example, an image IM10 (see FIG. 18) at the start of parking including a mark M6 indicating another recognized vehicle is detected when another vehicle that has already been recognized as a cautionary object is not recognized. , The image IM11 does not include the mark M6 (see FIG. 19).

  As described above, when the parking assistance image is switched in accordance with the change in the recognition state, it is necessary to notify the driver 302 of the change in the recognition state and to confirm the presence or absence of the attention object. Therefore, even when the recognition status changes during the execution of the parking assistance control, the first notification and the second notification are output as in the case where the recognition status changes during the execution of the follow-up running control. It should be noted that at the time of executing the parking assist control, in addition to detecting that the driver's line of sight has visually recognized the display device 8 (the above-described images IM4 to IM11), the driver's 302's line of sight is the attention object. It may be set as a condition for stopping the output of the first notification and the second notification. For example, in the example of FIG. 13, the first notification is made when it is detected that the line of sight of the driver 302 is directed in a direction in which another vehicle indicated by the mark M3 is considered to be located (that is, right rear of the vehicle 1). The condition may be set such that the output of the second notification or the like is stopped. Similarly, in the example of FIG. 17, when it is detected that the line of sight of the driver 302 is directed in the direction in which the person indicated by the mark M5 is supposed to be located (that is, the left rear of the vehicle 1), Condition setting may be performed so that output of the second notification or the like is stopped. If such condition setting is performed, the driver 302 can be more surely checked for the presence or absence of the target object.

  As mentioned above, although embodiment and the modification of this invention were described, embodiment and the modification which were mentioned above are an example to the last, Comprising: It is not intending limiting the range of invention. The above-described embodiments and modifications can be implemented in various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

1 Vehicle 9 Audio output device (notification unit)
14, 214, 314 ECU (driving support device)
25c display device 401 recognition processing unit 402 driving support processing unit 403 display processing unit 404 detection processing unit 405, 2405, 3405 output processing unit

Claims (6)

A recognition processor that recognizes the situation around the vehicle;
A driving support processing unit that executes driving support control according to a situation around the vehicle recognized by the recognition processing unit;
A display processing unit for displaying a change in the situation around the vehicle on a display unit when the situation around the vehicle changes;
A detection processing unit for detecting information related to the line of sight of the driver of the vehicle;
When a situation around the vehicle has changed, a notification unit different from the display unit is used to output a first notification that notifies the driver that the situation around the vehicle has changed, After outputting the first notification, based on the detection result by the detection processing unit, when it is not detected that the driver's line of sight is facing the display unit, the notification unit is used to A driving support device comprising: an output processing unit that outputs a second notification that notifies the driver again that the surrounding situation has changed.   The driving support device according to claim 1, wherein the output processing unit repeatedly outputs the second notification until it is detected that the driver's line of sight is facing the display unit.   The output processing unit outputs the second notification in an output form different from that of the first notification, and outputs the second notification two or more times when the second notification is output twice or more. The driving support device according to claim 2, wherein the output is different from the second notification output in the past.   The output processing unit, when outputting the second notification two or more times, after outputting the second notification for the first time, repeatedly outputs the second notification up to a preset number of times. The driving support device according to claim 2 or 3.   When the output processing unit outputs the second notification more than once, after the first notification of the second notification is output, the driver's line of sight is directed to the display unit by the detection processing unit. 4. The driving according to claim 2, wherein the second notification is repeatedly output until it is detected that the driver has performed a predetermined active eye operation. 5. Support device. The notification unit includes a sound output unit capable of outputting sound,
The driving support device according to any one of claims 1 to 5, wherein the output processing unit outputs the first notification and the second notification by sound using the sound output unit.

Images