WO2022259410A1

WO2022259410A1 - Notification control device, information distribution server, and notification control method

Info

Publication number: WO2022259410A1
Application number: PCT/JP2021/021892
Authority: WO
Inventors: 哲木谷; 光生下谷
Original assignee: 三菱電機株式会社
Priority date: 2021-06-09
Filing date: 2021-06-09
Publication date: 2022-12-15
Also published as: JPWO2022259410A1; JP7466776B2

Abstract

The purpose of the present disclosure is to provide a notification control device, information distribution server, and notification control method, which are capable of appropriately notifying a driver of a hazardous situation not recognized by the driver. The notification control device according to the present disclosure comprises: a driver state recognition unit; a surrounding situation recognition unit; a host vehicle driving information acquisition unit; an other vehicle driving information acquisition unit; a control unit that uses the state of the driver of the host vehicle, a surrounding situation thereof, host vehicle driving information, and other vehicle driving information to determine whether or not the host vehicle has performed a hazard avoidance operation by autonomous driving or another vehicle has performed a hazard avoidance operation when a hazardous situation has occurred and the driver of the host vehicle is not recognizing the hazardous situation during traveling of the host vehicle; and a notification unit that notifies, if it is determined that the host vehicle has performed a hazard avoidance operation by autonomous driving or another vehicle has performed a hazard avoidance operation, the driver of the host vehicle of the hazardous situation at a time when the host vehicle has performed a hazard avoidance operation by autonomous driving or another vehicle has performed a hazard avoidance operation.

Description

Notification control device, information distribution server, and notification control method

The present disclosure provides a notification control device and a notification control method for notifying the driver of a dangerous situation that occurred while driving when the driver did not recognize it, and a notification control method for collecting information from the notification control device. and an information distribution server that distributes various information to a notification control device.

Conventionally, technologies have been disclosed for informing the driver of various types of information when a dangerous situation occurs due to high-risk driving and the driver is unaware of the dangerous situation ( For example, see Patent Documents 1 and 2).

JP-A-2007-65997 JP 2010-15451 A

In the conventional technology, when the driver is unaware of the occurrence of a dangerous situation, the driver is in a dangerous situation when the self-driving self-driving vehicle performs a risk avoidance action or another vehicle performs a risk avoidance action. are not aware of Here, the danger avoidance operation is an operation for avoiding a dangerous situation when the dangerous situation occurs. Therefore, the driver may again find himself in a similar dangerous situation. As described above, conventionally, it cannot be said that the driver is appropriately notified of a dangerous situation that the driver has not recognized.

The present disclosure has been made in order to solve such problems, and includes a notification control device, an information distribution server, and An object of the present invention is to provide an information control method.

In order to solve the above problems, the notification control device according to the present disclosure includes a driver state recognition unit that recognizes the state of the driver of the own vehicle, a surrounding situation recognition unit that recognizes the surrounding situation of the own vehicle, and An own vehicle driving information acquisition unit that acquires own vehicle driving information that is information related to the driving of another vehicle, an other vehicle driving information acquisition unit that acquires other vehicle driving information that is information related to the driving of another vehicle, and a driver state recognition unit The state of the driver of the own vehicle recognized, the surrounding situation recognized by the surrounding situation recognition unit, the own vehicle driving information acquired by the own vehicle driving information acquisition unit, and the other vehicle driving information acquired by the other vehicle driving information acquisition unit Based on this, when a dangerous situation occurs while the own vehicle is running and the driver of the own vehicle is not aware of the dangerous situation, the danger avoidance action by the self-driving vehicle or the danger avoidance action by other vehicles is performed. and if the control unit determines that the risk avoidance action by the self-driving vehicle or the risk avoidance action by the other vehicle has been performed, the risk avoidance action by the self-driving car or the other vehicle is performed. a notification unit that notifies the driver of the own vehicle of a dangerous situation when the danger avoidance operation is performed by the vehicle.

According to the present disclosure, when a dangerous situation occurs while the own vehicle is traveling and the driver of the own vehicle is unaware of the dangerous situation, a danger avoidance action by automatic driving of the own vehicle or a danger avoidance action by another vehicle is performed. has been performed, the driver of the own vehicle is notified of the danger situation when the danger avoidance action by the self-driving vehicle or the danger avoidance action by another vehicle is performed. It is possible to appropriately notify the driver of the situation.

The objects, features, aspects, and advantages of the present disclosure will become more apparent with the following detailed description and accompanying drawings.

1 is a block diagram showing an example of the configuration of a notification control device according to Embodiment 1; FIG. 1 is a block diagram showing an example of the configuration of a notification control device according to Embodiment 1; FIG. 4 is a flow chart showing an example of the operation of the notification control device according to Embodiment 1; 4 is a diagram showing an example of notification according to Embodiment 1; FIG. FIG. 9 is a block diagram showing an example of the configuration of a notification control device according to Modification 2 of Embodiment 1; FIG. 10 is a diagram showing an example of notification according to Modification 2 of Embodiment 1; FIG. 12 is a diagram showing an example of notification according to Modification 3 of Embodiment 1; FIG. 13 is a diagram showing an example of notification according to Modification 4 of Embodiment 1; FIG. 13 is a diagram showing an example of notification according to Modification 4 of Embodiment 1; FIG. 9 is a block diagram showing an example of the configuration of a notification control device according to Embodiment 2; 9 is a flow chart showing an example of the operation of the notification control device according to Embodiment 2; 1 is a diagram showing an example of a hardware configuration of a notification control device according to Embodiment 1; FIG. 1 is a diagram showing an example of a hardware configuration of a notification control device according to Embodiment 1; FIG.

<Embodiment 1>
<Configuration>
FIG. 1 is a block diagram showing an example of the configuration of the notification control device 1 according to Embodiment 1. As shown in FIG. It should be noted that FIG. 1 shows the minimum necessary configuration of the notification control device according to the first embodiment.

The notification control device 1 includes a driver state recognition unit 2, a surrounding situation recognition unit 3, an own vehicle driving information acquisition unit 4, an other vehicle driving information acquisition unit 5, a control unit 6, and a notification unit 7. ing.

The driver state recognition unit 2 recognizes the state of the driver of the own vehicle. The surrounding situation recognition unit 3 recognizes the surrounding situation of the own vehicle.

The own vehicle driving information acquisition unit 4 acquires own vehicle driving information, which is information related to the driving of the own vehicle. The other-vehicle driving information acquisition unit 5 acquires other-vehicle driving information, which is information relating to the driving of the other vehicle.

The control unit 6 recognizes the state of the driver of the own vehicle recognized by the driver state recognition unit 2, the surrounding situation recognized by the surrounding situation recognition unit 3, and the own vehicle driving information acquired by the own vehicle driving information acquisition unit 4. , and the other vehicle driving information acquired by the other vehicle driving information acquisition unit 5, when a dangerous situation occurs while the own vehicle is traveling and the driver of the own vehicle is not aware of the dangerous situation, the own vehicle It is determined whether or not a risk avoidance action by automatic driving or a risk avoidance action by another vehicle has been performed.

When the control unit 6 determines that the danger avoidance action by the self-driving of the own vehicle or the danger avoidance action by the other vehicle is performed, the notification part 7 performs the danger avoidance action by the self-driving of the self-vehicle or the danger avoidance action by the other vehicle. It notifies the driver of the own vehicle of the dangerous situation when it is broken.

Next, another configuration of the notification control device including the notification control device 1 shown in FIG. 1 will be described.

FIG. 2 is a block diagram showing an example of the configuration of the notification control device 8 according to another configuration.

The notification control device 8 includes a driver state recognition unit 2, a surrounding situation recognition unit 3, an own vehicle driving information acquisition unit 4, an other vehicle driving information acquisition unit 5, a control unit 6, a notification unit 7, a situation and a storage unit 9 . The notification control device 8 is connected to the driver state detection device 10 , the surrounding situation detection device 11 , the photographing device 12 , the automatic driving control device 13 , the communication device 14 and the notification device 15 . The notification control device 8, the driver state detection device 10, the surrounding situation detection device 11, the imaging device 12, the automatic driving control device 13, the communication device 14, and the notification device 15 are mounted on the own vehicle. The self-vehicle is not limited to a drive system such as an engine-driven vehicle, an electric vehicle, or a fuel cell vehicle.

The notification control device 8 is an in-vehicle navigation device, that is, not only a car navigation device, but also a PND (Portable Navigation Device) that can be mounted on the vehicle, and a server provided outside the vehicle. The present invention can also be applied to a built navigation device or a device other than a navigation device built as a system. In this case, each function or each component of the notification control device 8 is distributed to each function that constructs the above system.

The driver state recognition unit 2 acquires the detection result of the state of the driver of the own vehicle from the driver state detection device 10, and recognizes the state of the driver of the own vehicle based on the detection result. Further, the driver state recognition unit 2 determines whether or not the driver of the own vehicle recognizes the dangerous situation when a dangerous situation occurs while the own vehicle is running. When determining whether or not the driver of the vehicle recognizes a dangerous situation, the driver state recognition unit 2 detects not only the detection result obtained from the driver state detection device 10 but also the surrounding situation detection device 11. You may use the detection result which carried out.

For example, the driver state recognizing unit 2 recognizes the line of sight of the driver of the own vehicle detected by the driver state detection device 10 and the other vehicle present in the vicinity of the own vehicle detected by the surrounding situation detection device 11. Based on the position, if another vehicle exists in front of the line of sight of the driver of the own vehicle, it is recognized that the driver of the own vehicle has recognized a dangerous situation. The other vehicle detected by the surrounding situation detection device 11 at this time is a vehicle involved in the dangerous situation when the dangerous situation occurs. On the other hand, if there is no other vehicle in the line of sight of the driver of the own vehicle, it is recognized that the driver of the own vehicle does not recognize the dangerous situation. The line of sight of the driver of the vehicle detected by the driver state detection device 10 includes not only the direct line of sight, which is the actual line of sight, but also the line of sight when looking at the rear view mirror or the electronic mirror. .

The driver state detection device 10 is, for example, a DMS (Driver Monitoring System) and detects the state of the driver of the own vehicle. The driver state detection device 10 detects the state of the driver of the vehicle during driving, such as the line of sight, arousal (physiology), expression (psychology), and line of sight of the driver of the vehicle. The driver's state detection device 10 outputs the detection result of the driver's state of the own vehicle to the driver's state recognition unit 2 .

The surrounding situation recognition unit 3 acquires the detection result of the surrounding situation of the own vehicle from the surrounding situation detection device 11, and recognizes the surrounding situation of the own vehicle based on the detection result. In addition, the surrounding situation recognition unit 3 recognizes a dangerous situation that has occurred while the host vehicle is traveling, based on the detection result acquired from the surrounding situation detection device 11 .

The surrounding situation detection device 11 detects the surrounding situation of the own vehicle. The peripheral situation detection device 11 detects various peripheral situations related to traffic conditions, including, for example, the distance and relative positional relationship between the host vehicle and other vehicles.

The own vehicle driving information acquisition unit 4 acquires own vehicle driving information, which is information related to the driving of the own vehicle. Specifically, the own vehicle driving information acquisition unit 4 acquires information about automatic driving of the own vehicle as the own vehicle driving information from the automatic driving control device 13 . The own vehicle driving information acquisition unit 4 also acquires information on manual driving by the driver of the own vehicle as own vehicle driving information.

The automatic driving control device 13 performs automatic driving by controlling the travel control system of the own vehicle. Further, the automatic driving control device 13 outputs information to the own vehicle driving information acquiring unit 4 to the effect that a danger avoidance action has been performed by intervening the driving operation when a dangerous situation occurs while the own vehicle is running. Here, the risk avoidance action by driving operation intervention is to automatically switch from manual driving by the driver to automatic driving by the automatic driving control device 13 when a dangerous situation occurs during manual driving by the driver of the own vehicle. It means to perform danger avoidance action. In addition, the automatic driving control device 13 may perform the danger avoiding operation when a dangerous situation occurs during automatic driving by the automatic driving control device 13, without being limited to the danger avoiding operation by the intervention of the driving operation. In this case, the automatic driving control device 13 outputs information to the effect that the danger avoidance operation has been performed to the own vehicle driving information acquiring section 4 . In this way, the own vehicle driving information includes information indicating that the own vehicle has been automatically driven to avoid danger, and information indicating that the own vehicle has been manually driven to avoid danger. include.

The other-vehicle driving information acquisition unit 5 acquires other-vehicle driving information, which is information related to the driving of the other vehicle. Specifically, the other-vehicle driving information acquiring unit 5 acquires other-vehicle driving information, which is information related to the driving of the other vehicle, from the other vehicle via the communication device 14 . The other vehicle driving information includes information indicating that the other vehicle has performed a danger avoidance action. Examples of danger avoidance actions by other vehicles include danger avoidance actions by automatic driving of other vehicles, danger avoidance actions by drivers of other vehicles, and the like. Here, the danger avoidance operation by the automatic driving of the other vehicle refers to the operation of avoiding the dangerous situation by the automatic driving of the other vehicle under the control of the automatic driving control device mounted on the other vehicle. The danger avoidance action by the driver of the other vehicle refers to the action of the other vehicle avoiding a dangerous situation by manually driving the other vehicle.

Specifically, when the surrounding situation recognition unit 3 recognizes that a dangerous situation has occurred while the own vehicle is traveling, the control unit 6 determines that a dangerous situation has occurred while the own vehicle is traveling. Further, the control unit 6 determines that the driver of the vehicle does not recognize the dangerous situation when the driver state recognition unit 2 recognizes that the driver of the own vehicle does not recognize the dangerous situation.

Furthermore, based on the own vehicle driving information acquired by the own vehicle driving information acquiring unit 4 and the other vehicle driving information acquired by the other vehicle operating information acquiring unit 5, the control unit 6 automatically drives the own vehicle to avoid danger. It is determined whether or not a movement or a danger avoidance action by another vehicle has been performed.

The danger avoidance action by automatic driving of the own vehicle means the action of avoiding the dangerous situation by automatic driving under the control of the automatic driving control device 13 when a dangerous situation occurs. It is assumed that the danger avoidance operation by automatic driving of the own vehicle is performed, for example, in the following situations. It should be noted that the following is merely an example, and is not limited to the following as long as danger avoidance actions are performed by automatic driving of the own vehicle.
・When the distance between the host vehicle and other objects (moving objects including other vehicles, or non-moving objects) becomes equal to or less than a predetermined distance, the automatic driving control device 13 performs one of the brake, accelerator, and steering wheel operations. automatically activates at least one of to avoid a dangerous situation.
When the vehicle is about to ignore the red light, the automatic driving control device 13 automatically activates at least one of brake, accelerator, and steering wheel operation to avoid the dangerous situation.
・When the driver of the own vehicle opens the door, a moving object is approaching from behind, and the moving object is expected to collide with the door of the own vehicle or the driver. automatically activates at least one of braking, accelerating and steering to avoid a dangerous situation.

Note that the own vehicle driving information acquisition unit 4 may acquire information related to automatic driving of the own vehicle by a method other than acquiring the own vehicle driving information from the automatic driving control device 13 . For example, the control unit 6 compares the information that the driver has operated the actuator with the information of the speed sensor and the acceleration sensor that detect the physical movement of the vehicle, and determines whether the driver has operated the actuator. In spite of this, if the own vehicle performs at least one operation of steering, accelerating, and braking, it may be determined that the own vehicle is running in automatic driving mode. In this case, the own vehicle driving information acquisition unit 4 acquires the information that the driver has operated the actuator, and the information of the speed sensor and the acceleration sensor that detect the physical movement of the own vehicle.

A danger avoidance action by another vehicle is an action by which another vehicle avoids a dangerous situation when a dangerous situation occurs. It is assumed that danger avoidance actions by other vehicles are performed, for example, in the following situations. It should be noted that the following is merely an example, and is not limited to the following as long as danger avoidance operations are performed by other vehicles.
・When the distance between the own vehicle and the other vehicle is less than or equal to a predetermined distance, the other vehicle automatically drives to avoid a dangerous situation.
When the driver of the other vehicle avoids the dangerous situation when the distance between the own vehicle and the other vehicle is less than or equal to a predetermined distance.
・When a collision between the own vehicle and another vehicle other than the above is assumed, the dangerous situation is avoided by automatic driving of the other vehicle or by the driver of the other vehicle. An example where a collision between your vehicle and another vehicle is assumed is that the distance between your vehicle and the other vehicle is still far, but the driver of the other vehicle is asleep, etc., and a collision is possible if things continue as they are. may be highly sensitive. The state of the driver of the other vehicle can be detected by image recognition of the image of the driver of the other vehicle captured by the surrounding situation detection device 11 or the imaging device 12 . Further, as an example in which a collision between one's own vehicle and another vehicle is assumed, there is a case where the other vehicle is in a runaway state although the distance between the own vehicle and the other vehicle is still large.
・When a collision between the driver of the own vehicle and another vehicle is assumed, the dangerous situation is avoided by automatic driving of the other vehicle or by the driver of the other vehicle.
・When a collision between the own vehicle and the driver of another vehicle is assumed, the dangerous situation is avoided by automatic driving of the other vehicle or by the driver of the other vehicle.

It should be noted that the other-vehicle driving information acquisition unit 5 may acquire information about the driving of the other vehicle by a method other than acquiring the other-vehicle driving information from the other vehicle via the communication device 14 . For example, the control unit 6 may estimate the motion related to the driving of the other vehicle based on the surrounding conditions detected by the surrounding condition detection device 11 . In addition, the control unit 6 detects the motion of the driver of the other vehicle by image recognition of the image of the driver of the other vehicle captured by the surrounding situation detection device 11 or the imaging device 12, and detects the motion of the driver of the other vehicle. If the actuator is not operated, it may be determined that the other vehicle is automatically driving.

Returning to the description of FIG. 2, the situation storage unit 9 stores the video of the dangerous situation captured by the imaging device 12. The video imaged by the imaging device 12 is stored in the situation storage unit 9 in a format reproducible by a television or video/audio reproduction software. The video may be processed such as image processing, and the audio may be stored in the situation storage unit 9 together with the video. Note that the status storage unit 9 may be provided outside the notification control device 8 . In this case, the situation storage unit 9 and the photographing device 12 may be integrally configured, for example, like a drive recorder. In addition, the situation storage section 9 may store the surrounding situation of the host vehicle recognized by the surrounding situation recognition section 3 .

The photographing device 12 photographs dangerous situations that occur while the vehicle is traveling. Note that the photographing device 12 and the surrounding situation detection device 11 may be configured integrally.

When the control unit 6 determines that the danger avoidance action by the self-driving of the own vehicle or the danger avoidance action by the other vehicle is performed, the notification part 7 performs the danger avoidance action by the self-driving of the self-vehicle or the danger avoidance action by the other vehicle. It notifies the driver of the own vehicle of the dangerous situation when it is broken. Specifically, the notification unit 7 uses the notification device 15 to notify the driver of the host vehicle of the dangerous situation stored in the situation storage unit 9 .

The notification device 15 notifies the dangerous situation according to the instruction of the notification unit 7. Examples of the notification device 15 include a HUD (Head Up Display), a CID (Center Information Display), an LED (Light Emitting Diode), a speaker, and a device that vibrates the seat. Any notification means may be used as long as it can do so.

<Action>
FIG. 3 is a flow chart showing an example of the operation of the notification control device 8. As shown in FIG. In FIG. 3, it is assumed that the host vehicle is running at the start of the operation.

In step S11, the surrounding situation recognition unit 3 acquires the detection result of the surrounding situation of the own vehicle from the surrounding situation detection device 11, and recognizes the surrounding situation of the own vehicle based on the detection result.

In step S12, the driver state recognition unit 2 acquires the detection result of the state of the driver of the own vehicle from the driver state detection device 10, and recognizes the state of the driver of the own vehicle based on the detection result.

At step S13, the control unit 6 determines whether or not the driving of the own vehicle has ended. For example, when the vehicle is an engine-driven vehicle, the engine is turned off, or when the vehicle is an electric vehicle, the motor power for driving the vehicle is turned off. You may judge that driving|running was complete|finished. Further, the control unit 6 may determine that the driving of the own vehicle has ended when the own vehicle arrives at a preset destination. When the driving of the host vehicle has ended, the process proceeds to step S18. On the other hand, if the driving of the host vehicle has not ended, the process proceeds to step S14.

In step S14, the control unit 6 automatically drives the own vehicle based on the own vehicle driving information obtained by the own vehicle driving information obtaining unit 4 and the other vehicle driving information obtained by the other vehicle driving information obtaining unit 5. It is determined whether or not a danger avoidance action or a danger avoidance action by another vehicle has been performed. When the danger avoidance operation by the self-driving of the own vehicle or the danger avoidance operation by the other vehicle is performed, the process proceeds to step S15. On the other hand, if the risk avoidance action by the self-driving vehicle or the risk avoidance action by the other vehicle is not performed, the process returns to step S11.

At step S15, the control unit 6 determines whether or not the driver of the own vehicle has performed a danger avoidance action. Specifically, when the own vehicle driving information acquisition unit 4 acquires information indicating that the driver of the own vehicle has performed a risk avoidance operation by manual driving, the control unit 6 determines that the driver of the own vehicle is in danger. It is determined that an evasive action has been taken. Here, when the danger avoidance action is performed by the other vehicle, it is clear that the driver of the own vehicle is not performing the danger avoidance action. is executed only when That is, when the danger avoidance action by the other vehicle is performed, the process of step S15 is omitted and the process proceeds to step S16. If the driver of the own vehicle is not performing a danger avoidance action, the process proceeds to step S16. On the other hand, if the driver of the own vehicle has performed a danger avoidance action, the process returns to step S11.

In step S16, the driver state recognition unit 2 determines whether or not the driver of the vehicle recognizes a dangerous situation based on the detection result of the state of the driver of the vehicle acquired from the driver state detection device 10. to judge. If the driver of the own vehicle does not recognize the dangerous situation, the process proceeds to step S17. At this time, the driver state recognition unit 2 notifies the control unit 6 that the driver of the own vehicle did not recognize the dangerous situation. On the other hand, if the driver of the vehicle recognizes the dangerous situation, the process returns to step S11.

In step S<b>17 , the situation storage unit 9 stores the image of the dangerous situation captured by the imaging device 12 in accordance with the instruction from the control unit 6 . At this time, the situation storage section 9 may store the dangerous situations recognized by the surrounding situation recognition section 3 .

In step S18, if it is determined in step S13 that the vehicle has finished driving, the notification unit 7 notifies the driver of the vehicle of the dangerous situation stored in the situation storage unit 9 according to the instruction of the control unit 6. do. For example, as shown in FIG. 4, the notification contents may be displayed on the HUD, which is the notification device 15 . In the example of FIG. 4, it is notified that the danger avoidance motion has been detected. In addition, it is reported that the danger avoidance action is a sudden steering operation of another vehicle. In FIG. 4, the avoidance category indicates what kind of subject performed what kind of danger avoidance action.

It should be noted that the timing of notifying the driver of the own vehicle is not limited to the end of driving the own vehicle. For example, the notification may be made after a predetermined period of time has elapsed since the occurrence of the dangerous situation, and when the driving load around the own vehicle is low, for example, when the density of other vehicles existing around the own vehicle is low. The notification may be made when the density becomes equal to or less than a predetermined density, or the notification may be made when a sufficient inter-vehicle distance between the own vehicle and the other vehicle is obtained. Further, after the driving operation is finished once, the notification may be made when the driving is started next time.

<Modification 1>
Although the case where both the own vehicle driving information and the other vehicle driving information are obtained has been described above, only one of the information may be obtained. For example, when acquiring only own vehicle driving information, the other vehicle driving information acquisition unit 5 and the communication device 14 shown in FIG. 2 are unnecessary.

<Modification 2>
FIG. 5 is a block diagram showing an example of the configuration of the notification control device 16 according to Modification 2. As shown in FIG.

The notification control device 16 is characterized by having a position information acquisition section 17 and a map information acquisition section 18 . The position information acquisition unit 17 is connected to a positioning device 19 , and the map information acquisition unit 18 is connected to a map DB (database) storage device 20 . The positioning device 19 and the map DB storage device 20 are mounted on the own vehicle. Since other configurations and basic operations are the same as those of the notification control device 8 shown in FIG. 2, detailed description thereof will be omitted here.

The position information acquisition unit 17 acquires current position information of the own vehicle from the positioning device 19 . The positioning device 19, for example, receives signals from GPS (Global Positioning System) satellites and measures the current position of the vehicle.

The map information acquisition unit 18 acquires map information including the road on which the vehicle is traveling from the map DB storage device 20. The map DB storage device 20 stores map information including roads. Note that the map DB storage device 20 may be provided outside the host vehicle.

The situation storage unit 9 stores the position information where the dangerous situation occurred based on the current position information of the own vehicle acquired by the position information acquisition unit 17 and the map information acquired by the map information acquisition unit 18. The situation storage unit 9 associates and stores at least one of the position information where the dangerous situation has occurred, the image of the dangerous situation captured by the imaging device 12, and the dangerous situation recognized by the peripheral situation recognition unit 3.

For example, as shown in FIG. 6, the notification unit 7 sends a map including the position where the dangerous situation occurred, that is, the position where the danger avoidance action by the self-driving vehicle or the danger avoidance action by the other vehicle was performed to the notification device 15. indicate. In addition, although the example of FIG. 6 shows the case where the notification device 15 is the CID, it is not limited to the CID.

In addition to the above, a map including marking objects indicating positions where dangerous situations have occurred is displayed on the notification device 15, and when an arbitrary marking object is selected, a display as shown in FIG. 6 or an image of the dangerous situation is displayed. may

In addition to information about a single drive, it also stores dangerous situations that occurred in the past for a predetermined number of days or a predetermined distance traveled, and adds information such as the date and time when a dangerous situation occurred in the past. may be notified by

<Modification 3>
As shown in FIG. 7, the positional relationship between the host vehicle and the other vehicle, and whether the risk avoidance action by automatic driving of the host vehicle or the risk avoidance action by the other vehicle was performed may be reported. In FIG. 7, the vehicle object indicates the own vehicle, and the hatched portions indicate the positions of other vehicles.

Specifically, the control unit 6 determines which of the own vehicle or the other vehicle executed the danger avoidance operation from the time series of the relative positional relationship between the own vehicle and the other vehicle recognized by the surrounding situation recognition unit 3. to judge.

Also, based on the recognition result of the surrounding situation recognition unit 3, it is determined whether the danger avoidance action was performed by the automatic driving of another vehicle or whether the danger avoidance action was performed by the driver of the other vehicle. It can be determined from the own vehicle driving information acquired by the own vehicle driving information acquiring unit 4 whether or not the own vehicle has performed a risk avoidance operation by automatically driving the own vehicle.

Although the example of FIG. 7 shows the case where the notification device 15 is a HUD, it is not limited to the HUD.

<Modification 4>
As shown in FIG. 8, a pattern of past dangerous situations may be notified. Specifically, based on a plurality of dangerous situations that the driver of the vehicle did not recognize in the past, the control unit 6 determines the pattern of the past dangerous situations that the driver of the vehicle did not recognize (the (patterns that are likely to cause dangerous situations that the driver of the vehicle was not aware of) are generated.

In FIG. 8, [data], [analysis results], and [patterns in which dangerous situations are likely to occur] indicate, for example, the transition of operations from the menu screen. In the example of FIG. 8, patterns in which dangerous situations are likely to occur are shown in a ranking format, but the present invention is not limited to this.

As shown in FIG. 9, the notification unit 7 may notify that a situation in which danger is likely to occur while the own vehicle is traveling has occurred. In other words, the reporting unit 7 may report that the current driving situation of the host vehicle is the same as the past dangerous situation pattern.

Although the examples of FIGS. 8 and 9 show the case where the notification device 15 is the CID, it is not limited to the CID.

<Modification 5>
In the above description, the driver state recognition unit 2 determines whether or not the driver of the vehicle recognizes a dangerous situation based on the line of sight of the driver of the vehicle, but the present invention is not limited to this. do not have.

The driver state recognition unit 2 determines whether or not the driver of the vehicle recognizes a dangerous situation based on other information such as arousal (physiology), facial expression (psychology), and line of sight. good. The driver state recognition unit 2 recognizes that the driver of the vehicle is in a dangerous situation, for example, when the expression of the driver shows fear or surprise, or when the driver recognizes palpitations or cold sweat. You can judge that there is In addition, the driver state recognition unit 2 recognizes that the driver of the vehicle recognizes a dangerous situation when the driver's limbs are jerking or the driver's eyes are closed. You can judge.

<Modification 6>
The driver state recognition unit 2 may determine whether or not the driver of the own vehicle recognizes the dangerous situation based on the behavior of the driver of the own vehicle when the dangerous situation occurs. The action of the driver of the own vehicle includes the action of avoiding danger by the driver of the own vehicle and the action of the driver of the own vehicle trying to avoid danger. When the driver of the own vehicle is making an action to avoid danger, the driver state recognition unit 2 determines that the driver is aware of a dangerous situation based on the driver's actions up to the actuator operation. You can judge.

<effect>
From the above, according to Embodiment 1, when the driver does not recognize that the dangerous situation has been avoided by the danger avoidance action by the own vehicle or the danger avoidance action by the other vehicle, the dangerous situation It is possible to notify the driver of the occurrence. That is, it is possible to appropriately notify the driver of a dangerous situation that the driver has not recognized.

<Embodiment 2>
<Configuration>
FIG. 10 is a block diagram showing an example of the configuration of the notification control device according to the second embodiment.

As shown in FIG. 10 , the notification control device 21 is characterized by having a communication section 22 . Since other configurations and basic operations are the same as those of the notification control device 16 shown in FIG. 5, detailed description thereof will be omitted here.

The communication unit 22 transmits the dangerous situations stored in the situation storage unit 9 as dangerous information to the information distribution server 23 via the communication device 14 . The communication unit 22 has the other vehicle driving information acquisition unit 5 .

The information distribution server 23 collects the danger information transmitted from the notification control device 21 and distributes the collected danger information to the notification control device 21 as appropriate.

The notification control device 21 shown in FIG. 10 is based on the notification control device 16 shown in FIG. 5, but may be based on the notification control device 8 shown in FIG.

<Action>
FIG. 11 is a flow chart showing an example of the operation of the notification control device 21. As shown in FIG. Note that the processes other than step S28 in FIG. 11 are the same as steps S11 to S18 in FIG. 3, so descriptions thereof are omitted here. Here, step S28 will be described.

In step S28, the communication unit 22 transmits the dangerous situation stored in the situation storage unit 9 to the information distribution server 23 via the communication device 14 as danger information. Specifically, the communication unit 22 converts the dangerous situations stored in the situation storage unit 9 into an appropriate format according to instructions from the control unit 6, and transmits the converted information to the information distribution server 23 as danger information. Danger information may include only video and audio related to the dangerous situation, but it also includes location information of the place where the dangerous situation occurred and detailed information on the dangerous situation (object, cause of occurrence, time of occurrence, condition of the driver of the own vehicle, etc.). etc.) may be included.

The information distribution server 23 collects and manages the danger information transmitted from the notification control device 21 . The danger information managed by the information distribution server 23 may be viewable from the outside, for example, by the driver of the vehicle.

<Modification 1>
Based on the collected danger information, the information distribution server 23 identifies the position where a dangerous situation occurred when the driver of the own vehicle was not aware of it (danger avoidance action by automatic driving of the own vehicle or danger avoidance action by another vehicle). It is possible to create a danger map indicating the location where the accident occurred on a map, and distribute the created danger map to the outside (for example, the notification control device 21).

<Modification 2>
The information distribution server 23 may store the vehicle equipped with the notification control device 21 that has transmitted the danger information as the vehicle requiring caution.

The information distribution server 23 notifies vehicles existing in the vicinity of the caution vehicle to keep a distance from the caution vehicle, or performs automatic driving control to keep a distance from the caution vehicle. may As a method of keeping a distance from the vehicle requiring caution, for example, it is possible to maintain a distance between vehicles equal to or greater than a predetermined distance, or to present another route away from the vehicle requiring caution.

<effect>
As described above, according to the second embodiment, in addition to the effects of the first embodiment, it is possible to manage more detailed and numerous dangerous situations. Also, the information managed in this way can be viewed from the outside.

<Hardware configuration>
The driver state recognition unit 2, the surrounding situation recognition unit 3, the host vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 in the notification control device 8 described in the first embodiment. Each function is implemented by a processing circuit. That is, the notification control device 8 recognizes the state of the driver of the own vehicle, recognizes the surrounding situation of the own vehicle, acquires the own vehicle driving information, acquires the other vehicle driving information, When a dangerous situation occurs and the driver of the own vehicle is unaware of the dangerous situation, it is determined whether the danger avoidance action by the self-driving of the own vehicle or the danger avoidance action by another vehicle is performed. If it is determined that the danger avoidance action by automatic driving or the danger avoidance action by another vehicle has been performed, the dangerous situation when the danger avoidance action by automatic driving of the own vehicle or the danger avoidance action by the other vehicle is performed A processing circuit is provided for notifying the person. The processing circuit may be dedicated hardware, and a processor (CPU, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, DSP (Digital Signal Processor)) that executes a program stored in memory may be called).

If the processing circuit is dedicated hardware, as shown in FIG. 12, the processing circuit 24 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit) , FPGA (Field Programmable Gate Array), or a combination thereof. Even if each function of the driver state recognition unit 2, the surrounding situation recognition unit 3, the own vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 is realized by the processing circuit 24, Alternatively, each function may be collectively realized by one processing circuit 24 .

When the processing circuit 24 is the processor 25 shown in FIG. 13, the driver state recognition unit 2, the surrounding situation recognition unit 3, the host vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit. Each function of 7 is implemented by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 26 . The processor 25 implements each function by reading and executing the program recorded in the memory 26 . That is, the notification control device 8 performs a step of recognizing the state of the driver of the own vehicle, a step of recognizing the surrounding situation of the own vehicle, a step of acquiring own vehicle driving information, a step of acquiring other vehicle driving information, and a step of acquiring the own vehicle driving information. When a dangerous situation occurs while the vehicle is running and the driver of the own vehicle is not aware of the dangerous situation, it is determined whether or not danger avoidance actions have been taken by the self-driving vehicle or by other vehicles. step, when it is determined that the danger avoidance action by automatic driving of the own vehicle or the danger avoidance action by another vehicle is performed, the dangerous situation when the danger avoidance action by automatic driving of the own vehicle or the danger avoidance action by the other vehicle is performed A memory 26 is provided for storing a program that results in the execution of the step of informing the driver of the own vehicle of. In addition, these programs make the procedures or methods of the driver state recognition unit 2, the surrounding situation recognition unit 3, the own vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 It can also be said that it is something that is executed by Here, memory means non-volatile or volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), etc. a flexible semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a DVD (Digital Versatile Disc), etc., or any storage medium that will be used in the future.

Some of the functions of the driver state recognition unit 2, the surrounding situation recognition unit 3, the own vehicle driving information acquisition unit 4, the other vehicle driving information acquisition unit 5, the control unit 6, and the notification unit 7 are hardware, and other functions may be implemented by software or firmware.

In this way, the processing circuit can implement each of the functions described above by means of hardware, software, firmware, or a combination thereof.

Although the hardware configuration of the notification control device 8 described in Embodiment 1 has been described above, the hardware configuration of the notification control device 16 shown in FIG. 5 and the hardware configuration of the notification control device 21 shown in FIG. The same applies to the configuration.

Within the scope of the present disclosure, it is possible to freely combine each embodiment, and to modify or omit each embodiment as appropriate.

Although the present disclosure has been described in detail, the above description is, in all aspects, exemplary and non-limiting. It is understood that a myriad of variations not illustrated may be envisioned.

1 Notification control device, 2 Driver state recognition unit, 3 Surrounding situation recognition unit, 4 Own vehicle driving information acquisition unit, 5 Other vehicle driving information acquisition unit, 6 Control unit, 7 Notification unit, 8 Notification control device, 9 Situation storage Section, 10 Driver state detection device, 11 Surrounding situation detection device, 12 Photographing device, 13 Automatic operation control device, 14 Communication device, 15 Notification device, 16 Notification control device, 17 Location information acquisition unit, 18 Map information acquisition unit, 19 positioning device, 20 map DB storage device, 21 notification control device, 22 communication unit, 23 information distribution server, 24 processing circuit, 25 processor, 26 memory.

Claims

a driver state recognition unit that recognizes the state of the driver of the own vehicle;
a peripheral situation recognition unit that recognizes the surrounding situation of the own vehicle;
an own vehicle driving information acquisition unit that acquires own vehicle driving information that is information relating to driving of the own vehicle;
an other-vehicle driving information acquisition unit that acquires other-vehicle driving information that is information about driving of another vehicle;
the state of the driver of the own vehicle recognized by the driver state recognition unit; the surrounding situation recognized by the surrounding situation recognition unit; the own vehicle driving information acquired by the own vehicle driving information acquisition unit; Based on the other vehicle driving information acquired by the other vehicle driving information acquisition unit, when a dangerous situation occurs while the own vehicle is traveling and the driver of the own vehicle is not aware of the dangerous situation, a control unit that determines whether or not a risk avoidance action by automatic driving of the own vehicle or a risk avoidance action by the other vehicle has been performed;
When the control unit determines that the risk avoidance action by the automatic driving of the own vehicle or the danger avoidance action by the other vehicle is performed, the risk avoidance action by the automatic driving of the own vehicle or the danger avoidance action by the other vehicle is performed. a notification unit that notifies the driver of the own vehicle of the dangerous situation when
A notification control device.
The notification control device according to claim 1, wherein the own vehicle driving information includes information indicating that the own vehicle has been automatically driven to avoid danger.
The notification control device according to claim 1, wherein the other vehicle driving information includes information indicating that the other vehicle has performed a danger avoidance action.
The information control device according to claim 1, wherein the risk avoidance action by the other vehicle includes a risk avoidance action by automatic driving of the other vehicle or a risk avoidance action by the driver of the other vehicle.
further comprising a situation storage unit that stores the dangerous situation when the risk avoidance action by automatic driving of the own vehicle or the risk avoidance action by the other vehicle is performed,
2. The notification control device according to claim 1, wherein said notification unit notifies said dangerous situation stored in said situation storage unit.
The notification control device according to claim 5, wherein the dangerous situation includes video.
a location information acquisition unit that acquires current location information of the own vehicle;
a map information acquisition unit that acquires map information including the road on which the vehicle is traveling;
further comprising
2. The notification control device according to claim 1, wherein the notification unit displays on a map a position where a risk avoidance action by automatic driving of the own vehicle or a risk avoidance action by the other vehicle was performed.
5. The reporting unit reports the relative position of the other vehicle with respect to the host vehicle, and whether a risk avoidance action by automatic driving of the host vehicle or a risk avoidance action by the other vehicle has been performed. A notification control device as described.
The control unit generates a pattern of past dangerous situations that the driver of the vehicle did not recognize based on a plurality of the dangerous situations that the driver of the vehicle did not recognize in the past,
The notification control device according to claim 1, wherein the notification unit notifies the pattern of the past dangerous situation.
The notification control device according to claim 9, wherein the notification unit notifies that the current driving situation of the own vehicle is the same as the past dangerous situation pattern.
the state of the driver of the own vehicle includes at least one of psychology, physiology, facial expression, and line of sight of the driver of the own vehicle;
2. The notification control device according to claim 1, wherein the driver state recognition unit determines whether or not the driver of the own vehicle recognizes the dangerous situation based on the state of the driver of the own vehicle. .
The state of the driver of the own vehicle includes actions of the driver of the own vehicle,
The driver state recognition unit determines whether or not the driver of the own vehicle recognizes the dangerous situation based on the behavior of the driver of the own vehicle when the dangerous situation occurs. Item 1. The information control device according to item 1.
13. The notification control device according to claim 12, wherein the action of the driver of the own vehicle includes a risk avoidance action by the driver of the own vehicle and an action of the driver of the own vehicle trying to avoid danger.
Further comprising a driver state detection device that detects the state of the driver of the own vehicle,
The notification control device according to claim 1, wherein the driver state recognition unit recognizes the state of the driver of the own vehicle based on the state of the driver of the own vehicle detected by the driver state detection device. .
The notification control device according to claim 5, further comprising a communication unit that transmits the dangerous situation stored in the situation storage unit as dangerous information to an external information distribution server.
2. An information distribution server that collects, from the notification control device according to claim 1, the dangerous situation as danger information when a danger avoidance action by automatic driving of the own vehicle or a danger avoidance action by the other vehicle is performed. and an information distribution server that distributes the collected danger information to the notification control device.
Based on the collected danger information, a danger map is created showing on a map the positions where the danger avoidance action by the self-driving vehicle or the danger avoidance action by the other vehicle is performed, and the created danger map is generated. 17. The information distribution server according to claim 16, which distributes to said notification control device.
The information distribution server according to claim 16, wherein the vehicle equipped with the notification control device that transmitted the danger information is stored as a caution vehicle.
Recognizing the state of the driver of the own vehicle,
recognizing the surrounding situation of the own vehicle,
Acquiring own vehicle driving information, which is information related to driving of the own vehicle;
Acquiring other vehicle driving information, which is information related to the driving of other vehicles,
Based on the recognized state of the driver of the own vehicle, the recognized surrounding situation, the acquired own vehicle driving information, and the acquired other vehicle driving information, a dangerous situation occurs while the own vehicle is running. determining whether or not a risk avoidance action by automatic driving of the own vehicle or a risk avoidance action by the other vehicle has been performed when the driver of the own vehicle is unaware of the dangerous situation,
When it is determined that the risk avoidance action by the automatic driving of the own vehicle or the danger avoidance action by the other vehicle is performed, the above-described risk avoidance action by the automatic driving of the own vehicle or the risk avoidance action by the other vehicle is performed. A notification control method for notifying a driver of the own vehicle of a dangerous situation.