JP2021012624A - Driving support device, and driving support program - Google Patents

Abstract

To provide a driving support device that alleviates obstruction of a passenger side's field of vision by a passenger in a passenger seat when a driver turns his/her view toward the passenger seat side.SOLUTION: A driving support device includes: a driver detection portion that detects information about a driver based on images captured in a vehicle; a mirror direction calculation portion that calculates a direction of a door mirror on a passenger seat side as seen by the driver; a passenger detection portion that detects a seating position and posture of the passenger based on the images captured in the vehicle; a passenger condition determination portion that determines whether or not the passenger obstructs the driver's view of the passenger seat side based on the direction of the door mirror on the passenger seat side and the seating position and posture of the passenger; a visual prediction portion that predicts whether or not the driver's view is directed toward the door mirror on the passenger seat side, when the passenger obstructs the driver's view of the passenger seat side; and a warning portion that outputs warning information when the visual prediction portion predicts that the driver's view is directed toward the door mirror on the passenger seat side.SELECTED DRAWING: Figure 1

Description

The present invention relates to a driving support device for assisting a driver of a vehicle in driving, and a driving support program.

Conventionally, in a vehicle, when there is a passenger in the passenger seat, the driver's view on the passenger seat side (hereinafter, also simply referred to as "passenger seat side view") is obstructed by the passenger in the passenger seat. There was a problem that there was something.
As a countermeasure to the above-mentioned problem, for example, in Patent Document 1, when it is determined that the passenger in the passenger seat obstructs the view on the passenger seat side, the passenger seat is changed to the passenger in the passenger seat. Disclosed is a technique for moving the passenger seat side in a direction in which the degree of obstruction of the field of vision is alleviated.

JP-A-2017-154715

In the conventional technique represented by the technique disclosed in Patent Document 1, the passenger's seat side's view is obstructed after the passenger actually in the passenger seat obstructs the passenger's side's view. I try to relax the degree of. Therefore, there is still a problem that the field of vision on the passenger seat side is obstructed when the driver turns his or her line of sight to the passenger seat side.

The present invention has been made to solve the above-mentioned problems, and alleviates that when the driver turns his / her line of sight toward the passenger seat side, the passenger in the passenger seat obstructs the view of the passenger seat side. The purpose is to provide a driving support device.

The driving support device according to the present invention includes an image acquisition unit that acquires an in-vehicle image, a driver detection unit that detects information about the driver based on the in-vehicle image acquired by the image acquisition unit, and a driver detection unit. Based on the detected information about the driver, the mirror direction calculation unit that calculates the direction of the door mirror on the passenger side as seen by the driver, and the passenger in the passenger seat based on the in-vehicle image captured by the image acquisition unit. Based on the passenger detection unit that detects the seating position and posture, the direction of the door mirror on the passenger side calculated by the mirror direction calculation unit, and the passenger seating position and posture detected by the passenger detection unit. The passenger status determination unit determines whether or not the driver obstructs the view of the passenger side of the driver, the status information acquisition unit that acquires vehicle status information, and the passenger status determination unit that the passenger is the driver. When it is determined that the visibility on the passenger side is obstructed, the visual prediction unit predicts whether the driver's line of sight is directed toward the door mirror on the passenger side based on the vehicle surroundings information, and the visual prediction. The unit is provided with a warning unit that outputs warning information when the driver predicts that the driver's line of sight will be directed toward the door mirror on the passenger side.

According to the present invention, when the driver turns his / her line of sight to the passenger seat side, it is possible to alleviate that the passenger in the passenger seat obstructs the view of the passenger seat side.

It is a figure which shows the configuration example of the driving support apparatus which concerns on Embodiment 1. FIG. In the driving support device according to the first embodiment, when it is predicted that the driver's line of sight is directed toward the door mirror in a situation where the passenger is obstructing the driver's field of vision, until the warning information is output. It is a flowchart for demonstrating operation. In the driving support device according to the first embodiment, after the operation described with reference to FIG. 2 is performed, an operation of switching the display of driving-related information when the obstruction of the driver's field of vision by the passenger is not alleviated will be described. It is a flowchart for doing. 4A and 4B are diagrams showing an example of the hardware configuration of the driving support device according to the first embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of the driving support device 1 according to the first embodiment.
The driving support device 1 is mounted on a vehicle, for example. If it is predicted that the driver's line of sight will be directed toward the door mirror on the passenger seat side in a situation where the passenger in the passenger seat obstructs the driver's field of vision, the driving support device 1 will be used in advance. Warn passengers in the passenger seat. In the following description, when the term "passenger" is used, the "passenger" shall mean a passenger in the passenger seat.

As shown in FIG. 1, the driving support device 1 is connected to the image pickup device 2, the output device 3, and the display device 4.
The image pickup device 2 is at least an in-vehicle camera that images the inside of the vehicle. The in-vehicle camera is installed, for example, on the dashboard or ceiling of the vehicle. The in-vehicle camera captures at least an area including the driver's seat and the passenger's seat in the vehicle. More specifically, the in-vehicle camera captures at least the area including the driver's seat headrest and the passenger's seat headrest from the front.
The image pickup device 2 may be an outside-vehicle camera that images the outside of the vehicle in addition to the in-vehicle camera. The vehicle exterior camera is installed, for example, on the upper part of the windshield of the vehicle or the upper part of the rear glass of the vehicle, and captures the scenery around the vehicle.
The image pickup device 2 may be shared with a so-called "driver monitoring system (DMS)".
The image pickup device 2 outputs the captured image to the driving support device 1.

The output device 3 is, for example, a speaker mounted on a car navigation device (not shown) and installed in the vehicle. The output device 3 outputs a sound or voice based on the alarm information (details will be described later) output from the driving support device 1.

The display device 4 is a display provided in the vehicle and displays various information. The display device 4 is, for example, a center display or a meter display.
Further, in the first embodiment, the door mirror and the rearview mirror are assumed to be electronic mirrors, and the door mirror and the rearview mirror are also included in the display device 4.

As shown in FIG. 1, the driving support device 1 includes an image acquisition unit 101, a driver detection unit 102, a mirror direction calculation unit 103, a passenger detection unit 104, a passenger state determination unit 105, a situation information acquisition unit 106, and visual inspection. It includes a prediction unit 107, a warning unit 108, a display control unit 109, and an improvement presence / absence determination unit 110. The visual prediction unit 107 includes a timing determination unit 1071.

The image acquisition unit 101 acquires an image (hereinafter, referred to as “in-vehicle image”) captured by the in-vehicle camera.
The image acquisition unit 101 outputs the acquired in-vehicle image to the driver detection unit 102 and the passenger detection unit 104.

The driver detection unit 102 detects information about the driver based on the in-vehicle captured image acquired by the image acquisition unit 101. Specifically, the driver detection unit 102 detects, for example, the coordinates of the center position of the driver's head in the in-vehicle image as information about the driver, based on the in-vehicle image. The driver detection unit 102 may detect the center position of the driver's head by using a known image processing technique.
In the first embodiment, the driver detection unit 102 detects the coordinates of the center position of the driver's head as information about the driver, but this is only an example. For example, the driver detection unit 102 may detect the coordinates of the center positions of both eyes of the driver as information about the driver by using a known image processing technique based on the captured image in the vehicle. The driver detection unit 102 may detect information about a point that can be regarded as a reference for calculating the line-of-sight direction of the driver as information about the driver.
The driver detection unit 102 outputs the detected information about the driver (hereinafter referred to as “driver-related information”) to the mirror direction calculation unit 103.

The mirror direction calculation unit 103 calculates the direction of the door mirror on the passenger seat side as seen by the driver based on the driver-related information output from the driver detection unit 102. Hereinafter, the direction of the door mirror on the passenger seat side as seen from the driver is also simply referred to as "door mirror direction".
Specifically, the mirror direction calculation unit 103 calculates the direction from the center position of the driver's head to the door mirror on the passenger seat side in the real space inside the vehicle as the door mirror direction.
In the first embodiment, it is assumed that the imaging range of the in-vehicle camera, the installation position of the in-vehicle camera, and the position of the door mirror are known in advance. If the center position of the driver's head on the in-vehicle image is known, the mirror direction calculation unit 103 can determine the center position, the position of the headrest on the in-vehicle image, or the driver's position and size on the in-vehicle image. From the above, the central position of the driver's head in the real space inside the vehicle can be calculated. If the center position of the driver's head in the real space in the vehicle can be calculated, the mirror direction calculation unit 103 will use the actual space in the vehicle based on the positional relationship between the installation position of the in-vehicle camera and the position of the door mirror on the passenger seat side. The direction from the center position of the driver's head to the door mirror on the passenger seat side can be calculated.
The mirror direction calculation unit 103 outputs information regarding the door mirror direction to the passenger state determination unit 105 and the improvement presence / absence determination unit 110.

The passenger detection unit 104 detects the seating position and posture of the passenger based on the in-vehicle image captured by the image acquisition unit 101.
Specifically, first, the passenger detection unit 104 detects a passenger on an in-vehicle image by using a known image processing technique. Then, the passenger detection unit 104 detects the seating position and posture of the passenger in the real space in the vehicle from the position of the passenger on the captured image in the vehicle. As described above, since the imaging range of the in-vehicle camera and the installation position of the in-vehicle camera are known in advance, the passenger detection unit 104 can see the position and size of the passenger on the in-vehicle captured image in the vehicle. The seating position and posture of the passenger in the real space can be detected.
The passenger detection unit 104 outputs the detected information on the seating position and posture of the passenger (hereinafter referred to as “passenger-related information”) to the passenger state determination unit 105 and the improvement presence / absence determination unit 110.

The passenger state determination unit 105 determines that the passenger is on the passenger side of the driver based on the door mirror direction calculated by the mirror direction calculation unit 103 and the seating position and posture of the passenger detected by the passenger detection unit 104. It is determined whether or not the visibility (hereinafter referred to as "passenger seat side visibility") is obstructed.
The passenger state determination unit 105 determines, for example, that when a passenger is present in the direction of the door mirror, the passenger is obstructing the passenger seat side view.
When the passenger status determination unit 105 determines that the passenger is obstructing the passenger seat side view, the passenger status determination unit 105 provides the visual prediction unit 107 with passenger information indicating that the passenger is obstructing the passenger seat side view. Output.

The status information acquisition unit 106 acquires information on the status of the vehicle (hereinafter referred to as "vehicle status information").
The vehicle status information is, for example, navigation route guidance information, an image obtained by capturing the scenery around the vehicle (hereinafter referred to as “outside vehicle captured image”), or current position information. The status information acquisition unit 106 acquires route guidance information from, for example, a car navigation device. Further, the situation information acquisition unit 106 acquires an image captured outside the vehicle from, for example, a camera outside the vehicle. Further, the status information acquisition unit 106 acquires the current position information from the GPS information received from the GPS satellite by the GPS (Global Positioning System) receiver mounted on the vehicle, for example.
Note that this is only an example, and the vehicle status information may be vehicle speed information acquired from a vehicle speed sensor mounted on the vehicle, gear information acquired from a shift sensor mounted on the vehicle, or the like. The vehicle condition information may be any information that can be used by the visual prediction unit 107 to predict whether or not the driver's line of sight is directed toward the door mirror. The visual prediction unit 107 will be described later.
The status information acquisition unit 106 outputs the acquired vehicle status information to the visual prediction unit 107. The status information acquisition unit 106 can also store the acquired vehicle status information in a storage unit (not shown) provided in a place where the driving support device 1 can refer.

When the passenger condition determination unit 105 determines that the passenger is obstructing the passenger seat side view, in other words, when the passenger condition determination unit 105 outputs passenger presence information, the visual prediction unit 107 Based on the vehicle status information acquired by the status information acquisition unit 106, it is predicted whether or not the driver's line of sight is directed toward the door mirror.
More specifically, the visual prediction unit 107 predicts whether or not an event that the driver is expected to confirm the door mirror (hereinafter referred to as "door mirror confirmation expected event") will occur, and the door mirror confirmation expected event will occur. Then, when it is predicted, the driver's line of sight is predicted to be directed toward the door mirror.

When the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror, the timing determination unit 1071 of the visual prediction unit 107 predicts the timing at which the driver's line of sight is directed toward the door mirror, and the timing is determined. Determine if they are close. In other words, the timing determination unit 1071 predicts the timing at which the door mirror confirmation prediction event occurs, and determines whether or not the timing is near.
In the first embodiment, the timing at which the driver's line of sight is directed toward the door mirror is near, which means that the driver's line of sight is directed toward the door mirror within a preset time (hereinafter referred to as "determination time"). To be done. The determination time is preset by the driver or the like, and the driver or the like can appropriately change the determination time. The driver or the like sets the judgment time to be shorter than the time expected as the time until the door mirror confirmation expected event occurs.
When the timing determination unit 1071 determines that the timing at which the driver's line of sight is predicted to be directed toward the door mirror is near, the timing determination unit 1071 provides information that the timing at which the driver's line of sight is predicted to be directed toward the door mirror is near. Output to the visual prediction unit 107.

The door mirror confirmation expected event means, for example, the following.
・ Turn right or left to the passenger seat side of the vehicle Note that turning right or left to the passenger seat side means turning left to the passenger seat side when the passenger seat is on the left side of the driver's seat in the direction of travel of the vehicle. When the passenger seat is on the right side of the driver's seat in the direction of travel of the vehicle, it means a right turn to the passenger seat side.
The visual prediction unit 107 predicts that a door mirror confirmation prediction event will occur when, for example, the vehicle is predicted to turn right or left toward the passenger seat side based on the route guidance information. That is, the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror.
Based on the route guidance information, vehicle speed information, and current position information, the timing determination unit 1071 moves the vehicle to the passenger seat side based on the distance from the current position to the point where the vehicle turns right or left to the passenger seat side and the vehicle speed. Predict the timing of turning right or left, and determine whether the timing is near.

-Change of lane to the passenger side lane of the vehicle For example, when the vehicle is traveling in the right lane of the highway, the visual prediction unit 107 determines the distance and relative speed between the vehicle and the following vehicle based on the image captured outside the vehicle. Is calculated. Based on the calculated distance and relative speed, the visual prediction unit 107 predicts that the vehicle will change lanes to the passenger seat side lane when the following vehicle is approaching the vehicle at a speed equal to or higher than a preset threshold value. In this case, it is assumed that the passenger seat is on the left side of the driver's seat with respect to the traveling direction of the vehicle.
When it is predicted that the vehicle will change lanes to the passenger seat side lane, the visual prediction unit 107 predicts that a door mirror confirmation prediction event will occur. That is, the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror.
The timing determination unit 1071 predicts the timing at which the vehicle changes lanes based on the distance between the vehicle and the following vehicle and the relative speed. For example, the timing determination unit 1071 predicts that the timing at which the following vehicle approaches the vehicle to a position at a predetermined distance is the timing at which the vehicle changes lanes. Then, the timing determination unit 1071 determines whether or not the predicted timing is close.

When the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror, and the timing determination unit 1071 determines that the timing at which the driver's line of sight is directed toward the door mirror is near, the driver's line of sight is determined. Is predicted to be directed toward the door mirrors, and information indicating that the timing at which the driver's line of sight is directed toward the door mirrors is near is output to the warning unit 108.

The warning unit 108 outputs warning information when the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror. More specifically, the warning unit 108 is a case where the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror, and the timing determination unit 1071 determines that the driver's line of sight is the passenger seat side door mirror. When it is determined that the timing to be directed in the direction of is near, warning information is output.
The warning information is, for example, a buzzer sound or a message for alerting the passenger to the effect that the driver's field of vision is obstructed.
For example, the warning unit 108 outputs warning information to the output device 3, and causes the output device 3 to output a buzzer sound or the voice of the above message. The warning unit 108 may output warning information to the output device 3 and output warning information to the display control unit 109. When the warning information is output from the warning unit 108, the display control unit 109 displays, for example, the above message on the display device 4.

The display control unit 109 controls the display of various information on the display device 4.
For example, the display control unit 109 controls the display of vehicle-related information on the door mirrors. The vehicle-related information is, for example, an image of the rear of the vehicle. For example, the display control unit 109 causes the door mirror to display the image captured outside the vehicle acquired by the image acquisition unit 101 from the camera outside the vehicle.
Further, when the display control unit 109 determines that the improvement presence / absence determination unit 110 obstructs the passenger seat side view, the display control unit 109 displays the vehicle-related information displayed on the passenger seat side door mirror on the passenger seat side. Display on a display device 4 other than the door mirror. The improvement presence / absence determination unit 110 will be described later. At this time, the display control unit 109 displays the vehicle-related information displayed on the door mirror on the passenger seat side on the display device 4 other than the door mirror on the passenger seat side (hereinafter referred to as "display switching information"). Is displayed on a display device 4 other than the door mirror. For example, the display control unit 109 says that "the information of the door mirror is displayed" on the display device 4 other than the door mirror on the passenger seat side, which displays the vehicle-related information displayed on the door mirror on the passenger seat side. Display a message. Further, for example, the display control unit 109 may output the display switching information to the warning unit 108. When the display switching information is output from the display control unit 109, the warning unit 108 displays the vehicle-related information displayed on the door mirror on the passenger seat side from the output device 3, for example, to the driver on the passenger seat side. A buzzer sound or a voice message is output to notify the display device 4 other than the door mirror of the above.

The improvement presence / absence determination unit 110 rides on the vehicle based on the door mirror direction calculated by the mirror direction calculation unit 103 after the warning information is output by the warning unit 108 and the seating position and posture of the passenger detected by the passenger detection unit 104. Determine if the person is obstructing the passenger side view.
When the passenger determines that the passenger is obstructing the view on the passenger seat side, the improvement presence / absence determination unit 110 outputs to the display control unit 109 no improvement information indicating that the passenger is obstructing the view on the passenger seat side. To do. The fact that the improvement presence / absence determination unit 110 has determined that the passenger is obstructing the view on the passenger seat side means that the passenger obstructs the driver's view in the direction of the door mirror even though the warning information is output. It means that it has not been relaxed.
When the improvement non-information is output from the improvement presence / absence determination unit 110, the display control unit 109 displays the vehicle-related information displayed on the passenger seat side door mirror as described above by a display device other than the passenger seat side door mirror. Display in 4.

The operation of the driving support device 1 according to the first embodiment will be described.
2 and 3 are flowcharts for explaining the operation of the driving support device 1 according to the first embodiment.
FIG. 2 shows warning information in the driving support device 1 according to the first embodiment when it is predicted that the driver's line of sight is directed toward the door mirror in a situation where the passenger is obstructing the driver's field of vision. It is a flowchart for demonstrating the operation until the output.
FIG. 3 shows switching of the display of driving-related information in the driving support device 1 according to the first embodiment when the obstruction of the driver's field of vision by the passenger is not alleviated after the operation described in FIG. 2 is performed. It is a flowchart for demonstrating the operation which performs.
The operations described with reference to FIGS. 2 and 3 are continuously performed, for example, at a predetermined cycle while the engine of the vehicle is operating.

The image acquisition unit 101 acquires an in-vehicle captured image (step ST201).
The image acquisition unit 101 outputs the acquired in-vehicle image to the driver detection unit 102 and the passenger detection unit 104.

The driver detection unit 102 detects the driver-related information based on the in-vehicle image captured by the image acquisition unit 101 in step ST201 (step ST202).
The driver detection unit 102 outputs the detected driver-related information to the mirror direction calculation unit 103.

The mirror direction calculation unit 103 calculates the direction of the door mirror based on the driver-related information output from the driver detection unit 102 in step ST202 (step ST203).
The mirror direction calculation unit 103 outputs information regarding the door mirror direction to the passenger state determination unit 105 and the improvement presence / absence determination unit 110.

The passenger detection unit 104 detects the seating position and posture of the passenger based on the in-vehicle image captured by the image acquisition unit 101 in step ST201 (step ST204).
The passenger detection unit 104 outputs the passenger-related information to the passenger status determination unit 105 and the improvement presence / absence determination unit 110.

The passenger state determination unit 105 is based on the door mirror direction calculated by the mirror direction calculation unit 103 in step ST203 and the seating position and posture of the passenger detected by the passenger detection unit 104 in step ST204. It is determined whether or not the passenger side view is obstructed (step ST205).

When the passenger status determination unit 105 determines in step ST205 that the passenger does not obstruct the passenger seat side view (when “NO” in step ST205), the driving support device 1 is the driving support device 1. Ends the operation of.
When the passenger condition determination unit 105 determines in step ST205 that the passenger is obstructing the passenger seat side view (when “YES” in step ST205), the passenger condition determination unit 105 is determined by the passenger. Passenger seat information indicating that the passenger seat side visibility is obstructed is output to the visual prediction unit 107.

The visual prediction unit 107 predicts whether or not the driver's line of sight is directed toward the door mirror based on the vehicle condition information acquired by the condition information acquisition unit 106 (step ST206). The status information acquisition unit 106 acquires vehicle status information at an appropriate timing up to the step ST206.
In step ST206, when the visual prediction unit 107 predicts that the driver's line of sight will not be directed toward the door mirror (when “NO” in step ST206), the driving support device 1 operates the driving support device 1. finish.

In step ST206, when the visual prediction unit 107 predicts that the driver's line of sight is directed toward the door mirror (when “YES” in step ST206), the timing determination unit 1071 directs the driver's line of sight toward the door mirror. The timing is predicted, and it is determined whether or not the timing is close (step ST207).

In step ST207, when the timing determination unit 1071 determines that the timing at which the driver's line of sight is predicted to be directed toward the door mirror is not near (when “NO” in step ST207), the driving support device 1 is concerned. The operation of the driving support device 1 is terminated.
In step ST207, when the timing determination unit 1071 determines that the timing at which the driver's line of sight is predicted to be directed toward the door mirror is near (when “YES” in step ST207), the visual prediction unit 107 determines that the driver Information that it is predicted that the line of sight of the driver will be directed toward the door mirror and that the timing at which the driver's line of sight is directed toward the door mirror is near is output to the warning unit 108.

The warning unit 108 outputs warning information (step ST208).

As described above, in the first embodiment, the driving support device 1 warns information when it is predicted that the driver's line of sight is directed toward the door mirror in a situation where the passenger obstructs the driver's field of vision. Is output. When the driving support device 1 outputs warning information, the passenger warns that he / she is obstructing the driver's view in the direction of the door mirrors. Since the driving support device 1 outputs warning information when it is predicted that the driver's line of sight is directed toward the door mirror, it is possible to warn the passenger in advance and the driver. It is possible to alleviate that the passenger in the passenger seat obstructs the view of the passenger seat when the driver turns his / her line of sight to the passenger seat side.

The flowchart of FIG. 3 will be described. The operation described in the flowchart of FIG. 3 is performed after the operation described in FIG.
Since the specific operations of steps ST301 to ST304 of FIG. 3 are the same as the specific operations of steps ST201 to ST204 of FIG. 2, duplicate description will be omitted.
After the warning unit 108 outputs warning information in step ST208 of FIG. 2, the improvement presence / absence determination unit 110 has the door mirror direction calculated by the mirror direction calculation unit 103 in step ST303 and the passenger detection unit 104 in step ST304. Based on the seating position and posture of the passenger detected by, it is determined whether or not the passenger is obstructing the passenger side view (step ST305).

When the improvement presence / absence determination unit 110 determines in step ST305 that the passenger does not obstruct the passenger seat side view (when “NO” in step ST305), the driving support device 1 is the driving support device 1. End the operation. The fact that the improvement presence / absence determination unit 110 has determined that the passenger does not obstruct the passenger's side view means that the warning information is output so that the passenger can alleviate the obstruction of the driver's view in the direction of the door mirrors. It can be inferred that the seating position or posture was changed.
On the other hand, in step ST305, when the improvement presence / absence determination unit 110 determines that the passenger is obstructing the view on the passenger seat side (when “YES” in step ST305), the passenger determines the view on the passenger seat side. No improvement information indicating that it is hindering is output to the display control unit 109. The fact that the improvement presence / absence determination unit 110 has determined that the passenger is obstructing the view on the passenger seat side means that the passenger obstructs the driver's view in the direction of the door mirror even though the warning information is output. It means that it has not been relaxed.

When the improvement / non-improvement information is output from the improvement / non-improvement determination unit 110 in step ST305, the display control unit 109 displays the vehicle-related information displayed on the door mirror on the passenger seat side by the display device 4 other than the door mirror on the passenger seat side. Is displayed (step ST306).

As described above, the driving support device 1 according to the first embodiment is on the passenger seat side unless the situation in which the passenger is obstructing the driver's view on the passenger seat side is alleviated despite the warning in advance. The vehicle-related information displayed on the door mirror is displayed on the display device 4 other than the door mirror on the passenger seat side. As a result, the driver can confirm the vehicle-related information displayed by the door mirror on the passenger seat side on the other display device 4, which cannot be confirmed by the passenger.

In the first embodiment described above, it is assumed that the door mirror and the rearview mirror are electronic mirrors, but the door mirror and the rearview mirror are not limited to the electronic mirrors. The door mirror and the rearview mirror may be optical mirrors.
However, in that case, the driving support device 1 displays on the door mirror on the passenger seat side when the situation in which the passenger is obstructing the driver's view on the passenger seat side is not alleviated despite the warning in advance. It does not have a switching function for displaying vehicle-related information on another display device 4. In this case, the driving support device 1 does not include the improvement presence / absence determination unit 110.

Further, in the above-described first embodiment, the driving support device 1 is provided with the timing determination unit 1071, but it is not essential to include the timing determination unit 1071. In the driving support device 1, when the driver's line of sight is predicted to be directed toward the door mirror in a situation where the passenger is obstructing the driver's field of vision, the driver's line of sight is actually directed toward the door mirror. It would be good if the passengers were warned before. However, by providing the driving support device 1 with the timing determination unit 1071, the driving support device 1 outputs warning information as soon as it is predicted that the driver's line of sight is directed toward the door mirror. The warning information will be output after waiting until the timing is closer to the timing when the driver's line of sight is predicted to be directed toward the door mirror. Therefore, the driving support device 1 can prevent the output of unnecessary warning information at an excessively early timing. From the time when the driver's line of sight is predicted to be directed toward the door mirrors to the time when the driver's line of sight is actually directed toward the door mirrors, the passenger will obstruct the driver's view without warning. This is because the situation may be resolved.

4A and 4B are diagrams showing an example of the hardware configuration of the driving support device 1 according to the first embodiment.
In the first embodiment, the image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the situation information acquisition unit 106, the visual prediction unit 107, and the warning unit 108 The functions of the display control unit 109 and the improvement presence / absence determination unit 110 are realized by the processing circuit 401. That is, the driving support device 1 controls to output warning information when it is predicted that the driver's line of sight is directed toward the door mirror in a situation where the passenger is obstructing the driver's field of vision. A processing circuit 401 is provided.
The processing circuit 401 may be dedicated hardware as shown in FIG. 4A, or may be a CPU (Central Processing Unit) 405 that executes a program stored in the memory 406 as shown in FIG. 4B.

When the processing circuit 401 is dedicated hardware, the processing circuit 401 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array) or a combination of these is applicable.

When the processing circuit 401 is the CPU 405, the image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the situation information acquisition unit 106, the visual prediction unit 107, and the warning unit The functions of 108, the display control unit 109, and the improvement presence / absence determination unit 110 are realized by software, firmware, or a combination of software and firmware. That is, the image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the situation information acquisition unit 106, the visual prediction unit 107, the warning unit 108, and the display control unit. The 109 and the improvement presence / absence determination unit 110 are realized by a processing circuit such as a CPU 405 that executes a program stored in an HDD (Hard Disk Drive) 402, a memory 406, and a system LSI (Large-Scale Integration). The programs stored in the HDD 402, the memory 406, etc. are visually predicted by the image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the situation information acquisition unit 106, and the like. It can also be said that the computer is made to execute the procedure or method of the unit 107, the warning unit 108, the display control unit 109, and the improvement presence / absence determination unit 110. Here, the memory 406 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Online Memory), an EEPROM (Electrically Memory), or an EEPROM (Electrically Memory). This includes sexual or volatile semiconductor memories, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs (Digital Versaille Disc), and the like.

The image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the situation information acquisition unit 106, the visual prediction unit 107, the warning unit 108, and the display control unit Regarding the functions of 109 and the improvement presence / absence determination unit 110, a part may be realized by dedicated hardware and a part may be realized by software or firmware. For example, the image acquisition unit 101 and the status information acquisition unit 106 are realized by the processing circuit 401 as dedicated hardware, and the driver detection unit 102, the mirror direction calculation unit 103, and the passenger detection unit 104 The functions of the visual prediction unit 107, the warning unit 108, the display control unit 109, and the improvement presence / absence determination unit 110 can be realized by the processing circuit 401 reading and executing the program stored in the memory 406. It is possible.
Further, the driving support device 1 includes devices such as an image pickup device 2, an output device 3 or a display device 4, and an input interface device 403 and an output interface device 404 that perform wired communication or wireless communication.

In the above-described first embodiment, the driving support device 1 is mounted on a vehicle, for example, but this is only an example.
For example, the image acquisition unit 101, the driver detection unit 102, the mirror direction calculation unit 103, the passenger detection unit 104, the passenger status determination unit 105, the situation information acquisition unit 106, the visual prediction unit 107, and the warning provided in the driving support device 1. A part or all of the unit 108, the display control unit 109, and the improvement presence / absence determination unit 110 are provided in the server (not shown), and the server, the driving support device 1, the image pickup device 2, and the like. The output device 3 or the display device 4 may be connected via a network.

As described above, the driving support device 1 according to the first embodiment is a driving that detects information about the driver based on the image acquisition unit 101 that acquires the in-vehicle image and the in-vehicle image that the image acquisition unit 101 acquires. Based on the information about the driver detected by the person detection unit 102 and the driver detection unit 102, the mirror direction calculation unit 103 that calculates the direction of the door mirror on the passenger side as seen from the driver, and the image acquisition unit 101 acquire the image. The passenger detection unit 104 that detects the seating position and posture of the passenger in the passenger seat, the direction of the door mirror on the passenger seat side calculated by the mirror direction calculation unit 103, and the passenger detection unit based on the captured image in the vehicle. Based on the seating position and posture of the passenger detected by 104, the passenger status determination unit 105 that determines whether or not the passenger is obstructing the view of the driver on the passenger side, and the situation of acquiring vehicle status information. When the information acquisition unit 106 and the passenger status determination unit 105 determine that the passenger is obstructing the driver's view on the passenger side, the driver's line of sight is the door mirror on the passenger side based on the vehicle status information. When the visual prediction unit 107 that predicts whether or not the vehicle is directed in the direction of the driver and the visual prediction unit 107 predict that the driver's line of sight is directed toward the door mirror on the passenger side, a warning unit that outputs warning information. It was configured to include 108. Therefore, the driving support device 1 can alleviate that when the driver turns his / her line of sight to the passenger seat side, the passenger in the passenger seat obstructs the view of the passenger seat side.

In the present invention, within the scope of the invention, it is possible to modify any component of the embodiment or omit any component of the embodiment.

1 Driving support device, 101 image acquisition unit, 102 driver detection unit, 103 mirror direction calculation unit, 104 passenger detection unit, 105 passenger status determination unit, 106 status information acquisition unit, 107 visual prediction unit, 1071 timing determination unit , 108 Warning unit, 109 Display control unit, 110 Improvement presence / absence judgment unit, 2 Imaging device, 3 Output device, 4 Display device, 401 Processing circuit, 402 HDD, 403 Input interface device, 404 output interface device, 405 CPU, 406 memory ..

Claims (6)

An image acquisition unit that acquires images captured inside the vehicle,
A driver detection unit that detects information about the driver based on the in-vehicle captured image acquired by the image acquisition unit, and a driver detection unit.
Based on the information about the driver detected by the driver detection unit, the mirror direction calculation unit that calculates the direction of the door mirror on the passenger seat side as seen from the driver, and the mirror direction calculation unit.
Based on the in-vehicle image captured by the image acquisition unit, the passenger detection unit that detects the seating position and posture of the passenger in the passenger seat, and the passenger detection unit.
Based on the direction of the door mirror on the passenger seat side calculated by the mirror direction calculation unit and the seating position and posture of the passenger detected by the passenger detection unit, the passenger is on the passenger seat side of the driver. Passenger condition determination unit that determines whether or not the visibility is obstructed,
The status information acquisition department that acquires vehicle status information,
When the passenger state determination unit determines that the passenger is obstructing the view of the driver on the passenger seat side, the driver's line of sight is the door mirror on the passenger seat side based on the vehicle condition information. A visual prediction unit that predicts whether or not it is directed in a direction,
A driving support device including a warning unit that outputs warning information when the visual prediction unit predicts that the driver's line of sight is directed toward the door mirror on the passenger seat side. When the visual prediction unit predicts that the driver's line of sight is directed toward the passenger seat side door mirror, it is predicted that the driver's line of sight is directed toward the passenger seat side door mirror. Equipped with a timing determination unit that determines whether the timing is close
The warning unit
1. The timing determining unit outputs the warning information when it determines that the timing at which the driver's line of sight is predicted to be directed toward the door mirror on the passenger seat side is near. The described driving support device. The driving support device according to claim 1 or 2, wherein the vehicle status information includes route guidance information. The driving support device according to claim 1 or 2, wherein the vehicle condition information includes an image captured outside the vehicle. The door mirror on the passenger seat side is an electronic mirror.
A display control unit that controls the display of vehicle-related information on the door mirror on the passenger seat side,
Based on the direction of the door mirror on the passenger seat side calculated by the mirror direction calculation unit after the warning unit outputs the warning information, and the seating position and posture of the passenger detected by the passenger detection unit. Provided with an improvement presence / absence determining unit for determining whether or not the passenger obstructs the view of the driver on the passenger seat side.
The display control unit
When the improvement presence / absence determination unit determines that the passenger is obstructing the view of the driver on the passenger seat side, the vehicle-related information displayed on the door mirror on the passenger seat side is displayed on the passenger seat side. The driving support device according to any one of claims 1 to 4, wherein the display is displayed on a display device other than the door mirror. Computer,
An image acquisition unit that acquires images captured inside the vehicle,
A driver detection unit that detects information about the driver based on the in-vehicle captured image acquired by the image acquisition unit, and a driver detection unit.
Based on the information about the driver detected by the driver detection unit, the mirror direction calculation unit that calculates the direction of the door mirror on the passenger seat side as seen from the driver, and the mirror direction calculation unit.
Based on the in-vehicle image captured by the image acquisition unit, the passenger detection unit that detects the seating position and posture of the passenger in the passenger seat, and the passenger detection unit.
Based on the direction of the door mirror on the passenger seat side calculated by the mirror direction calculation unit and the seating position and posture of the passenger detected by the passenger detection unit, the passenger is on the passenger seat side of the driver. Passenger condition determination unit that determines whether or not the visibility is obstructed,
The status information acquisition department that acquires vehicle status information,
When the passenger state determination unit determines that the passenger is obstructing the view of the driver on the passenger seat side, the driver's line of sight is the door mirror on the passenger seat side based on the vehicle condition information. A visual prediction unit that predicts whether or not it is directed in a direction,
A driving support program for functioning as a warning unit that outputs warning information when the visual prediction unit predicts that the driver's line of sight will be directed toward the door mirror on the passenger seat side.

Images