JP7236897B2 - Driving support method and driving support device - Google Patents

Description

The present invention relates to a driving assistance method and a driving assistance device.

In Patent Document 1, the cause of the difficulty in implementing the automatic driving function is the first sensor that provides information to this function, and when it is difficult for the passenger to take over driving, the second An automatic driving control system that temporarily controls running such as stopping or decelerating using information from two sensors is described.

JP 2017-165296 A

However, the substitution of the second sensor in Patent Document 1 is a temporary measure until handover to manual driving, and is insufficient to continue driving the vehicle in automatic driving.
An object of the present invention is to enable continuation of automatic driving even when a sensor for detecting objects around an automatic driving vehicle enters a non-detection state in which the object cannot be detected.

According to one aspect of the present invention, there is provided a driving assistance method for determining the driving behavior of an automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction. In the driving support method, the controller determines whether or not the sensor is in a non-detection state in which the sensor cannot detect an object, and if the sensor determines that the sensor is in a non-detection state, automatically performs A process of granting monitoring authority to one of the multiple occupants of the driving vehicle to monitor objects around the autonomous driving vehicle, and executing driving actions on the autonomous driving vehicle according to the monitoring results of the occupants who have been granted the monitoring authority. and perform processing to cause

ADVANTAGE OF THE INVENTION According to the aspect of this invention, even if the sensor which detects the object around an automatic driving vehicle will be in the non-detection state which cannot detect an object, automatic driving|operation can be continued.

1 is a schematic configuration diagram of an example of a driving assistance device according to an embodiment of the present invention; FIG. It is a figure which shows an example of the information presentation image for surveillance authorities. It is a figure which shows an example of the image presentation image for operation-authorized persons. 2 is a block diagram showing an example of a functional configuration of a controller in FIG. 1; FIG. It is a figure which shows an example of a passenger|crew state display image. It is a figure which shows an example of a passenger|crew state display image. It is a figure which shows an example of a passenger|crew state display image. It is a figure which shows an example of a passenger|crew state display image. FIG. 10 is an explanatory diagram of an example of a method of determining a seat where a monitoring authority sits; FIG. 10 is an explanatory diagram of an example of a method of determining a seat where a monitoring authority sits; FIG. 10 is an explanatory diagram of an example of a method of determining a seat where a monitoring authority sits; FIG. 10 is an explanatory diagram of an example of a method of determining a seat where a monitoring authority sits; FIG. 10 is an explanatory diagram of an example of a method of determining a seat where a monitoring authority sits; FIG. 11 is a diagram showing an example of an image presentation image for an operation authority when giving authority to a monitoring authority; FIG. 10 is a diagram showing an example of an information presentation image for a monitoring authority while waiting for approval or disapproval of a driving action by an operating authority; FIG. 10 is a diagram showing an example of an information presentation image for a monitoring authority that notifies approval of a driving action by an operating authority. FIG. 10 is a diagram showing an example of an information presentation image for a supervisory authority when a supervisory authority is given authority to approve a driving action. FIG. 10 is a diagram showing an example of an information presentation image for an operating authority when a monitoring authority is given authority to approve driving behavior. FIG. 10 is a diagram showing an example of an information presentation image for a supervisory authority while the system accepts approval or disapproval of a driving action by a supervisory authority; FIG. 11 is a diagram showing an example of an authorized supervisor information presentation image that notifies that the system has accepted the approval of the driving action by the authorized supervisor. It is a flow chart which shows an example of operation of a driving assistance device of an embodiment. It is a flow chart which shows an example of operation of a driving assistance device of an embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that each drawing is schematic and may differ from the actual one. Further, the embodiments of the present invention shown below are examples of apparatuses and methods for embodying the technical idea of the present invention. are not specific to the following: Various modifications can be made to the technical idea of the present invention within the technical scope defined by the claims.

(composition)
Please refer to FIG. The driving support device 1 automatically drives the self-driving vehicle without the involvement of the driver, based on the surrounding driving environment of the self-driving vehicle (hereinafter referred to as "own vehicle") equipped with the driving support device 1. It performs automatic driving control and driving support control that assists the driver in driving the own vehicle.
The driving support control may include driving control such as automatic steering, automatic braking, constant speed driving control, lane keeping control, and merging support control, as well as outputting a message prompting the driver to perform steering operation or deceleration operation. .

The driving assistance device 1 includes an ambient environment sensor 2, a map database 3, a vehicle sensor 4, an occupant sensor 5, user interfaces 6FR, 6FL, 6RR, and 6RL, a navigation system 7, a controller 8, and a vehicle control actuator. 9. In the drawings, the map database is denoted as "map DB".
The surrounding environment sensor 2 may be, for example, a group of sensors mounted on the own vehicle to detect the surrounding environment of the own vehicle and objects around the own vehicle. The surrounding environment sensors 2 include, for example, a left sensor that detects objects on the left side of the vehicle, a left rear sensor that detects objects on the left rear side, a left front sensor that detects objects on the left front side, and an object on the right side of the vehicle. A right side sensor, a right rear sensor that detects a right rear object, a right front sensor that detects a right front object, a front sensor that detects a front object, and a rear sensor that detects a rear object.

That is, the detection directions of the left side sensor, the left rear sensor, the left front sensor, the right side sensor, the right rear sensor, the right front sensor, the front sensor, and the rear sensor are respectively the left side, the left rear, the left front, and the right side of the vehicle. forward, right rear, right front, front and rear.
These sensors may include rangefinders and cameras, for example.
A rangefinder and a camera detect the surrounding environment of the own vehicle, such as objects existing around the own vehicle, the relative position between the own vehicle and the object, and the distance between the own vehicle and the object.

The ranging device may be, for example, a laser range finder (LRF) or radar. The camera may be, for example, a stereo camera.
The camera may be a monocular camera, or the same object may be photographed from a plurality of viewpoints by the monocular camera, and the distance to the object may be calculated. Also, the distance to the object may be calculated based on the grounding position of the object detected from the image captured by the monocular camera.
The ambient environment sensor 2 outputs ambient environment information, which is information about the detected ambient environment, to the controller 8 .

The surrounding environment sensor 2 may be any sensor that detects objects around the vehicle, and is not limited to a sensor mounted on the vehicle. For example, the surrounding environment sensor 2 may be a sensor provided in infrastructure such as other vehicles around the own vehicle or a road on which the own vehicle travels. The controller 8 may acquire ambient environment information from these sensors via vehicle-to-vehicle communication or road-to-vehicle communication.

The map database 3 may store high-precision map data (hereinafter simply referred to as "high-precision map") suitable as a map for automatic driving. A high-precision map is map data with higher precision than map data for navigation (hereinafter simply referred to as "navigation map"), and includes more detailed lane-by-lane information than road-by-road information.
For example, a high-definition map includes lane node information indicating a reference point on a lane reference line (for example, a central line within a lane) and lane link information indicating a section of a lane between lane nodes as information for each lane. including.

The lane node information includes the identification number of the lane node, the position coordinates, the number of connected lane links, and the identification number of the connected lane link. The lane link information includes the lane link identification number, lane type, lane width, lane boundary line type, lane shape, lane marking line shape, and lane reference line shape. The high-precision map further includes the types and positional coordinates of features such as traffic lights, stop lines, signs, buildings, utility poles, curbs, crosswalks, etc. that exist on or near the lane, and lane nodes corresponding to the positional coordinates of the features. includes feature information, such as the identification number of the lane and the identification number of the lane link.

Since the high-precision map includes node and link information for each lane, it is possible to specify the lane in which the host vehicle travels on the travel route. The high-precision map has coordinates that can express the position in the extension direction and width direction of the lane. A high-precision map has coordinates (for example, longitude, latitude, and altitude) that can express a position in a three-dimensional space, and lanes and the features may be described as shapes in the three-dimensional space.

The vehicle sensors 4 include sensors that detect the running state of the host vehicle and sensors that detect driving operations performed by the driver.
Sensors that detect the running state of the own vehicle include a vehicle speed sensor, an acceleration sensor, and a gyro sensor.
Sensors that detect driving operations include a steering angle sensor, an accelerator sensor, and a brake sensor. If the vehicle does not have a driver's seat, the sensor for detecting the driving operation can be omitted.

The vehicle speed sensor detects the wheel speed of the own vehicle and calculates the speed of the own vehicle based on the wheel speed.
The acceleration sensor detects acceleration in the longitudinal direction, the acceleration in the vehicle width direction, and the acceleration in the vertical direction of the host vehicle.
The gyro sensor detects the angular velocity of the rotation angle of the host vehicle around three axes including the roll axis, pitch axis and yaw axis.

The steering angle sensor detects the current steering angle, which is the current rotation angle (steering operation amount) of the steering wheel, which is a steering operator.
The accelerator sensor detects the accelerator opening of the own vehicle. For example, the accelerator sensor detects the amount of depression of the accelerator pedal of the host vehicle as the accelerator opening.

The brake sensor detects the amount of brake operation by the driver. For example, the brake sensor detects the depression amount of the brake pedal of the own vehicle as the brake operation amount.
Information on the vehicle's speed, acceleration, angular velocity, steering angle, accelerator opening, and brake operation amount detected by the vehicle sensor 4 is collectively referred to as "vehicle sensor information". Vehicle sensor 4 outputs vehicle sensor information to controller 8 .

The occupant sensor 5 is a sensor that detects the seating position of the occupant in the vehicle and the state of the occupant. The occupant sensor 5 may be an in-vehicle camera that captures images of the occupants on board the vehicle and their surroundings. Further, for example, the occupant sensor 5 may be a pressure sensor that detects the weight and body surface movement of the occupant, or a sensor that detects the direction of the occupant's body and face.
The video imaged by the occupant sensor 5 and information on the weight and body movement of the occupant are collectively referred to as "occupant sensor information". The occupant sensor 5 outputs occupant sensor information to the controller 8 .

The user interfaces 6FR, 6FL, 6RR and 6RL are interface devices for exchanging information between the occupants in the right front seat, the left front seat, the right rear seat and the left rear seat, the controller and the occupants, respectively. The user interfaces 6FR, 6FL, 6RR and 6RL may be collectively referred to as "user interfaces 6".
The user interface 6 may be fixedly installed in the driving assistance device 1 or the interior of the own vehicle, or may be a terminal device (for example, a tablet terminal) separate from the driving assistance device 1 .

The user interfaces 6 each include an input device, a display device, and an audio signal output device.
The input device receives an operation of the controller 8 by the passenger. The input device may be, for example, buttons, dials, sliders, or the like, or may be a touch panel provided on the display device. The input device may be a microphone for receiving voice instructions and information input from the passenger.

The display device outputs various visual information provided by the controller 8 .
In addition, the user interface 6 may include an interface device that provides stimulation by senses other than sight among the five senses of the occupant. For example, the user interface 6 may include an interface device that provides the passenger with tactile or olfactory stimuli.
The audio signal output device outputs various auditory information provided by the controller 8 .

Note that the number of user interfaces 6 is not limited to four, and may be increased or decreased according to the number of seats in the vehicle. Moreover, it may be provided in some seats instead of being provided in all seats. The seats on which the user interface 6 is provided may be restricted so that only some passengers can use it (for example, only the front right seat and the front left seat are provided), and a single user interface 6 may be shared by multiple passengers. may be set to

The navigation system 7 includes a global positioning system (GNSS) receiver, receives radio waves from a plurality of navigation satellites, and measures the current position of the vehicle. A GNSS receiver may be, for example, a global positioning system (GPS) receiver or the like. The navigation system 7 may measure the current position of the host vehicle by, for example, odometry.

When an occupant operates the navigation system 7 to input a destination, the navigation system 7 uses a method based on graph search theory such as the Dijkstra method or A* to calculate a route that the vehicle will travel from the current position to the destination. set. The navigation system 7 presents the passenger with route guidance based on the set scheduled route.
For example, the navigation system 7 may display the planned route and route guidance on the display of the user interface 6 .
The navigation system 7 also outputs information on the set scheduled route and information on the current position of the vehicle to the controller 8 .

The controller 8 is an electronic control unit (ECU) that performs automatic driving control and driving support control of the host vehicle. The controller 8 includes a processor 12 and peripheral components such as a storage device 13 and the like. The processor 12 may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit).
The storage device 13 may include any one of a semiconductor storage device, a magnetic storage device, and an optical storage device. The storage device 13 may include memories such as a register, a cache memory, a ROM (Read Only Memory) used as a main memory, and a RAM (Random Access Memory).

The functions of the controller 8 to be described below are realized, for example, by the processor 12 of the controller 8 executing a computer program stored in the storage device 13 .
Note that the controller 8 may be formed of dedicated hardware for executing each information processing described below.
For example, controller 8 may comprise functional logic circuitry implemented in a general-purpose semiconductor integrated circuit. For example, the controller 8 may comprise a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

The controller 8 receives ambient environment information input from the ambient environment sensor 2 , a high-precision map stored in the map database 3 (a navigation map included in the navigation system 7 may be used, the same applies hereinafter), and from the vehicle sensor 4 Based on the input vehicle sensor information and the current position of the vehicle input from the navigation system 7, a driving action plan for causing the vehicle to travel the planned route set by the navigation system 7 is generated.

Here, the driving action plan is a driving action at the lane level (lane level) in the range of medium and long distances, which defines the lane (lane) in which the vehicle is to be driven and the driving action required to drive in this lane. is the plan. For example, the driving action plan is a plan that determines the driving action such as how many meters before the intersection the lane should be changed to the right turn lane in a scene where the driver turns right at an intersection in front of the intersection.

The controller 8 generates trajectory candidates for driving the own vehicle based on the driving action plan, the motion characteristics of the own vehicle, and the like. The controller 8 evaluates the future risk of each of the track candidates, selects the optimum track, and sets it as the travel track for the vehicle to travel. The travel trajectory includes a target steering angle as well as a speed plan and acceleration/deceleration therefor.
The controller 8 automatically drives the vehicle by driving the vehicle control actuator 9 so that the vehicle travels along the set travel trajectory.

The vehicle control actuator 9 operates the steering mechanism, the accelerator opening and the brake device of the own vehicle according to the control signal from the controller 8 to generate the vehicle behavior of the own vehicle. The vehicle control actuators 9 may comprise, for example, a steering actuator, an accelerator opening actuator and a brake control actuator.
The steering actuator controls the steering direction and steering amount of the steering mechanism of the host vehicle. The accelerator opening actuator controls the accelerator opening of the host vehicle. The brake control actuator controls the braking operation of the brake system of the host vehicle.

Next, operation instructions by the occupant for the automatic driving control of the own vehicle by the controller 8 will be described.
A crew member can give various operational instructions to the automatic driving control by the controller 8 . Operation instructions for automatic driving control include, for example, instructions specifying driving behavior of the host vehicle (for example, instructions to turn left or right at an intersection ahead, lane change, acceleration/deceleration, etc.) and a driving behavior plan.
Further, for example, the operation instruction to the own vehicle may include an instruction to designate the destination or waypoint of the own vehicle.

The passenger inputs operational instructions to the controller 8 via the user interface 6 to designate driving behavior, driving behavior plans, destinations, waypoints, and the like.
Further, for example, the operation instruction for the own vehicle may be an override operation for the automatic driving control of the own vehicle. A passenger may input an override operation to the controller 8 using, for example, a steering wheel, an accelerator pedal, and a brake pedal.

The controller 8 stores the authority (hereinafter referred to as "operation authority") to operate the own vehicle given to the occupant.
When an operation instruction is issued by a passenger, the controller 8 determines whether the passenger who issued the operation instruction is a person to whom operation authority has been granted, ie, an operation authority.
When the person authorized to operate has given an operation instruction, the controller 8 controls the running of the own vehicle according to the operation instruction. That is, the driving action, the driving action plan, the destination, or the waypoint is set according to the operation instruction. Alternatively, the vehicle behavior of the own vehicle is generated according to the override operation.
If a crew member other than the person authorized to operate the vehicle issues an operation instruction, the controller 8 discards the operation instruction.

Next, a case where any one of the surrounding environment sensors 2 cannot detect objects around the host vehicle (that is, a non-detection state) due to a cause such as failure or adhesion of dirt will be described.
When any of the surrounding environment sensors 2 becomes non-detecting, the controller 8 generates a driving action plan based on the surrounding environment information of the remaining sensors that are in the detecting state.
Then, the controller 8 gives the occupant the monitoring authority to monitor the detection direction of the sensor in the non-detection state. Hereinafter, a sensor in a non-detection state may be referred to as a "non-detection state sensor".
The controller 8 determines the occupant to whom the monitoring authority is given according to, for example, the detection direction of the non-detection state sensor, the driving action scheduled to be performed by the own vehicle, and the occupant's seat.

The controller 8 instructs the occupant to whom monitoring authority is granted (hereinafter referred to as "monitoring authority") to detect the non-detecting state sensor and the monitoring object to be monitored by the monitoring authority (for example, the range around the own vehicle to be monitored, direction to be monitored).
See FIG. 2A. For example, the controller 8 displays the information presentation image 20 for the supervisory authority on the display device of the user interface 6 used by the supervisory authority.

The monitoring authority information presentation image 20 includes, for example, a non-detection sensor display 21, a monitoring target display 22, an granted authority display 23, a driving action display 24, a first confirmation button image 25, and a second confirmation button. An image 26 may be included.
A non-detecting sensor display 21 indicates a non-detecting sensor. In the example of FIG. 2A, the non-detection state sensor display 21 indicates that the left front sensor is in the non-detection state.

The monitoring target display 22 indicates the monitoring target to be monitored by the monitoring authority. In the example of FIG. 2A, the monitoring target display 22 indicates that the monitoring authority should monitor the left front area (or the left front direction) of the host vehicle.
The granted authority display 23 indicates that the monitoring authority is currently granted. The driving behavior display 24 displays the driving behavior that the host vehicle is scheduled to perform from now on.
The first confirmation button image 25 and the second confirmation button image 26 receive an input from an authorized person to indicate the monitoring result of the monitoring target.

If there is an obstacle, the first confirmation button image 25 is pressed (or a pressing gesture is performed), and the controller 8 accepts input of an instruction confirming that there is an obstacle in the monitoring target. Conversely, when there are no obstacles, the second confirmation button image 26 is pressed (or a pressing gesture is made) to accept input of an instruction confirming that there are no obstacles in the monitored object.

It should be noted that the notification of non-detection state sensors, monitoring targets, and granted authority is not limited to notification by image display. For example, an audio signal may notify the non-detecting sensor and target. Similarly, input of monitoring results is not limited to manual input such as button operation, and monitoring results may be input by voice input, for example.

When the monitoring result is input by the monitoring authority, the controller 8 notifies the operating authority of the input monitoring result.
See FIG. 2B. For example, the controller 8 displays the information presentation image 30 for the operating authority on the display device of the user interface 6 used by the operating authority.
The operating authority information presentation image 30 includes, for example, a non-detection sensor display 31, a monitored object display 32, an occupant state display image 33, an granted authority display 34, a driving behavior display 35, and a monitoring result display 36. , an approve button image 37 and a deny button image 38 .

The undetected state sensor display 31 and the monitored object display 32 are similar to the undetected state sensor display 21 and the monitored object display 22 shown in FIG. 2A.
The occupant state display image 33 displays the occupant's state. In the example of FIG. 2B, icons 33FR, 33FL, 33RR, and 33RL represent the states of the occupants sitting in the right front seat, left front seat, right rear seat, and left rear seat, respectively.

For example, the icons 33FR to 33RL indicate the orientation of the passenger's seat according to the direction of the projections provided on the icons 33FR to 33RL.
Further, for example, by hatching displayed on the icon 33FL, it is displayed that the occupant sitting in the left front seat is the monitoring authority.
Note that the display mode shown in FIG. 2B is an example, and the mode of displaying the state of the occupant is not limited to this. The occupant status may be displayed by changing the icon color, line type, and size. Also, the status of the occupant may be displayed by character information instead of the image.

In addition to the orientation of the passenger's seat and the monitoring authority, other conditions of the passenger may also be displayed. For example, the occupant state display image 33 may display whether or not the state of the object to be monitored is suitable for monitoring (that is, the state in which monitoring authority can be granted).
For example, when the occupant has concentration and is in a safe or stable state, it may be displayed that the state is suitable for monitoring the object to be monitored.

Conversely, if the occupant is asleep, in a distracted state, or in an unsteady or anxious state, it may indicate an unsuitable condition for monitoring the monitored object.
In addition, the crew status display image 33 may display whether or not the crew has an attribute suitable for monitoring a monitoring target (that is, an attribute for which monitoring authority can be granted). For example, if the occupant is a child, it may be displayed that the occupant does not have attributes suitable for monitoring the object to be monitored. Also, for example, if the person does not have a driver's license, it may be displayed that the person does not have attributes suitable for monitoring the object to be monitored.

The granting authority display 34 displays that the monitoring authority is currently granted to the monitoring authority. The driving behavior display 35 displays the driving behavior that the host vehicle is scheduled to perform from now on. The monitoring result display 36 displays the monitoring result by the monitoring authority.
From these displays, even if one of the surrounding environment sensors 2 is in a non-detecting state, the operator can judge whether the driving behavior determined based on the surrounding environment information of the remaining sensors is safe or not. can.

The approval button image 37 and the denial button image 38 respectively receive an input instructing approval and denial of the driving action by the operating authority.
When approving the driving action, the approval button image 37 is pressed (or a pushing gesture is made), so that the controller 8 accepts input of an instruction to approve the driving action. On the contrary, in the case of denial, the denial button image 38 is pressed (or a pushing gesture is made), so that the controller 8 accepts input of an instruction to denial of the driving action.

It should be noted that the notification of non-detection state sensor, monitoring target notification, occupant state, granted authority, driving behavior, and monitoring result is not limited to notification by image display. For example, these notifications may be provided by audio signals. Similarly, approval and disapproval of driving behavior are not limited to manual input such as button operation, and approval and disapproval of driving behavior may be input by voice input, for example.

When the driving action is approved, the controller 8 causes the own vehicle to perform the approved driving action. As a result, the host vehicle continues to run under automatic driving control. If the driving action is approved, the controller 8 executes minimum risk driving to reduce the risk of running the own vehicle. For example, the minimum risk driving may be driving to stop the own vehicle on the shoulder of the road or driving at a reduced speed.

Next, with reference to FIG. 3, the function of the controller 8 will be explained in detail. The controller 8 includes a three-dimensional object behavior prediction unit 40, a map generation unit 41, a driving action plan generation unit 42, a trajectory generation unit 43, a travel control unit 44, a surrounding environment determination unit 50, and an occupant information acquisition unit 51. , a sensor state determination unit 52 , an occupant state estimation unit 53 , an occupant attribute determination unit 54 , an authority management unit 55 , a display state control unit 56 , a driving behavior approval unit 57 and an alarm generation unit 58 .

The three-dimensional object behavior prediction unit 40 predicts the behavior of mobile objects around the vehicle based on the surrounding environment information input from the surrounding environment sensor 2 and the high-precision map stored in the map database 3 .
Based on the surrounding environment information, the high-precision map, and the prediction result of the three-dimensional object behavior prediction unit 40, the map generation unit 41 generates a route space map that expresses the route around the vehicle and the presence or absence of objects, and the danger of the driving area. Generate a quantified risk map.

The driving action plan generator 42 generates a driving action plan based on these route space map and risk map.
The trajectory generator 43 generates trajectory candidates for driving the own vehicle based on the driving action plan generated by the driving action plan generator 42, the motion characteristics of the own vehicle, and the route space map. The trajectory generation unit 43 evaluates the future risk of each trajectory candidate based on the risk map, selects the optimum trajectory, and sets it as the travel trajectory for the own vehicle to travel.
The travel control unit 44 drives the vehicle control actuator 9 so that the own vehicle travels along the travel trajectory set by the trajectory generation unit 43 to automatically travel the own vehicle.

The surrounding environment determination unit 50 determines the environment around the own vehicle. For example, the surrounding environment determination unit 50 identifies surrounding objects existing around the own vehicle, the direction in which the surrounding object exists, the distance to the surrounding object, the relative distance to the surrounding object, the type of the surrounding object, the attribute (for example, static whether it is an object or a moving object, whether it is a vehicle or a pedestrian, etc.). Surrounding environment determination unit 50 outputs the determination result to occupant state estimation unit 53 .

The occupant information acquisition unit 51 acquires occupant information based on occupant sensor information from the occupant sensor 5 and input information input by the occupant through the input device of the user interface 6, and recognizes the occupant.
For example, the occupant information acquisition unit 51 acquires the positions of the occupants (seating positions), the number of occupants, the postures of the occupants, the directions of the seats of the occupants, the direction of the line of sight of the occupants, the facial Information such as body orientation, eye open rate, heart rate, objects near the occupant, whether the occupant is an adult or a child, etc. may be acquired as occupant information.

Further, for example, the occupant information acquisition unit 51 may acquire the occupant's posture and heartbeat as occupant information based on the occupant's weight and body surface movement detected by the pressure sensor included in the occupant sensor 5 .
Further, for example, the occupant information acquisition unit 51 may acquire the presence or absence of a driver's license and the age of the occupant as the occupant information based on the information input by the input device of the user interface 6 . The occupant information acquisition unit 51 may acquire an image of a driver's license (whether or not each occupant has a driver's license) using an in-vehicle camera.
The occupant information acquisition unit 51 outputs the acquired occupant information to the occupant state estimation unit 53 , the occupant attribute determination unit 54 , and the display state control unit 56 .

The sensor state determination unit 52 determines whether or not any one of the ambient environment sensors 2 is in a non-detection state. For example, the sensor state determination unit 52 may determine that the ambient environment sensor 2 is in the non-detection state when the ambient environment information is no longer output from the ambient environment sensor 2 .
Further, for example, the sensor state determination unit 52 may determine that the ambient environment sensor 2 has entered the non-detection state when the ambient environment information output from the ambient environment sensor 2 has remained unchanged for a predetermined period of time.
Sensor state determination unit 52 outputs the determination result to authority management unit 55 .

The occupant state estimation unit 53 determines the state of the occupant, specifically, the state in which the occupant can check the surroundings of the vehicle, based on the determination result of the surrounding environment determination unit 50 and the occupant information acquired by the occupant information acquisition unit 51. In other words, it is estimated whether the target is in a state suitable for monitoring (state in which monitoring authority can be granted).
For example, the occupant state estimating unit 53 determines the degree of danger of the surrounding objects identified by the surrounding environment determining unit 50, and determines whether the surrounding objects of high danger are detected by the occupant based on the posture of the occupant, the direction of the line of sight of the occupant, and the direction of the face of the occupant. is responding appropriately.

When the occupant is appropriately reacting to surrounding objects with a high degree of danger, the occupant state estimation unit 53 estimates that the occupant has concentration and is in a state suitable for monitoring the object to be monitored. When the occupant does not react appropriately to surrounding objects with a high degree of danger, the occupant state estimation unit 53 estimates that the occupant is in a careless state lacking concentration and is not in a state suitable for monitoring the object to be monitored. .
Further, for example, the occupant state estimating unit 53 determines whether the occupant is concentrated or in a careless state based on the occupant's eye opening rate, and estimates whether the occupant is in a state suitable for monitoring the object to be monitored. .

Further, for example, if the occupant's heart rate exceeds a threshold value or the heart rate is not stable, the occupant state estimating unit 53 determines that the occupant is in an unstable or anxious state and is not in a state suitable for monitoring the object to be monitored. presume. If the occupant's heart rate is equal to or less than the threshold and the heart rate is stable, it is estimated that the occupant is in a stable or secure state and is in a state suitable for monitoring the object to be monitored.
Further, for example, the occupant state estimating unit 53 determines whether the occupant is in an awake state or in a sleeping state based on the image of the occupant, and estimates that if the occupant is in a sleeping state, the state is not suitable for monitoring the object to be monitored. .
Occupant state estimation unit 53 outputs the estimation result to authority management unit 55 and display state control unit 56 .

In addition, for example, the occupant state estimation unit 53 determines whether the occupant is performing an activity using the eyes other than monitoring based on the image of the occupant and objects (for example, smartphones, games, books, food and drink) around the occupant. determine whether or not Activities that use eyes other than monitoring may be, for example, operating a smart phone, playing games, reading, eating and drinking.
The occupant state estimation unit 53 outputs to the authority management unit 55 information as to whether or not the occupant is performing an activity that requires eyes other than monitoring.

The occupant attribute determination unit 54 determines occupant attributes based on the occupant information acquired by the occupant information acquisition unit 51 . Specifically, it is determined whether or not the occupant has an attribute suitable for monitoring the object to be monitored.
Attributes suitable for monitoring of a monitored subject may include, for example, possession of a driver's license, being of age, and the like. Attributes unsuitable for monitoring of a monitored subject may include, for example, not having a driver's license or being a child.

For example, the occupant attribute determination unit 54 determines whether or not the occupant has a driver's license based on the information input by the input device of the user interface 6 . The occupant attribute determination unit 54 may determine whether or not the occupant has a driver's license based on the image of the driver's license acquired by the occupant information acquisition unit 51 . Alternatively, whether or not the occupant has a driver's license can be determined by inputting whether or not the occupant has a driver's license in advance using the user interface 6 when boarding the vehicle, and storing the entered information in a memory (not shown) together with the seating position. The occupant attribute determination unit 54 may read out information stored in the memory for determination.
Also, the occupant attribute determination unit 54 determines whether the occupant is an adult or a child based on the information input by the input device of the user interface 6 . Alternatively, whether the occupant is an adult or a child may be determined based on the weight or physique detected by the occupant sensor 5 .
When the occupant information acquisition unit 51 acquires information as to whether the occupant is an adult or a child from the image, the occupant attribute determination unit 54 may acquire this information from the occupant information acquisition unit 51 .
The occupant attribute determination unit 54 outputs the determination result to the authority management unit 55 and the display state control unit 56 .

The display state control unit 56 generates the occupant state display image 33 based on the occupant information acquired by the occupant information acquisition unit 51 , the estimation result of the occupant state estimation unit 53 , and the determination result of the occupant attribute determination unit 54 .
For example, the display state control unit 56 may indicate which seat the occupant is seated by displaying or not displaying the icons 33FR to 33RL of the occupant state display image 33 .
Further, for example, the directions of the icons 33FR to 33RL of the occupant state display image 33 (that is, the directions of the projections provided on the icons 33FR to 33RL) may be set according to the orientation of the occupant's seat acquired by the occupant information acquisition unit 51. may indicate the orientation of the occupant's seat.

For example, when the right front seat, left front seat, right rear seat, and left rear seat all face forward, the display state control unit 56 displays the occupant state display image 33 shown in FIG. 2B.
Please refer to FIGS. 4A-4D. The occupant state display image 33 of FIG. 4A is a state in which the right front seat and left front seat face rearward, and the right rear seat and left rear seat face front, that is, each seat faces inward in the front-rear direction. indicates

The occupant state display image 33 in FIG. 4B is a state in which the right front seat and left front seat face forward, and the right rear seat and left rear seat face rearward, that is, each seat faces outward in the front-rear direction. indicates
The occupant state display image 33 in FIG. 4C is a state in which the right front seat and the right rear seat face left, and the left front seat and left rear seat face right, that is, each seat faces inward in the lateral direction (vehicle width direction). occupies a state facing the

The occupant state display image 33 of FIG. 4D is in a state in which the right front seat and the right rear seat face the right side, and the left front seat and the left rear seat face the left side, that is, each seat faces outward in the lateral direction. occupy the state.
Further, the display state control section 56 may display whether or not the occupant is in a state suitable for monitoring the monitoring target based on the estimation result of the occupant state estimating section 53 . The display state control unit 56 may display whether or not the occupant has an attribute suitable for monitoring the object to be monitored based on the determination result of the occupant attribute determination unit 54 .

Please refer to FIG. The authority management unit 55 stores the operation authority given to the passenger. That is, it memorizes which passenger has the operation authority.
Authority management unit 55 outputs the information of the person authorized to operate to display state control unit 56 and driving behavior approval unit 57 .
Further, when any of the surrounding environment sensors 2 is in a non-detecting state, the authority management unit 55 grants monitoring authority to monitor objects around the own vehicle to any of the plurality of occupants.

The authority management unit 55 gives the passenger (i.e., (supervisory authority) to determine the seat position.
In the determination method described below with reference to FIGS. 5 to 9, the authority management unit 55 determines the detection direction of the non-detection state sensor, the scheduled driving behavior, and the orientation of the seat where the passenger sits. to decide who has the monitoring authority.

5 to 9 show driving behavior in various road environments (e.g., signalized intersections, railroad crossings, multi-lane roads on one side, intersections, narrow roads, parking lots, highways, branch sections, and merging sections) and non-detection state sensors. Shows candidates for the seat of the person authorized to monitor according to the direction of detection.
FIG. 5 shows candidates for the case where each seat faces forward (FIG. 2B), FIG. 6 shows candidates for the case where each seat faces inward in the longitudinal direction (FIG. 4A), and FIG. FIG. 8 shows candidates for the case where the seats face outward in the longitudinal direction (FIG. 4B), FIG. 8 shows candidates for the case where each seat faces inward in the lateral direction (FIG. 4C), and FIG. The candidates are shown when facing outwards (FIG. 4D).

A double circle indicates the first priority candidate for the seat of the supervisory authority, and a single circle indicates the second priority candidate. For example, in FIG. 5 (when each seat faces forward), the left rear sensor whose detection direction is the left rear is in a non-detection state, and the scheduled driving action is to change to the left lane (left lane change). ), the left rear seat is selected as the first priority candidate and the left front seat is selected as the second priority candidate.

In addition, when the right side sensor whose detection direction is the right side is in a non-detecting state, and the scheduled driving action is to turn right or enter a right branch road in a branch section (right exit), the right front Both the seat and the right rear seat are selected as first priority candidates.
Also, in FIG. 8 (when each seat faces inward in the lateral direction), the left side sensor whose detection direction is the left side is in a non-detecting state, and the planned driving action is a left turn or a left turn in a fork section. In the case of advancing to a branch road (leaving left), both the left front seat and the left rear seat are selected as the first priority candidates, and both the right front seat and the right rear seat are selected as the second priority candidates. be done.

When the first priority candidate and the second priority candidate are selected, the authority management unit 55 first determines whether or not an occupant is sitting in the first priority candidate seat.
When an occupant sits on the seat of the first priority candidate, the authority management unit 55 makes the occupant sitting on the seat of the first priority candidate a candidate for the monitoring authority. If no occupant is sitting in the first priority candidate seat, the authority management unit 55 determines whether or not the occupant is sitting in the second priority candidate seat.
When an occupant sits in the seat of the second priority candidate, the authority management unit 55 makes the occupant sitting in the seat of the second priority candidate a candidate for the monitoring authority. If no occupant is seated in the seat of the second priority candidate, no monitoring authority is designated.

When a plurality of first priority candidates are selected and crew members are seated in each of the plurality of first priority candidate seats, the authority management unit 55 selects one of these crew members as a candidate for the monitoring authority. Alternatively, a plurality of crew members may be candidates for the supervisory authority.
Further, when a plurality of second priority candidates are selected, no crew member is seated in the seat of the first priority candidate, and a crew member is seated in each of the plurality of seats of the second priority candidate, the authority management unit 55 may select one of these crew members as a candidate for the supervisory authority, or may select a plurality of crew members as candidates for the supervisory authority.

Next, the authority management unit 55 determines whether or not the candidate for the monitoring authority has an attribute suitable for monitoring the monitoring target based on the determination result of the passenger attribute determination unit 54 . Moreover, the authority management unit 55 determines whether or not the candidate for the monitoring authority is in a state suitable for monitoring the monitoring object based on the estimation result of the passenger state estimation unit 53 .
The authority management unit 55 designates a candidate who has an attribute suitable for monitoring the monitoring target and is in a state suitable for monitoring the monitoring target as the final candidate for the monitoring authority. If none of the monitoring authority candidates has attributes suitable for monitoring the monitoring target or is not in a state suitable for monitoring the monitoring target, the authority management unit 55 does not designate a monitoring authority.
The authority management unit 55 outputs the information on the final candidate for the person with authority to monitor to the display state control unit 56 .

When the authority management unit 55 designates the final candidate for the monitoring authority, the display state control unit 56 changes the image for receiving the input for selecting the authority to be granted to the final candidate by the operation authority to the information presentation image for the operation authority. 30.
Please refer to FIG. The operating authority information presentation image 30 includes a candidate display 60 indicating the final candidate for the monitoring authority specified by the authority management unit 55, a first authorization button image 61, and a second authorization button image 62. , a cancel button image 63 is displayed. In the example of FIG. 10, the occupant in the left front seat is shown as the final candidate for the supervisory authority.

In the case of granting the monitoring authority, the first authority granting button image 61 is pressed (or a pressing gesture is made), so that the authority management unit 55 prompts the final candidate to input an instruction to grant only the monitoring authority. accept. The authority management unit 55 determines the final candidate shown in the candidate display 60 as a monitoring authority having only monitoring authority.
When granting not only the monitoring authority but also the approval authority for approving driving behavior, the second authorization button image 62 is pressed (or a pressing gesture is made).

As a result, the authority management unit 55 receives input of an instruction to give the final candidate the monitoring authority and the approval authority. The authority management unit 55 determines the final candidate shown in the candidate display 60 as a monitoring authority having monitoring authority and approval authority.
On the other hand, when the cancel button image 63 is pressed (or a pressing gesture is made), the authority management unit 55 accepts an instruction to grant neither the monitoring authority nor the approval authority to the final candidate. The authority management unit 55 does not determine the monitoring authority, and the monitoring authority is absent.
The authority management unit 55 outputs information about the determined monitoring authority to the display state control unit 56 , the driving action approval unit 57 and the alarm generation unit 58 .

When the authority management unit 55 determines the monitoring authority having only the monitoring authority, the display state control unit 56 displays the information presentation image 20 for the monitoring authority shown in FIG. 2A on the display device of the user interface 6 used by the monitoring authority. display.
The display state control unit 56 receives an input from the supervisory authority to indicate the monitoring result on the supervisory authority information presentation image 20 .

When the monitoring result by the monitoring authority is input, the display state control unit 56 displays the operating authority information presentation image 30 shown in FIG. 2B on the display device of the user interface 6 used by the operating authority. Note that the sensor non-detection state display 31 and the monitored object display 32 may be displayed before the monitoring result is input (for example, when it is determined that the sensor is in the non-detection state). The occupant state display image 33 may be displayed before the monitoring result is input (for example, when the occupant is detected). The granted authority display 34 may be displayed before the monitoring result is input (for example, when the monitoring authority is determined).

When a plurality of authorized supervisors input monitoring results, the display state control unit 56 may select the supervisory result of the supervisor with the longest monitoring time and display it in the operator-authorized information presentation image 30 . . The display state control unit 56 determines whether or not the monitoring authority is monitoring the monitoring target based on the occupant information (the occupant's posture, the direction of the occupant's line of sight, the direction of the face) acquired by the occupant information acquisition unit 51, for example. You can judge whether

On the other hand, the display state control unit 56 displays the supervisory authority information presentation image 20 shown in FIG. 11A on the display device of the user interface 6 used by the supervisory authority. The monitoring authority information presentation image 20 includes an indication 70 indicating that the operating authority is waiting for the operation to approve or disapprove of the driving action. This display 70 allows the monitoring authority to recognize that the monitoring target should be monitored continuously.

Please refer to FIG. The driving action approval unit 57 receives an input from the operating authorized person instructing approval or disapproval of the driving action on the operating authorized person information presentation image 30 .
Upon accepting the approval of the driving action, the driving action approval unit 57 outputs an approval signal for approving the driving action determined by the driving action plan generation unit 42 to the trajectory generation unit 43 .

On the other hand, the display state control unit 56 displays the information presentation image 20 for the authorized supervisor shown in FIG. 11B on the display device of the user interface 6 used by the authorized supervisor. The monitoring authority information presentation image 20 includes an indication 71 indicating that the driving behavior has been approved by the operating authority. This display 71 allows the monitoring authority to recognize that monitoring of the monitoring target may be stopped.
On the other hand, when receiving the denial of the driving action, the driving action approval unit 57 outputs a denial signal for denying the driving action to the trajectory generation unit 43 .

Further, the driving action approval unit 57 determines whether or not the monitoring authority has noticed that the monitoring authority has been granted before a predetermined period of time before the own vehicle performs the driving action. The predetermined time may be 10 seconds, for example.
For example, the driving action approval unit 57 allows the monitoring authority to display the information presentation image 20 for the monitoring authority based on the occupant information acquired by the occupant information acquisition unit 51 (the posture of the occupant, the direction of the occupant's line of sight, the direction of the face). You may determine whether or not you have seen

The driving action approval unit 57 may determine whether or not the monitoring authority has been granted the monitoring authority based on whether or not the monitoring authority has seen the information presentation image 20 for the monitoring authority.
Further, when the information presentation image 20 for the monitoring authority is displayed, the driving action approval unit 57 approves the monitoring authority according to whether or not a predetermined operation has been performed on the user interface 6 and whether or not a predetermined voice input has been performed. may be determined whether or not the user has noticed that the
If the monitoring authority is not aware that the monitoring authority has been granted before a predetermined period of time before the own vehicle performs the driving behavior, the driving behavior approval unit 57 sends a denial signal to the trajectory generation unit 43 to deny the driving behavior. Output.

Further, the driving behavior approval unit 57 determines whether or not the monitoring authority has input the monitoring result within a predetermined time after the driving behavior display 24 is displayed on the information presentation image 20 for the monitoring authority. If the monitoring result is not input within a predetermined time after the driving action display 24 is displayed, the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 to deny the driving action.

On the other hand, when the authority management unit 55 determines the monitoring authority having the monitoring authority and the approval authority, the display state control unit 56 displays the monitoring authority shown in FIG. 12A on the display device of the user interface 6 used by the monitoring authority. information presentation image 20 is displayed.
The monitoring authority information presentation image 20 may include an approval button image 72 and a denial button image 73 .

The approval button image 72 and the denial button image 73 respectively receive an input instructing approval and denial of the driving action by the monitoring authority.
When approving the driving action, the approval button image 72 is pressed (or a pushing gesture is made), whereby the driving action approval unit 57 accepts input of an instruction to approve the driving action. Conversely, when disapproving, the disapproval button image 73 is pressed (or a pushing gesture is made), whereby the driving action approval unit 57 accepts input of an instruction to disapproval of the driving action.

On the other hand, the display state control unit 56 displays the information presentation image 30 for the authorized operator shown in FIG. 12B on the display device of the user interface 6 used by the authorized operator.
When the monitoring authority is granted the approval authority, the approval button image 37 and the denial button image 38 may be omitted from the information presentation image 30 for the operating authority. Alternatively, the operating authority information presentation image 30 may include an indication 64 indicating that the monitoring authority is waiting for the operation to approve or disapprove of the driving behavior.

When the approval or disapproval of the driving action by the supervisory authority is input, the display state control unit 56 displays the supervisory authority information presentation image 20 shown in FIG. 13A on the display device of the user interface 6 used by the supervisory authority. indicate. The supervisory authority information presentation image 20 includes a display 74 indicating that the driving behavior approval unit 57 is processing the approval input or disapproval input of the driving behavior by the supervisory authority. This display 74 allows the monitoring authority to recognize that the monitoring target should be monitored continuously.

When the approval of the driving action by the monitoring authority is input, the driving action approval unit 57 outputs an approval signal for approval of the driving action to the trajectory generation unit 43 . When the output of the approval signal is completed, the display state control unit 56 displays the supervisory authority information presentation image 20 shown in FIG. 13B on the display device of the user interface 6 used by the supervisory authority. The supervisory authority information presentation image 20 includes a display 75 indicating that the reception of the approval input for the driving action has been completed. The display 75 allows the monitoring authority to recognize that monitoring of the monitoring target may be stopped.

On the other hand, when receiving the denial of the driving action, the driving action approval unit 57 outputs a denial signal for denying the driving action to the trajectory generation unit 43 .
Further, if the person with the authority to monitor is not aware that the person with the authority to monitor has been given the authority to monitor before a predetermined time before the own vehicle performs the driving action, the driving action approval unit 57 sends a denial signal to the trajectory generation unit to deny the driving action. 43.
If the monitoring result is not input within a predetermined time after the driving action display 24 is displayed, the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 to deny the driving action.

When the driving action approval unit 57 outputs an approval signal for approving the driving action to the trajectory generation unit 43, the trajectory generation unit 43 sends the trajectory generated according to the driving action plan generated by the driving action plan generation unit 42 to the driving control unit 44. Output to As a result, the self-vehicle continues to run under automatic driving control.
When the driving action approval unit 57 outputs a denial signal for denying the driving action to the trajectory generation unit 43 , the trajectory generation unit 43 generates a travel trajectory for performing minimum risk driving and outputs it to the travel control unit 44 . As a result, the host vehicle performs minimum risk driving.

Please refer to FIG. The authority management unit 55 acquires from the occupant state estimation unit 53 information as to whether or not the occupant is performing an activity that requires eyes other than monitoring. The authority management unit 55 determines whether or not the person with authority to monitor is performing an activity other than monitoring that uses eyes. Authority management unit 55 outputs the determination result to alarm generation unit 58 .

When the person authorized to monitor is engaged in an activity using eyes other than monitoring, the alarm generator 58 generates an alarm based on a stimulus other than sight and gives it to the person authorized to monitor via the user interface 6 . For example, the alarm generator 58 may generate an alarm based on any one of auditory, tactile, and olfactory stimuli. As a result, it is possible to inform the monitoring authority that the monitoring target should be monitored, and prompt the monitoring authority to perform the monitoring.

(motion)
Next, an example of the operation of the driving assistance system 1 of the embodiment will be described with reference to FIGS. 14A and 14B.
In step S1, the sensor state determination unit 52 determines whether or not any of the ambient environment sensors 2 is in a non-detection state. If any one of the ambient environment sensors 2 is in a non-detection state (step S1: Y), the process proceeds to step S3. If none of the ambient environment sensors 2 are in the non-detection state (step S1: N), the process proceeds to step S2.

In step S<b>2 , the driving action approval unit 57 outputs an approval signal for approving the driving action determined by the driving action plan generation unit 42 to the trajectory generation unit 43 . As a result, the vehicle continues to run under automatic driving control. After that, the process proceeds to step S23.
In step S3, the authority management unit 55 selects the seat of the passenger to whom the monitoring authority is granted according to the detection direction of the non-detection state sensor and the driving action that the vehicle is scheduled to perform from now on.

In step S4, the occupant information acquisition unit 51 detects the occupant.
In step S5, the authority management unit 55 determines whether or not an occupant is sitting in the seat selected in step S3. If an occupant is sitting in the seat selected in step S3 (step S5: Y), the authority management unit 55 makes this occupant a candidate for monitoring authority. After that, the process proceeds to step S7.

If no occupant is sitting in the seat selected in step S3 (step S5: N), the process proceeds to step S6.
In step S<b>6 , the driving action approval unit 57 outputs a denial signal for approving the driving action determined by the driving action plan generation unit 42 to the trajectory generation unit 43 . As a result, the own vehicle executes minimum risk driving to reduce the risk of running the own vehicle. After that, the process proceeds to step S23.

In step S7, the authority management unit 55 determines whether or not the candidate for the monitoring authority has an attribute suitable for monitoring. If the monitoring authority candidate has an attribute suitable for monitoring (step S7: Y), the process proceeds to step S9. If the monitoring authority candidate does not have attributes suitable for monitoring (step S7: N), the process proceeds to step S8.
In step S<b>8 , the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 . As a result, the host vehicle performs minimum risk driving. After that, the process proceeds to step S23.

In step S9, the authority management unit 55 determines whether or not the monitoring authority candidate is in a state suitable for monitoring. If the monitoring authority candidate is in a state suitable for monitoring (step S9: Y), the process proceeds to step S11. If the monitoring authority candidate is not in a state suitable for monitoring (step S9: N), the process proceeds to step S10.
In step S<b>10 , the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 . As a result, the host vehicle performs minimum risk driving. After that, the process proceeds to step S23.

In step S11, the display state control unit 56 displays the information presentation image 30 for the authorized operator shown in FIG. 10 on the display device of the user interface 6 used by the authorized operator. When the operating authority grants the monitoring authority to the candidate for the monitoring authority, the crew member to whom the monitoring authority is given becomes the monitoring authority.
The display state control unit 56 displays the monitoring authority information presentation image 20 shown in FIG. 2A on the display device of the user interface 6 used by the operating authority.

In step S12, the authority management unit 55 determines whether or not the monitoring authority is performing an activity other than monitoring that uses eyes. If the monitoring authority is performing an activity that uses eyes other than monitoring (step S12: Y), the process proceeds to step S13. If not (step S12: N), step S13 is skipped and the process proceeds to step S14.
In step S<b>13 , the alarm generator 58 generates an alarm based on a stimulus other than visual sense, and gives it to the person authorized to monitor via the user interface 6 . After that, the process proceeds to step S14.

In step S14, the driving action approval unit 57 determines whether or not the monitoring authority has noticed that the monitoring authority has been granted a predetermined time before the own vehicle performs the driving action. When it is noticed that the monitoring authority has been granted before the predetermined time before the driving action is performed (step S14: Y), the process proceeds to step S16. If the driver has not noticed that he/she has been given the monitoring authority before the predetermined time before the driving action is performed (step S14: N), the process proceeds to step S15.
In step S<b>15 , the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 . As a result, the host vehicle performs minimum risk driving. After that, the process proceeds to step S23.

In step S<b>16 , the driving behavior approval unit 57 determines whether or not the monitoring authority has input the monitoring result within a predetermined time after the driving behavior display 24 is displayed on the information presentation image 20 for the monitoring authority. If the monitoring result is input within a predetermined time after the driving action display 24 is displayed (step S16: Y), the process proceeds to step S18. If the monitoring result is not input within a predetermined time after the driving action display 24 is displayed (step S16: N), the process proceeds to step S17.
In step S<b>17 , the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 . As a result, the host vehicle performs minimum risk driving. After that, the process proceeds to step S23.

In step S18, the display state control unit 56 displays the information presentation image 30 for the authorized operator shown in FIG. 2B on the display device of the user interface 6 used by the authorized operator. The operating authority information presentation image 30 includes a monitoring result display 36 showing the monitoring result by the monitoring authority.
In step S19, the driving action approval unit 57 determines whether or not the operation authority has approved the driving action that the host vehicle is to perform from now on.

If the operating authority approves the driving action (step S19: Y), the process proceeds to step S20. If the operating authority does not approve the driving action (step S19: N), the process proceeds to step S21.
In step S<b>20 , the driving action approval unit 57 outputs an approval signal for approving the driving action to the trajectory generation unit 43 . As a result, the vehicle continues to run under automatic driving control. After that, the process proceeds to step S22.
In step S<b>21 , the driving action approval unit 57 outputs a denial signal to the trajectory generation unit 43 . As a result, the host vehicle performs minimum risk driving. After that, the process proceeds to step S22.

In step S<b>22 , the display state control unit 56 displays a display 71 indicating that the approval or disapproval of the driving behavior by the operating authority has been completed on the information presentation image 20 for the monitoring authority.
In step S23, it is determined whether or not the ignition switch (IGN) of the host vehicle has been turned off. If the ignition switch is not turned off (step S23: N), the process returns to step S1. If the ignition switch is turned off (step S23: Y), the process ends.

(Effect of Embodiment)
(1) The controller 8 determines the driving behavior of the automatically driven vehicle based on the detection result of the ambient environment sensor 2 that detects objects around the automatically driven vehicle in a predetermined detection direction. The controller 8 determines whether or not any of the ambient environment sensors 2 is a non-detection sensor that cannot detect an object, and determines that any of the ambient environment sensors 2 is a non-detection sensor. In this case, according to the predetermined detection direction of the non-detection state sensor, a process of granting monitoring authority to monitor objects around the automatically driving vehicle to one of a plurality of occupants of the automatically driving vehicle, and the monitoring authority is granted. and a process of causing the automatically driven vehicle to perform a driving action according to the result of monitoring by the occupant.

As a result, even if one of the surrounding environment sensors 2 is in a non-detection state in which an object cannot be detected, the surrounding environment information of the remaining sensors can be obtained based on the monitoring result of the passenger (monitoring authority) who is given the monitoring authority. It can be determined whether the driving behavior determined based on is safe. Therefore, even if any one of the surrounding environment sensors 2 is in a non-detection state in which an object cannot be detected, the own vehicle can continue to be driven by automatic driving control.

(2) The controller 8 performs a process of causing the automatically driven vehicle to execute a driving action in response to approval of the driving action by a passenger (monitoring authority) to whom monitoring authority is granted.
As a result, even if one of the surrounding environment sensors 2 is in a non-detection state in which an object cannot be detected, the surrounding environment information of the remaining sensors can be obtained based on the monitoring result of the passenger (monitoring authority) who is given the monitoring authority. can approve driving behavior determined based on Therefore, even if any one of the surrounding environment sensors 2 is in a non-detection state in which an object cannot be detected, the own vehicle can continue to be driven by automatic driving control.

(3) The controller 8 performs a process of determining the occupant to whom the monitoring authority is given according to the predetermined detection direction of the non-detection state sensor and the driving behavior. Accordingly, it is possible to determine whether or not the driving behavior performed in the detection direction in which the object should have been detected by the non-detection state sensor is safe.

(4) The controller 8 performs a process of determining to which occupant the monitoring authority is given according to the predetermined detection direction of the non-detection state sensor, the driving behavior, and the position of the seat on which the occupant sits. As a result, the authority to monitor can be given to the occupant who can easily monitor the detection direction in which the non-detection state sensor should have detected the object.

(5) The controller 8 performs a process of determining to which occupant the monitoring authority is given according to the predetermined detection direction of the non-detection state sensor, the driving behavior, and the orientation of the seat on which the occupant sits. As a result, the monitoring authority can be determined so that the occupant can monitor in a natural posture without taking an unreasonable posture.

(6) The controller 8 performs a process of determining whether to grant monitoring authority according to the attributes of the occupant. Accordingly, it is possible to grant monitoring authority to crew members who have attributes suitable for monitoring. Therefore, it is possible to travel based on the traffic rules.
(7) The controller 8 performs a process of determining whether to grant monitoring authority according to the state of the occupant. Accordingly, it is possible to grant monitoring authority to the occupant in a state suitable for monitoring. Therefore, it is possible to travel based on the traffic rules.

(8) The controller 8 performs a process of notifying the passenger (monitoring authority) of the non-detection state to whom the monitoring authority is granted. This allows the monitoring authority to know the area where the detection by the sensor is insufficient.
(9) The controller 8 performs a process of accepting an input indicating that the passenger (monitoring authority) who is given the monitoring authority has confirmed the presence or absence of an obstacle. As a result, it becomes possible to notify the passenger (operating authority) who has the operating authority for the automatic driving vehicle that the monitoring authority has confirmed the presence or absence of the obstacle.

(10) The controller 8 performs a process of notifying a crew member (operation authority) who has the operation authority for the automatic driving vehicle of the monitoring result. Thereby, the operating authority can know the monitoring result by the passenger who is given the monitoring authority.
(11) The controller 8 performs a process of accepting an input indicating that the driver who has the operation authority approves the driving action. Thereby, the vehicle can be run based on the approval of the operating authority who has recognized the monitoring result of the monitoring authority.

(12) The controller 8 performs processing to give a stimulus to an occupant with monitoring authority (monitoring authority) before the driving action. This makes it possible to urge the monitoring authority to monitor.
(13) The controller 8 causes the automatic driving vehicle to perform driving behavior according to the monitoring result of the passenger with the longest monitoring time among the plurality of passengers (monitoring authority) to whom the monitoring authority is granted. As a result, it is possible to give priority to the decision of the supervisory authority with a longer confirmation time and a higher degree of certainty.

REFERENCE SIGNS LIST 1 driving support device 2 ambient environment sensor 3 map database 4 vehicle sensor 5 occupant sensor 6, 6FL, 6FR, 6RL, 6RR user interface 7 navigation system 8 controller 9 Vehicle control actuator 12 Processor 13 Storage device 20 Information presentation image for monitoring authority 30 Information presentation image for operation authority 33 Passenger state display image 40 Three-dimensional object behavior prediction unit 41 Map generation unit 42 Driving action plan generation unit 43 Trajectory generation unit 44 Driving control unit 50 Surrounding environment determination unit 51 Occupant information acquisition unit 52 Sensor state determination unit 53 Occupant state Estimation unit 54 Passenger attribute determination unit 55 Authority management unit 56 Display state control unit 57 Driving behavior approval unit 58 Alarm generation unit

Claims (16)

A driving support method for determining the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
the controller
a process of determining whether the sensor is in a non-detection state in which the object cannot be detected;
When it is determined that the sensor is in the non-detection state, one of a plurality of occupants of the autonomous vehicle is selected according to the predetermined detection direction , and the selected occupant is instructed to a process of granting monitoring authority to monitor the object of
A process of causing the automatically driven vehicle to perform the driving behavior according to the result of monitoring by the crew member to whom the monitoring authority is granted;
A driving support method characterized by performing 2. The driving support method according to claim 1, wherein the controller performs processing for causing the automatically driven vehicle to execute the driving action in response to approval of the driving action by the passenger to whom the monitoring authority is granted. . A driving support method for determining the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
the controller
a process of determining whether the sensor is in a non-detection state in which the object cannot be detected;
When it is determined that the sensor is in the non-detection state, according to the predetermined detection direction and the driving behavior, which of the plurality of occupants of the automatically driven vehicle should be hit by an object around the automatically driven vehicle. a process of determining whether to grant monitoring authority to monitor ;
A process of causing the automatically driven vehicle to perform the driving behavior according to the result of monitoring by the crew member to whom the monitoring authority is granted;
A driving support method characterized by performing 4. The controller according to claim 3, wherein the controller determines the occupant to whom the monitoring authority is granted according to the predetermined detection direction, the driving behavior, and the position of the seat on which the occupant sits. Driving assistance method. 5. The controller according to claim 3 or 4, wherein the controller determines the passenger to whom the monitoring authority is granted according to the predetermined detection direction, the driving behavior, and the orientation of the seat on which the passenger sits. The described driving assistance method. The driving support method according to any one of claims 1 to 5, wherein the controller determines whether or not to grant the monitoring authority according to attributes of the passenger. The driving support method according to any one of claims 1 to 6, wherein the controller determines whether or not to grant the monitoring authority according to the state of the passenger. The driving support method according to any one of claims 1 to 7, wherein the controller performs processing for informing the passenger to whom the monitoring authority is granted of the non-detection state. The driving support according to any one of claims 1 to 8, wherein the controller performs processing for receiving an input indicating that the occupant to whom the monitoring authority has been granted has confirmed the presence or absence of an obstacle. Method. 10. The driving support method according to any one of claims 1 to 9, wherein the controller performs a process of notifying an occupant who has operation authority over the automatic driving vehicle of the monitoring result. 11. The driving support method according to claim 10, wherein the controller performs a process of accepting an input for approving the driving behavior from the passenger having the operation authority. The driving support method according to any one of claims 1 to 11, wherein the controller performs a process of giving a stimulus to the passenger to whom the monitoring authority is granted before the driving action. A driving support method for determining the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
the controller
a process of determining whether the sensor is in a non-detection state in which the object cannot be detected;
When it is determined that the sensor is in the non-detection state, according to the predetermined detection direction, one of the plurality of occupants of the autonomous vehicle is authorized to monitor objects around the autonomous vehicle. a process to give;
A process of causing the automated driving vehicle to perform the driving behavior according to the monitoring result of the crew member with the longest monitoring time among the plurality of crew members to whom the monitoring authority is granted;
A driving support method characterized by performing A driving support device that determines the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
It is determined whether or not the sensor is in a non-detection state in which the object cannot be detected, and if it is determined that the sensor is in the non-detection state, a plurality of automatically driven vehicles are detected in accordance with the predetermined detection direction. select one of the occupants, give the selected occupant a monitoring authority to monitor objects around the automated driving vehicle, and according to the results of monitoring by the occupant to whom the monitoring authority is granted, the automated driving vehicle and a controller that causes the driver to execute the driving behavior. A driving support device that determines the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
It is determined whether or not the sensor is in a non-detection state in which the object cannot be detected, and if the sensor determines that the sensor is in the non-detection state, the automatic detection is performed according to the predetermined detection direction and the driving behavior. Determine which of a plurality of occupants of the driving vehicle is to be given the authority to monitor objects around the automatically driven vehicle, and according to the results of monitoring by the occupant to whom the authority to monitor is granted, the automatic driving A driving support device comprising a controller that causes a vehicle to execute the driving behavior. A driving support device that determines the driving behavior of the automatically driving vehicle based on the detection result of a sensor that detects objects around the automatically driving vehicle in a predetermined detection direction,
It is determined whether the sensor is in a non-detection state in which the object cannot be detected, and if it is determined that the sensor is in the non-detection state, a plurality of the automatically driven vehicles are detected in accordance with the predetermined detection direction. to any of the occupants of the automatic driving vehicle, and according to the monitoring result of the occupant with the longest monitoring time among the plurality of occupants to whom the monitoring authority has been granted, A driving support device comprising a controller that causes an automatically driven vehicle to execute the driving action.

Images