JP7107060B2 - Driving support method and driving support device - Google Patents

Description

The present invention relates to a driving assistance method and a driving assistance device.

Techniques for assisting driving of a vehicle based on biological information such as brain activity of the driver have been proposed. For example, in a vehicle driving support system disclosed in Patent Document 1, a brain network interface is attached to the head of a driver to detect brain activity data of the driver. The driving support control unit specifies a target operation amount of an operation target (steering, braking, etc.) of the vehicle based on the detected brain activity data, and performs drive control of the operation target based on the specified target operation amount.

Japanese Patent Application Laid-Open No. 2008-247118

When driving assistance is performed based on biometric information, there is a risk that inappropriate driving assistance will be performed due to erroneous analysis of the biometric information.
An object of the present invention is to prevent incorrect driving assistance from being performed based on biological information.

In a driving assistance method according to an aspect of the present invention, the presence or absence of a possibility that the driver will perform a driving operation of the own vehicle is determined based on the surrounding environment of the own vehicle and the state of the driver, and the biological information of the driver is used. Based on this, the operation type of the driving operation of the own vehicle by the driver is predicted, and driving assistance of the own vehicle is executed based on the predicted operation type when it is determined that there is a possibility that the driver will perform the driving operation.

According to the aspects of the present invention, it is possible to prevent incorrect driving assistance from being performed based on biometric information.

1 is a schematic configuration diagram of an example of a driving support device according to an embodiment; FIG. It is a graph showing an example of a waveform of an electroencephalogram. FIG. 4 is a schematic diagram for explaining a method of measuring a readiness potential for exercise; 2 is a block diagram showing an example of a functional configuration of a controller shown in FIG. 1; FIG. 4 is a time chart showing determination results as to whether there is a possibility of a driving operation; 4 is a time chart of electroencephalogram waveforms including motor readiness potentials. FIG. 11 is an explanatory diagram of operation timing when T1<T3a; FIG. 11 is an explanatory diagram of operation timing when T1>T3b; It is a flow chart of an example of the driving support method of the embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that each drawing is schematic and may differ from the actual one. Further, the embodiments of the present invention shown below are examples of apparatuses and methods for embodying the technical idea of the present invention. are not specific to the following: Various modifications can be made to the technical idea of the present invention within the technical scope defined by the claims.

(Constitution)
Please refer to FIG. When the driver drives a vehicle equipped with the driving assistance device 1 (hereinafter referred to as "self-vehicle"), the driving assistance device 1 assists the driver in driving the own vehicle. It is a device that supports
The driving assistance device 1 includes an ambient environment sensor 2 , an in-vehicle sensor 3 , a vehicle sensor 4 , an electroencephalogram sensor 5 , a navigation system 6 , a controller 7 and a vehicle control actuator 8 .

The surrounding environment sensor 2 is a sensor that detects the surrounding environment of the own vehicle, for example, objects such as other vehicles and obstacles around the own vehicle.
The ambient environment sensor 2 may be, for example, a rangefinder such as a laser range finder (LRF) or radar. The range finder detects, for example, objects (other vehicles and obstacles) existing around the own vehicle, the relative position between the own vehicle and the object, and the distance between the own vehicle and the object. The ranging device outputs the detected ranging data to the controller 7 .

The ambient environment sensor 2 may be a camera such as a tele camera or a monocular camera. The camera outputs to the controller 7 photographed data of objects existing around the vehicle, road markings such as lane boundaries (e.g., white lines), traffic lights, road signs, curbs, guardrails, and other features. .

The in-vehicle sensor 3 detects the state of the driver in the interior of the vehicle. For example, the in-vehicle sensor 3 may be an in-vehicle camera that is provided inside the vehicle and captures the driver. Further, for example, the in-vehicle sensor 3 may be a touch sensor that detects the state of the driver's grip on the steering wheel or the presence of the driver's foot on the accelerator or brake pedal.
The in-vehicle sensor 3 outputs image data of the driver and detection signals of the touch sensor to the controller 7 .

The vehicle sensors 4 include sensors that detect the running state of the host vehicle and sensors that detect driving operations performed by the driver.
A sensor that detects the running state of the host vehicle may be, for example, a vehicle speed sensor.
The sensors that detect driving operations may be, for example, a steering angle sensor, an accelerator sensor, a brake sensor, and a shift position sensor.

A vehicle speed sensor detects the speed of the own vehicle based on the wheel speed of the own vehicle.
The steering angle sensor detects the current steering angle, which is the current rotation angle (steering operation amount) of the steering wheel, which is a steering operator.
The accelerator sensor detects the accelerator opening of the own vehicle. For example, the accelerator sensor detects the amount of depression of the accelerator pedal of the host vehicle as the accelerator opening.

The brake sensor detects the amount of brake operation by the driver. For example, the brake sensor detects the depression amount of the brake pedal of the own vehicle as the brake operation amount.
The shift position sensor detects the state of the shift lever.
The vehicle sensor 4 outputs the detected vehicle speed, steering angle, accelerator opening, brake operation amount, and shift lever state to the controller 7 .

The electroencephalogram sensor 5 detects an electroencephalogram (brain activity) of a driver (human) who is a subject, and outputs the detected electroencephalogram signal (electroencephalogram data) to the controller 7 .
The electroencephalogram sensor 5 includes an electroencephalogram measurement electrode group (a plurality of electrodes), an amplifier that amplifies a plurality of electroencephalogram signals that are potential changes collected by the electrode group, and a plurality of electroencephalogram signals output from the amplifier. It has a filter for extracting frequency components in a predetermined passband, and an A/D converter for sampling analog data of the extracted electroencephalogram signal at a predetermined sampling period and converting it into digital data.

Note that part of the functions of the electroencephalogram sensor 5 other than the electrode group may be incorporated in the controller 7 .
Also, a sensor other than the electroencephalogram sensor 5 may be used to measure the brain activity of the driver. For example, a sensor that detects biological information from which the driver's brain activity can be estimated, such as cerebral blood flow, heart rate, respiration, perspiration, and face image, may be used.

The electroencephalogram sensor 5 detects weak potential difference signals generated between a plurality of electrodes attached to the driver's head as electrical activity generated in the driver's brain. For example, the controller 7 calculates a motor readiness potential (MRP) generated in the driver's brain by frequency-analyzing the potential difference signal between the electroencephalogram data of each electrode detected by the electroencephalogram sensor 5 .
The motor readiness potential is a type of event-related potential (ERP) indicating a brain reaction that appears as a result of thinking or cognition, and is a potential generated when a person tries to move a hand or leg voluntarily.

The electroencephalogram underlying the motor readiness potential occurs before the driver actually begins to act. Therefore, the ready-to-exercise potential is detected from about 2 seconds before the timing at which the driver starts to act, and is detected to be greater from about 400 ms before. Therefore, by calculating the exercise readiness potential, it is possible to predict the driver's action (behavioral intention) before the driver actually starts the action. Here, the case where the controller 7 calculates the exercise preparatory potential by frequency analysis is exemplified.

FIG. 2 is a diagram showing an example of feature vectors in an electroencephalogram signal. Here, N feature amounts are extracted from the electroencephalogram signal before the driver starts to act, and an electroencephalogram feature vector P=(p1, p2, . . . , pN) is generated. For the feature amount, for example, values sampled at regular equal intervals are used. As shown in FIG. 3, a feature amount of past exercise readiness potentials is stored in a database in advance, and a region D to be arranged in the feature space is determined.

As for the definition of area D, for example, if there are a plurality of samples, the area D is determined as a circle whose center is the center of gravity of vector set {P} and whose radius is standard deviation σ. Then, it is determined whether or not the characteristic vector P of the exercise readiness potential measured from the driver in real time falls within the region D.
The controller 7 determines that an exercise readiness potential is generated when the feature vector P is within the region D, and determines that an exercise readiness potential is not generated when the feature vector P is not within the region D.

Please refer to FIG. The navigation system 6 recognizes the current position of the own vehicle and the road map information at the current position. The navigation system 6 sets the travel route to the destination input by the passenger, and provides route guidance to the passenger according to this travel route.
Furthermore, the navigation system 6 outputs the set travel route and road map information on the travel route to the controller 7 . Information provided from the navigation system 6 to the controller 7 is referred to as "navigation information".

The navigation system 6 includes a navigation controller 61 , a positioning device 62 , a map database 63 , a display section 64 , an operation section 65 , an audio output section 66 and a communication section 67 . Note that the map database is referred to as a map DB in FIG.
The navigation controller 61 is an electronic control unit that controls information processing operations of the navigation system 6 . The navigation controller 61 includes a processor and its peripheral components. The processor may be, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit).

Peripheral parts include storage devices and the like. The storage device may comprise any one of a semiconductor storage device, a magnetic storage device and an optical storage device. The storage device may include memories such as registers, cache memories, ROMs (Read Only Memories) used as main storages, and RAMs (Random Access Memories).

The positioning device 62 measures the current position of the own vehicle. The positioning device 62 may be, for example, a GPS (Global Positioning System) receiver. The positioning device 62 may also measure the current position of the vehicle based on satellite signals from other satellite positioning systems such as GLONASS (Global Navigation Satellite System). The positioning device 62 may also be an inertial navigation device.

The map database 63 stores road map information. The road map information includes information on driving lanes, road line types, road shapes, gradients, number of lanes, legal speed limits (speed limits), road widths, stop lines, intersections, junctions, and junctions.
The display unit 64 outputs various visual information in the navigation system 6 . For example, the display unit 64 may display a map screen of the surroundings of the own vehicle or guidance on a recommended route. Also, the display unit 64 may display a message (for example, a message prompting the driver to perform a steering operation or a deceleration operation) generated by driving assistance control by the driving assistance device 1 .

The operation unit 65 receives the operation of the passenger in the navigation system 6 . The operation unit 65 may be, for example, buttons, dials, sliders, or the like, or may be a touch panel provided on the display unit 64 . For example, the operation unit 65 may receive a destination input operation by the passenger or a switching operation of the display screen of the display unit 64 .

The audio output section 66 outputs various audio information in the navigation system 6 . The voice output unit 66 may output driving guidance based on the set travel route and road guidance information based on map information around the vehicle. Further, the voice output unit 66 may output a message (for example, a message prompting the driver to perform a steering operation or a deceleration operation) generated by driving assistance control by the driving assistance device 1 .

The communication unit 67 performs wireless communication with a communication device outside the own vehicle. The communication method by the communication unit 67 may be, for example, wireless communication by a public mobile phone network, vehicle-to-vehicle communication, road-to-vehicle communication, or satellite communication. The navigation system 6 may acquire road map information from an external device through the communication section 67 .

The controller 7 is an electronic control unit that assists driving of the own vehicle. The controller 7 includes a processor 71 and peripheral components such as a storage device 72 and the like. The processor 71 may be, for example, a CPU or MPU.
The storage device 72 may comprise any one of a semiconductor storage device, a magnetic storage device, and an optical storage device. The storage device 72 may include memories such as registers, cache memory, and ROM and RAM used as main memory.

Note that the controller 7 may be realized by a functional logic circuit set in a general-purpose semiconductor integrated circuit. For example, the controller 7 may include a programmable logic device (PLD) such as a field-programmable gate array (FPGA).

The controller 7 receives the ambient environment detected by the ambient environment sensor 2, the driver state detected by the in-vehicle sensor 3, the vehicle state detected by the vehicle sensor 4, the electroencephalogram signal of the driver detected by the electroencephalogram sensor 5, and the navigation system 6. Based on the provided navigation information, the driving operation intended by the driver is predicted.
The controller 7 calculates a target operation amount of a driving operation system (for example, a steering wheel mechanism, an accelerator mechanism, and a brake mechanism) for realizing the predicted driving operation, and drives the vehicle control actuator 8 based on the target operation amount. It controls the driving operation system of the own vehicle.
As a result, the controller 7 drives the own vehicle so that the driver feels as if he/she is operating the vehicle by himself/herself, and assists the driver's driving operation.

The vehicle control actuator 8 operates the steering wheel, the accelerator opening and the braking device of the own vehicle according to the control signal from the controller 7 to generate the vehicle behavior of the own vehicle. The vehicle control actuator 8 includes a steering actuator, an accelerator opening actuator, and a brake control actuator.
The steering actuator controls the steering direction and amount of steering of the host vehicle. The accelerator opening actuator controls the accelerator opening of the host vehicle. The brake control actuator controls the braking operation of the brake system of the host vehicle.

Next, with reference to FIG. 4, the driving support function of the controller 7 will be described. The controller 7 includes a navigation information acquisition unit 101, a surrounding environment recognition unit 102, a vehicle data acquisition unit 103, a driver state recognition unit 104, a brain activity analysis unit 105, a brain activity database 106, and a sensitivity time storage unit. 107 , a driving operation database 108 , a driving operation determination unit 110 , a driving scene database 113 , an operation timing prediction unit 120 , an automatic function device 130 , and an automatic function control unit 140 . In FIG. 4, "database" is written as "DB".

For example, the controller 7 executes a computer program stored in the storage device 72 of FIG. , brain activity analysis unit 105 , driving operation determination unit 110 , operation timing prediction unit 120 , automatic function device 130 , and automatic function control unit 140 .

The navigation information acquisition unit 101 acquires navigation information from the navigation system 6 . The navigation information acquisition unit 101 acquires road map information such as intersections, junctions, junctions, winding roads, and stop lines existing in the traveling direction of the vehicle. The navigation information acquisition unit 101 outputs the acquired navigation information to the driving operation determination unit 110 .
The navigation information acquisition unit 101 also acquires from the navigation system 6 road map information such as driving lanes, road shapes, legal speeds, road widths, and stop lines around the host vehicle. The navigation information acquisition unit 101 outputs the acquired road map information to the automatic function device 130 .

The surrounding environment recognizing unit 102 acquires from the surrounding environment sensor 2 photographed data of the surroundings of the own vehicle and distance measurement data to surrounding objects. The surrounding environment recognition unit 102 recognizes the surrounding environment of the own vehicle based on the data acquired from the surrounding environment sensor 2 .
For example, the surrounding environment recognition unit 102 recognizes the following surrounding environment.
(1) Inter-vehicle distance, relative speed, time-to-collision (TTC), time-head-way (THW), vehicle distance to other vehicles Surrounding vehicle information such as whether or not the vehicle is surrounded and the steering and acceleration/deceleration are restricted

(2) Obstacles around the vehicle (for example, obstacles that block the path ahead)
(3) Traffic signal indication (red signal) or change in signal indication (e.g., change from red to green) of the traffic signal ahead of the vehicle
(4) Lane information such as whether or not there are adjacent lanes on the left and right in which the lane can be changed.

The vehicle data acquisition unit 103 acquires vehicle data of the own vehicle such as the speed of the own vehicle, the steering angle, the accelerator opening, the amount of brake operation, and the state of the shift lever from the vehicle sensor 4 . Vehicle data acquisition unit 103 outputs vehicle data to driving operation determination unit 110 .
The driver state recognition unit 104 acquires the photographed data of the driver and the detection signal of the touch sensor from the in-vehicle sensor 3 .

For example, the driver state recognition unit 104 may recognize the driver's posture, face orientation, and line of sight direction as the driver's state based on the photographed data of the driver.
The driver state recognizing unit 104 may recognize, as the state of the driver, the state of gripping the steering wheel and the state of contact with the accelerator pedal and the brake pedal based on the detection signal of the touch sensor.
The driver state recognition unit 104 outputs information on the recognized state of the driver to the driving operation determination unit 110 .

The brain activity analysis unit 105 acquires the electroencephalogram signal of the driver from the electroencephalogram sensor 5 . The brain activity analysis unit 105 generates the feature vector P of the electroencephalogram signal (see FIG. 2), and determines whether the feature vector P is included in the region D (see FIG. 3) arranged in the feature space stored in the brain activity database 106. It is determined whether or not an exercise readiness potential is generated according to whether or not.
Brain activity analysis unit 105 outputs the detected exercise readiness potential to driving operation determination unit 110 and operation timing prediction unit 120 .

The driving operation database 108 stores brain activity data indicating the relationship between driving operations and brain activity for each driver. For example, when the brain activity analysis unit 105 detects the motor readiness potential of steering operation by the driver, the time from the detection timing to the actual steering operation (delay time β) is Ask. Then, the delay time β is cumulatively stored. An average value of a plurality of detection data can also be used as the delay time β.

Based on the data stored in the driving operation database 108, the sensitivity time storage unit 107 obtains the sensitivity time (Δt described later) when each driver operates the operation type, and stores the sensitivity time. For example, when changing lanes by operating the steering wheel to avoid a preceding vehicle, the system distinguishes between a driver who changes lanes early and a driver who changes lanes after the distance between the vehicle and the preceding vehicle becomes shorter. , classifies drivers into three skill levels: advanced drivers with high driving skills, beginner drivers with low driving skills, and intermediate drivers with intermediate skills. For example, the sensitivity time for a beginner is 800 [msec], the sensitivity time for an intermediate level is 500 [msec], and the sensitivity time for an advanced level is 200 [msec].

The skill level can be set arbitrarily by the user, for example, in addition to being set based on each driver's past driving operation data. For example, it is possible to set a beginner as a driver with less than one year of driving experience, an intermediate driver as a driver with one year or more but less than ten years of driving experience, and an advanced driver as a driver with ten years or more of driving experience. Note that the present invention is not limited to three skill levels, and four or more skill levels may be set and different sensitivity times may be set for each.

The driving operation determination unit 110 uses the navigation information acquired by the navigation information acquisition unit 101, the surrounding environment recognized by the surrounding environment recognition unit 102, the vehicle data acquired by the vehicle data acquisition unit 103, and the driver state recognition unit 104. Based on the recognized state of the driver, it is determined whether or not there is a possibility that the driver will operate the own vehicle.
The driving operation determination unit 110 predicts the operation type of the driving operation of the own vehicle by the driver based on the driving preparation potential detected when it is determined that there is a possibility of performing the driving operation.

The driving operation determination unit 110 includes an operation possibility determination unit 111 and an operation type prediction unit 112 .
The operability determination unit 111 determines the surrounding environment of the host vehicle based on the navigation information acquired by the navigation information acquisition unit 101 and the recognition result of the surrounding environment recognition unit 102 . Further, the operability determination unit 111 determines the state of the driver based on the vehicle data acquired by the vehicle data acquisition unit 103 and the recognition result of the driver state recognition unit 104 . The operation possibility determination unit 111 determines whether or not there is a possibility that the driver will operate the vehicle based on the surrounding environment of the vehicle and the state of the driver.

For example, based on the navigation information acquired by the navigation information acquisition unit 101 and the recognition result of the surrounding environment recognition unit 102, the operation possibility determination unit 111 determines the location where the driving operation of the own vehicle is predicted and the location where the driving operation is required. It determines whether or not there is a point where the driver is required to drive or if there is a situation that requires a driving operation.
For example, the operability judging unit 111 determines points in front of intersections, junctions, junctions, curves with a radius of curvature equal to or less than a predetermined radius of curvature, stop lines, red lights, and main roads. It may be judged as a point where an operation is required.

For example, the operability determination unit 111 may determine that there is a situation in which a driving operation is required when the own vehicle is positioned within a predetermined distance from a point where the driving operation is required.
For example, if the host vehicle is approaching the preceding vehicle and the inter-vehicle distance is less than a predetermined distance, or if the host vehicle and the preceding vehicle are approaching at a relative speed equal to or greater than a predetermined speed, , it may be determined that there is a situation in which a driving operation is required.
For example, the operability determination unit 111 detects a situation in which a driving operation is required when there is an obstacle blocking the course ahead or when the own vehicle is surrounded by other vehicles and the steering and acceleration/deceleration are restricted. It can be judged that there is

On the other hand, based on the vehicle data (steering angle, accelerator opening, brake operation amount) acquired by the vehicle data acquisition unit 103, the operation possibility determination unit 111 determines whether the driver has performed a driving operation during the most recent predetermined distance traveled. determine whether there was.
The operability determination unit 111 determines whether the driver is expected to drive the vehicle in a situation where a driving operation is expected to occur in the direction in which the vehicle is traveling or when a driving operation is required. If there is an operation, it is determined that there is a possibility that the driver will operate the own vehicle.

Further, the operability determination unit 111 determines whether or not the driver is touching a driving operator such as a steering wheel, an accelerator pedal, a brake pedal, etc., and is in a state in which it is possible to operate these driving operators. For example, the operability determination unit 111 determines whether or not the driver is gripping the steering wheel, and whether or not the driver is putting his/her foot on the accelerator pedal or the brake pedal.
The operability determination unit 111 determines a state in which the driver can operate the driving operation element in a situation where a point where the driving operation of the own vehicle is predicted exists in the traveling direction of the own vehicle, or in a situation where the driving operation is required. , it is determined that there is a possibility that the driver will perform the driving operation of the own vehicle.

For example, the operability determination unit 111 determines that the driver's driving consciousness is lowered when the driver is looking away or taking his/her hands off the steering wheel.
The operation possibility determination unit 111 determines whether the driver's driving awareness is lowered even in a situation where a point where a driving operation of the own vehicle is predicted exists in the traveling direction of the own vehicle, or even in a situation where a driving operation is required. If so, it is determined that there is no possibility that the driver will operate the own vehicle.

In addition, the operation possibility determination unit 111 uses the navigation information acquired by the navigation information acquisition unit 101 and the recognition result of the surrounding environment recognition unit 102 as the driving scene of the point in the direction in which the vehicle is expected to be operated. judge based on
For example, the operability determination unit 111 determines, as driving scenes, road shapes at points in the traveling direction of the own vehicle, intersections, junctions/points, stop lines, the presence or absence of red lights, obstacles, and other vehicles. .
If the driving scene in which the driving operation is predicted is similar to the driving scene stored in the driving scene database 113 in which the driving steering occurred in the past, the operation possibility determination unit 111 determines whether the driver will perform the driving operation of the own vehicle. It is determined that there is a possibility of doing

When the operation possibility determination unit 111 determines that there is a possibility that the driver will perform the driving operation of the host vehicle, the operation possibility determining unit 111 determines a period during which the driver may perform the driving operation.
Referring to FIG. 5A, for example, the operability determination unit 111 predicts a time t1, which is a time period ΔT after the current time t0, as the time when the vehicle will reach a point where a driving operation is predicted or a point where a driving operation is required. The period ΔT is calculated based on the distance to the point where the driving operation is expected or the point where the driving operation is required, and the vehicle speed of the own vehicle.
The operation possibility determination unit 111 estimates a period starting at time t2, which is a predetermined time before time t1, and ending at time t3, a predetermined time after time t1, as a period Pd during which the driver may perform a driving operation. .

Please refer to FIG. The operation type prediction unit 112 predicts the operation type of the driver's driving operation of the vehicle based on the exercise preparation potential detected during the period Pd during which the driver may perform the driving operation.
See FIG. 5B. Now, it is assumed that the driving preparation potential is detected from the waveforms W1 and W2 of the electroencephalogram signal. The operation type prediction unit 112 predicts the operation type based on the driving preparation potential detected in the waveform W2 within the period Pd during which the driver may perform the driving operation. Not used for prediction.

That is, when the operation possibility determination unit 111 does not determine that the driver is likely to perform the driving operation of the own vehicle, the operation type prediction unit 112 does not predict the operation type of the driving operation of the own vehicle by the driver.
As a result, it is possible to prevent erroneous prediction of the operation type based on the exercise readiness potential of exercise other than the driving operation (for example, moving the hand to take water from a plastic bottle).

Please refer to FIG. For example, the operation type prediction unit 112 may predict which of the right hand, the left hand, or the foot the driver will move, depending on which part of the brain the detected motor readiness potential is generated. The operation type prediction unit 112 predicts which type of operation such as right steering operation, left steering operation, acceleration operation, and deceleration operation will be performed, depending on whether the driver moves the right hand, the left hand, or the foot. you can

Further, for example, the operation type prediction unit 112 may predict the operation type of the driving operation by the driver using a discriminator that performs machine learning of the driving operation performed by the driver when the exercise preparation potential is generated.
Further, for example, the operation type prediction unit 112, in addition to the exercise preparation potential, the navigation information acquired by the navigation information acquisition unit 101 (for example, the driving route, the road shape, the position of the merging lane), the recognition result of the surrounding environment recognition unit 102 (e.g., relative position and relative distance of surrounding objects) and the recognition results of the driver state recognition unit 104 (e.g., driver's posture, face orientation, line-of-sight direction). You can predict.
The operation type prediction section 112 outputs the predicted operation type to the automatic function control section 140 .

The automatic function device 130 has a function of assisting the driving operation of the own vehicle based on the road map information acquired by the navigation information acquisition unit 101 and the surrounding environment information recognized by the surrounding environment recognition unit 102 . The automatic function device 130 may be, for example, an ACC (Adaptive Cruise Control) device. Since the ACC device is a well-known device, detailed description thereof will be omitted. Also, the automatic function device may be an automatic driving device. Since the automatic driving device is also a well-known device, the explanation is omitted.

The operation timing prediction unit 120 predicts the start timing of the driver's driving operation based on the motor readiness potential detected by the brain activity analysis unit 105 .
See FIG. 5B. The timing at which the driver actually starts the driving operation is time t4, which is delayed by the delay time β from the time at which the exercise preparation potential is detected.
The operation timing prediction unit 120 reads the delay time β stored for each driver in the driving operation database 108, and determines the timing at which the driver performs the driving operation (this will be referred to as "target person operation timing T2").

As shown in the timing charts of FIGS. 6A and 6B, the time when the delay time β has passed from the timing T0 at which the exercise preparation potential is detected is the subject operation timing T2.
A time T1 indicates an operation timing (hereinafter referred to as "automatic operation timing T1") when the automatic function device 130 performs an automatic operation regardless of the driver's intention.

The operation timing prediction unit 120 reads the sensitivity time of the target driver (which is assumed to be “Δt”) from the sensitivity time storage unit 107 . As described above, the skill levels of beginner, intermediate, and advanced drivers are set based on the driving skill of each driver, and the sensitivity time Δt is set according to each skill level.
In this process, the sensitivity time Δt is acquired based on the driving skill of the target driver. For example, if the driver's driving skill is intermediate, the sensitivity time Δt is set to 500 [msec].

The operation timing prediction unit 120 calculates the difference “T2−T1” (this is referred to as the “time difference α”) between the subject operation timing T2 and the automatic operation timing T1, and calculates the time difference α and the driver’s sensitivity time Δt. compare.
If α>Δt, the operation timing prediction unit 120 sets the operation timing Tr of the automatic function device 130 to "T2-Δt". That is, as shown in FIG. 6A, the timing T3a that is the sensitivity time Δt before the target person's operation timing T2 is set as the actuation timing Tr for the automatic operation.

When α≦Δt, the operation timing prediction unit 120 sets the operation timing Tr of the automatic function device 130 to the automatic operation timing T1. That is, as shown in FIG. 6B, the timing T3b, which is the sensitivity time Δt before the subject operation timing T2, is the timing T3b earlier than the automatic operation timing T1. Set as timing Tr.

Please refer to FIG. The automatic function control unit 140 outputs a driving assistance command to the automatic function device 130 based on the operation type of the driver predicted by the driving operation determination unit 110 and the actuation timing Tr set by the operation timing prediction unit 120. .
Automatic functional device 130 controls automatic functions in response to driving assistance commands from automatic functional device 130 . Specifically, a target operation amount of a driving operation system (for example, a steering wheel mechanism, an accelerator mechanism, and a brake mechanism) that realizes a driving operation of the predicted operation type is calculated.
Then, the vehicle control actuator 8 is driven based on the target operation amount to control the driving operation system of the own vehicle.

(motion)
The operation of the driving assistance device of the embodiment will be described with reference to FIG.
In step S<b>1 , the navigation information acquisition unit 101 acquires navigation information from the navigation system 6 .
In step S<b>2 , the surrounding environment recognition unit 102 recognizes the surrounding environment of the vehicle based on the data acquired from the surrounding environment sensor 2 .

In step S<b>3 , the vehicle data acquisition unit 103 acquires vehicle data of the host vehicle, such as the speed of the host vehicle, the steering angle, the accelerator opening, the amount of brake operation, and the state of the shift lever, from the vehicle sensor 4 .
In step S4, the driver state recognition unit 104 recognizes the driver's posture, face orientation, and line of sight direction as the driver's state based on the photographed data of the driver. The driver state recognition unit 104 recognizes the state of contact with the driving operation element as the state of the driver based on the detection signal of the touch sensor.

In step S5, the electroencephalogram sensor 5 measures the electroencephalogram of the driver. The brain activity analysis unit 105 detects the motor readiness potential by analyzing the electroencephalogram signal of the driver.
In step S6, the operability determination unit 111 obtains the navigation information acquired by the navigation information acquisition unit 101, the surrounding environment recognized by the surrounding environment recognition unit 102, the vehicle data acquired by the vehicle data acquisition unit 103, and the driver state. Based on the state of the driver recognized by the recognition unit 104, it is determined whether or not there is a possibility that the driver will drive the host vehicle.

If there is a possibility of performing a driving operation (step S6: Y), the process proceeds to step S7. If there is no possibility of performing a driving operation (step S6: N), the process ends without performing driving assistance.
In step S7, the operation possibility determination unit 111 determines a period Pd during which the driver may perform a driving operation.

In step S8, the operation type prediction unit 112 determines whether or not the exercise readiness potential is detected within the period Pd during which the driver may perform the driving operation. The process proceeds to step S when the exercise preparation potential is detected within the period Pd (step S8: Y). If no exercise readiness potential is detected within the period Pd (step S8: N), the process ends without performing driving assistance.

In step S9, the operation type prediction unit 112 predicts the operation type of the driving operation of the host vehicle by the driver based on the exercise preparation potential.
In step S10, the operation timing prediction unit 120 predicts the start timing T2 of the driver's driving operation based on the detection timing of the exercise preparation potential. The operation timing prediction unit 120 sets the operation timing Tr of the automatic function device 130 based on the predicted driving operation start timing T2.

In step S11, the automatic function control unit 140 issues a driving assistance command to the automatic function device 130 based on the operation type of the driver predicted by the driving operation determination unit 110 and the actuation timing Tr set by the operation timing prediction unit 120. to output
The automatic functional device 130 performs driving assistance by controlling automatic functions according to the driving assistance command from the automatic functional device 130 .

(Effect of Embodiment)
(1) The operation possibility determination unit 111 determines whether there is a possibility that the driver will operate the vehicle based on the surrounding environment of the vehicle and the state of the driver. The operation type prediction unit 112 predicts the operation type of the driving operation of the host vehicle by the driver based on the biological information of the driver. The automatic function control unit 140 and the automatic function device 130 execute driving assistance for the host vehicle based on the type of operation predicted when it is determined that the driver may perform a driving operation.

As a result, it is possible to correctly predict the possibility of the driver's driving operation based on the surrounding environment and the driver's condition, so that it is possible to prevent erroneous driving assistance from being performed based on biological information.

(2) The operation possibility determination unit 111 determines a period Pd for predicting the operation type based on the biometric information based on the surrounding environment.
This makes it possible to determine the period Pd during which there is a possibility of the driver's driving operation based on the surrounding environment, and prevent the operation type from being erroneously predicted based on the biometric information occurring during a period other than the period Pd. For example, it is possible to prevent the operation type from being erroneously predicted based on biological information generated during exercise unrelated to the driving operation (such as taking water from a plastic bottle).

(3) The operation possibility determination unit 111, when the point where the driving operation of the own vehicle is predicted is in the traveling direction of the own vehicle, and when the driving operation by the driver is detected during the most recent predetermined distance traveling, It is determined that there is a possibility that the driver will perform a driving operation.
As a result, the possibility of a driving operation can be determined using both the surrounding environment and the driver's condition, so that the execution of the driving operation by the driver can be predicted correctly, and incorrect driving assistance is performed based on biological information. can prevent

(4) The operation possibility determination unit 111 determines that the driver will perform the driving operation when the point where the driving operation of the own vehicle is predicted is in the traveling direction of the own vehicle and the driver's driving awareness is low. Decide that it is not possible.
As a result, in a state in which the driver's driving awareness is lowered, if biometric information is detected, for example, when the driver moves his or her hand to take water from a plastic bottle, driving assistance may be incorrectly performed based on this biometric information. You can prevent it from running.

(5) The operation possibility determination unit 111 detects that the driver is driving when the driving scene in the traveling direction of the own vehicle in which the driving operation of the own vehicle is predicted is similar to the driving scene in which the driving operation has occurred in the past. It is judged that there is a possibility of performing the operation.
By learning the habits of each individual driver in this way, it is possible to correctly predict the execution of driving operations for each driver.

(6) The operation possibility determination unit 111 determines that there is a possibility that the driver will perform the driving operation when the own vehicle is located within a predetermined distance range from the point where the driving operation of the own vehicle is required.
As a result, the driving operation is predicted based on the surrounding environment in each target scene, and it is predicted that the driver is ready to perform the driving operation. It is possible to prevent wrong driving assistance from being executed based on

(7) The operation possibility determination unit 111 determines that there is a possibility that the driver will perform a driving operation when a driving operation of the own vehicle is required and the driver is in a state where the driver can operate the driving operation elements. to decide.
As a result, the driving operation is predicted based on the surrounding environment in each target scene, and it is predicted that the driver is ready to perform the driving operation. It is possible to prevent wrong driving assistance from being executed based on

(8) When the inter-vehicle distance between the own vehicle and the preceding vehicle is equal to or less than a predetermined distance, or when the own vehicle and the preceding vehicle are approaching each other at a relative speed equal to or greater than a predetermined speed, the operability determination unit 111 In addition, when the driver puts his or her foot on either the accelerator or the brake, it is determined that there is a possibility that the driver will perform a driving operation.
As a result, the driving operation is predicted based on the surrounding environment in each target scene, and it is predicted that the driver is ready to perform the driving operation. It is possible to prevent wrong driving assistance from being executed based on

(9) The electroencephalogram sensor 5 and the brain activity analysis unit 105 detect brain activity of the driver and obtain biological information.
By using brain activity analysis in this way, a driving operation can be quickly detected before the driver performs the operation.

DESCRIPTION OF SYMBOLS 1... Driving assistance device, 2... Surrounding environment sensor, 3... In-vehicle sensor, 4... Vehicle sensor, 5... Electroencephalogram sensor, 6... Navigation system, 7... Controller, 8... Vehicle control actuator, 61... Navigation controller, 62... Positioning Apparatus 63... Map database 64... Display unit 65... Operation unit 66... Voice output unit 67... Communication unit 71... Processor 72... Storage device 101... Navigation information acquisition unit 102... Surrounding environment recognition unit , 103 Vehicle data acquisition unit 104 Driver state recognition unit 105 Brain activity analysis unit 106 Brain activity database 107 Sensitivity time storage unit 108 Driving operation database 110 Driving operation determination unit 111 Operation possibility determination unit 112 Operation type prediction unit 113 Driving scene database 120 Operation timing prediction unit 130 Automatic function device 140 Automatic function control unit

Claims (10)

As the state of the driver, the posture of the driver, the direction of the driver's face, the direction of the driver's line of sight, the state of gripping the steering wheel, whether or not the foot is on the accelerator pedal, and the foot on the brake pedal. Detect at least one of whether or not it is loaded,
determining whether or not there is a possibility that the driver will perform a driving operation of the own vehicle based on the surrounding environment of the own vehicle and the state of the driver;
Predicting the operation type of the driving operation of the own vehicle by the driver based on the driver's biological information;
Execution of driving assistance of the own vehicle based on the operation type predicted when it is determined that the driver is likely to perform the driving operation;
A driving support method characterized by: 2. The driving support method according to claim 1, wherein a period for predicting said operation type based on said biological information is determined based on said surrounding environment. When the point where the driving operation of the own vehicle is predicted is in the traveling direction of the own vehicle, and the driving operation by the driver is detected while the own vehicle is running for the most recent predetermined distance, the driver performs the driving operation. 3. The driving support method according to claim 1, wherein it is determined that there is a possibility. Determining whether or not there is a possibility that the driver will perform a driving operation of the own vehicle based on the surrounding environment of the own vehicle and the state of the driver,
Predicting the operation type of the driving operation of the own vehicle by the driver based on the driver's biological information;
executing driving assistance for the own vehicle based on the operation type predicted when it is determined that the driver may perform a driving operation;
When the point where the driving operation of the own vehicle is predicted is in the traveling direction of the own vehicle and the driving consciousness of the driver is lowered, it is determined that there is no possibility of the driver performing the driving operation. A driving support method characterized by: Determining whether or not there is a possibility that the driver will perform a driving operation of the own vehicle based on the surrounding environment of the own vehicle and the state of the driver,
Predicting the operation type of the driving operation of the own vehicle by the driver based on the driver's biological information;
executing driving assistance for the own vehicle based on the operation type predicted when it is determined that the driver may perform a driving operation;
When the driving scene in the traveling direction of the own vehicle in which the driving operation of the own vehicle is predicted is similar to the driving scene in which the driving operation has occurred in the past, there is a possibility that the driver will perform the driving operation. A driving support method characterized by determining. 2. It is determined that there is a possibility that the driver will perform the driving operation when the own vehicle is located within a predetermined distance range from a point where the driving operation of the own vehicle is required. 2. The driving support method according to 2. Determining whether or not there is a possibility that the driver will perform a driving operation of the own vehicle based on the surrounding environment of the own vehicle and the state of the driver,
Predicting the operation type of the driving operation of the own vehicle by the driver based on the driver's biological information;
executing driving assistance for the own vehicle based on the operation type predicted when it is determined that the driver may perform a driving operation;
determining whether or not the driver is in a state in which the driver can operate the driving operator;
driving, wherein it is determined that there is a possibility that the driver will perform a driving operation when the driver is in a state in which the driving operation of the own vehicle is required and the driver is in a state where the driver can operate the driving operation device. how to help. Determining whether or not there is a possibility that the driver will perform a driving operation of the own vehicle based on the surrounding environment of the own vehicle and the state of the driver,
Predicting the operation type of the driving operation of the own vehicle by the driver based on the driver's biological information;
executing driving assistance for the own vehicle based on the operation type predicted when it is determined that the driver may perform a driving operation;
The distance between the subject vehicle and the preceding vehicle is less than or equal to a predetermined distance, or the subject vehicle and the preceding vehicle are approaching each other at a relative speed greater than or equal to a predetermined speed, and the driver presses the accelerator and the brake. and determining that there is a possibility that the driver will perform a driving operation when the driver places his or her foot on any of the above. The driving support method according to any one of claims 1 to 8, wherein the biological information is obtained by detecting brain activity of the driver. a first sensor that detects the surrounding environment of the own vehicle;
As the state of the driver, the posture of the driver, the direction of the driver's face, the direction of the driver's line of sight, the state of gripping the steering wheel, whether or not the foot is on the accelerator pedal, and the foot on the brake pedal. a second sensor that detects at least one of whether or not the
a third sensor that acquires the driver's biological information;
Based on the surrounding environment detected by the first sensor and the state of the driver detected by the second sensor, it is determined whether or not there is a possibility that the driver will perform a driving operation of the own vehicle. Predicting the operation type of the driving operation of the own vehicle by the driver based on the biological information acquired by the sensor, and predicting the operation type when it is determined that there is a possibility that the driver will perform the driving operation. a controller that executes driving assistance for the host vehicle based on
A driving support device comprising:

Images