EP3264211A1 - Fahrassistenzvorrichtung und fahrassistenzverfahren - Google Patents

Fahrassistenzvorrichtung und fahrassistenzverfahren Download PDF

Info

Publication number
EP3264211A1
EP3264211A1 EP17173443.7A EP17173443A EP3264211A1 EP 3264211 A1 EP3264211 A1 EP 3264211A1 EP 17173443 A EP17173443 A EP 17173443A EP 3264211 A1 EP3264211 A1 EP 3264211A1
Authority
EP
European Patent Office
Prior art keywords
driver
vehicle
take
over request
driving
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP17173443.7A
Other languages
English (en)
French (fr)
Other versions
EP3264211B1 (de
Inventor
Yoshihiro Kojima
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to EP20212082.0A priority Critical patent/EP3809227B1/de
Publication of EP3264211A1 publication Critical patent/EP3264211A1/de
Application granted granted Critical
Publication of EP3264211B1 publication Critical patent/EP3264211B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K28/00—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
    • B60K28/02—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
    • B60K28/06—Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver responsive to incapacity of driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/80—Arrangements for controlling instruments
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/182—Selecting between different operative modes, e.g. comfort and performance modes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W40/09—Driving style or behaviour
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/082—Selecting or switching between different modes of propelling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0053—Handover processes from vehicle to occupant
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0057—Estimation of the time available or required for the handover
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0059—Estimation of the risk associated with autonomous or manual driving, e.g. situation too complex, sensor failure or driver incapacity
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0061—Aborting handover process
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0055—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots with safety arrangements
    • G05D1/0061—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots with safety arrangements for transition from automatic pilot to manual pilot and vice versa
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/175—Autonomous driving
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/186—Displaying information according to relevancy
    • B60K2360/1868—Displaying information according to relevancy according to driving situations
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/55—Remote control arrangements
    • B60K2360/56—Remote control arrangements using mobile devices
    • B60K2360/566—Mobile devices displaying vehicle information
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0872—Driver physiology
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0062—Adapting control system settings
    • B60W2050/007—Switching between manual and automatic parameter input, and vice versa
    • B60W2050/0072—Controller asks driver to take over
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/54—Audio sensitive means, e.g. ultrasound
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/21—Voice
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/22—Psychological state; Stress level or workload
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/221—Physiology, e.g. weight, heartbeat, health or special needs
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/223—Posture, e.g. hand, foot, or seat position, turned or inclined
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/225—Direction of gaze
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/26—Incapacity
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/40—Dynamic objects, e.g. animals, windblown objects
    • B60W2554/406—Traffic density
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2555/00—Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/45—External transmission of data to or from the vehicle
    • B60W2556/50—External transmission of data to or from the vehicle of positioning data, e.g. GPS [Global Positioning System] data

Definitions

  • the present disclosure relates to a driving assistance apparatus and a driving assistance method that assist a transition from an autonomous driving mode in which the vehicle is driven under autonomous control to a manual driving mode in which the vehicle is driven by a driver.
  • An autonomous driving system enables driving controls of a vehicle (such as accelerating, steering, and braking) to be performed automatically.
  • autonomous driving systems widely used are autonomy levels which are defined from the viewpoint of the division of roles between the driver and the system. For example, according to the definition by the National Highway Traffic and Safety Administration in the United States Department of Transportation, there are five autonomy levels from Level 0 being manual driving to Level 4 being fully autonomous driving.
  • Level 2 is an autonomy level at which the system controls more than one of the accelerating, steering, and braking, but the driver needs to constantly monitor the driving situation. It is expected that Level-3 autonomous driving systems will be commercialized in the near future.
  • Level 3 is an autonomy level at which the driver does not need to monitor the driving situation while the system performs all of the accelerating, steering, and braking, but needs to be ready to take over the driving when the system reaches its functional limit.
  • the driver of a vehicle having a Level-3 autonomous driving system does not need to perform any driving task such as driving the vehicle or monitoring the driving situation, as long as the system is in stable operation. Nevertheless, the driver still needs to take over the driving task from the system when the system reaches its functional limit and switches the driving mode from the autonomous driving mode to the manual driving mode. However, it is likely that, when the driving mode is switched from the autonomous driving mode to the manual driving mode, the driver who has been disengaged from a driving task cannot immediately gain a sufficient situational awareness of the surrounding environment.
  • a method for achieving an appropriate transition from autonomous driving by the system to manual driving by the driver is an important issue to be overcome in order for Level-3 autonomous driving systems to be commercialized.
  • Various methods have been proposed to this end.
  • Patent Document 1 discloses a vehicle driving assistance apparatus that predicts road conditions ahead of the host vehicle, determines based on the prediction whether autonomous driving will be continuable hereafter, and when determining that the autonomous driving will not be continuable, gives the driver an advance notice that the autonomous driving mode of the system is going to be cancelled.
  • the vehicle driving assistance apparatus of Patent Document 1 gives the take-over request when the distance from the host vehicle to the predicted position at which the autonomous driving will be cancelled falls to or below a predetermined value.
  • the driver can be prepared for manual driving beforehand, such as gaining a situational awareness of the surrounding environment, and make a safe transition to manual driving.
  • One non-limiting and exemplary embodiment provides a driving assistance apparatus and a driving assistance method that help the driver return to manual driving in a transition from an autonomous driving mode in which the vehicle is driven under autonomous control to a manual driving mode in which the vehicle is driven by the driver.
  • the techniques disclosed here feature a driving assistance apparatus that assists a transition from an autonomous driving mode in which a vehicle is driven under autonomous control to a manual driving mode in which the vehicle is driven by a driver.
  • the driving assistance apparatus comprises one or more memories; and circuitry that, in operation, performs operations including detecting an activity by the driver, detecting a plurality of conditions of the driver, and determining a take-over request method which is a method of presenting, in the vehicle, a take-over request that informs the driver that the autonomous driving mode is going to be cancelled, the determining being based on at least the detected activity by the driver and the plurality of detected conditions.
  • the driving assistance apparatus and the driving assistance method according to the above aspect sets the take-over request method according to detection results of the driver's activity and conditions. This helps the driver return to manual driving in a transition from the autonomous driving mode in which the vehicle is driven under autonomous control to the manual driving mode in which the vehicle is driven by the driver.
  • the driver of a vehicle with a Level-3 autonomous driving system does not need to perform any driving task such as driving the vehicle or monitoring the driving situation, as long as the system is in stable operation. Freed from any driving tasks, the driver may engage in an activity unrelated to driving in the autonomous driving mode, such as watching TV or reading text messages on a smartphone.
  • a "driver" is a person assumed to perform driving tasks during the manual driving mode. The driver may also be a person sitting in the driver's seat or a person preregistered at the system as a driver.
  • the driver When the system reaches its functional limit, the driver needs to take over the driving tasks from the system. If the driver has been engaging in the driving-unrelated activity for a long time, it is possible that there is a notable decrease in the driver's consciousness of driving, such as situational awareness of the surrounding environment and perception of the conditions of the host vehicle. When the driving mode switches to the manual driving mode with the driver in such a condition, the driver may be slow to respond to, or even miss, objects to be watched out for while driving the vehicle, which in the worst scenario could lead to a traffic accident.
  • Level-3 autonomous driving systems requires a method for achieving a safe and smooth transition for a driver disengaged from a driving task to return to a state ready for manual driving.
  • Patent Document 1 discloses a method including: predicting road conditions ahead of the host vehicle; determining based on the prediction whether the system will be able to continue autonomous driving; and when determining that the autonomous driving will not be continuable, giving the driver a notice that the autonomous driving mode of the system is going to be cancelled. According to this method, the driver is given a notice of a transition from autonomous driving to manual driving, and therefore can get prepared for manual driving.
  • the method of Patent Document 1 starts presenting the take-over request once the distance from the host vehicle to a predicted position where the autonomous driving mode is going to cancelled falls to or below a predetermined value.
  • the take-over request may not give enough time for the driver to be ready for manual driving.
  • the driver does not notice an autonomous driving take-over request if engaging in an activity such as watching TV or playing a video game with a smartphone when the take-over request is presented.
  • the inventors of the present disclosure have therefore found that take-over requests need to be presented to the driver based on the driver's activity and consciousness level towards driving in the autonomous driving mode.
  • Patent Document 2 discloses a method that includes: estimating a driver's activity in an autonomous driving mode; if determining that the driver is engaging in an activity unrelated to driving, selecting a certain one of information presentation devices based on the activity; and presenting information to prompt the driver to stop the driving-unrelated activity.
  • the method of Patent Document 2 estimates takeover time necessary for the driver to bring themselves from the driving-unrelated activity to a state ready for manual driving, and arrival time after which the system cannot continue autonomous driving, and when the takeover time is longer than the arrival time, immediately presents information to prompt the driver to stop the activity.
  • This method can change, based on the driver's activity in the autonomous driving mode, what device to use for the presentation of the information which prompts the driver to stop the driving-unrelated activity. For example, if the driver is engaging in an activity such as watching TV or playing a video game with a smartphone, the above method can present, on the screen of the smartphone, information prompting the driver to stop the activity.
  • information prompting the driver to stop an activity can be regarded as a type of information by which a take-over request is presented.
  • the timing to start presenting the information prompting the driver to stop their driving-unrelated activity is determined based on the takeover time necessary for the driver to return from the driving-unrelated activity to manual driving. This takeover time is determined for each type of activity that the driver may engage in.
  • the driver when the driver is, for example, playing a video game using a smartphone in the autonomous driving mode, the driver may have been absorbed in the game for a long time, or may have been playing the game intermittently while occasionally monitoring the surrounding environment of the host vehicle.
  • the time required for the driver to fully return from the game activity to manual driving should be different between these two situations.
  • the actual takeover time is likely to be longer for the driver who has been absorbed in the game.
  • the conventional method does not take this into consideration and cannot ensure whether the driver can truly return to a state ready for manual driving when the autonomous driving mode is cancelled. This is the problem that the inventors of the present disclosure have noted in the conventional method. After earnest study, the inventors of the present disclosure have come to the view that the takeover time necessary to return to manual driving needs to be set in consideration of not only the type of activity that the driver is engaging in, but also physiological and psychological conditions of the driver engaging in that activity.
  • a driving assistance apparatus changes the method of presenting an autonomous driving take-over request to the driver, based on the driver's activity in the autonomous driving mode and the conditions of the driver engaging in that activity. This helps the driver return to manual driving safely and smoothly.
  • a driving assistance apparatus according to Embodiment 1 is described below.
  • Fig. 1 is a block diagram illustrating the configuration of a driving assistance apparatus 1 according to this embodiment.
  • the driving assistance apparatus 1 includes a first sensor group 2, a vehicle-mounted devices interface 20 ("interface” will be abbreviated as "IF” hereinbelow), an activity detection section 3, a driver conditions detection section 4, and a take-over request method determination unit 5.
  • the driving assistance apparatus 1 further includes a presentation controller 6, a second sensor group 7, a driving mode selection unit 8, and a vehicle controller 80.
  • the driving assistance apparatus 1 is installed in a target vehicle for autonomous driving, and assists a transition from an autonomous driving mode in which the vehicle is driven under autonomous control to a manual driving mode in which the vehicle is driven by the driver.
  • the first sensor group 2 is a group of sensor devices that sense various types of information used for monitoring the driver of the vehicle.
  • the first sensor group 2 includes an in-vehicle camera 21, a microphone 22, a body movement sensor 23, a blood pressure sensor 24, and a heartbeat sensor 25.
  • the sensors 21 to 35 of the first sensor group 2 and the vehicle-mounted devices IF 20 form an example of a sensor part in this embodiment.
  • the in-vehicle camera 21 is an example of a first camera that captures an image of an inside of the vehicle.
  • the in-vehicle camera 21 generates image data that indicates a captured image.
  • this embodiment assumes that the driver is in the driver's seat.
  • the in-vehicle camera 21 is placed inside the vehicle to be able to capture an image near the driver's seat.
  • the microphone 22 is placed inside the vehicle and picks up ambient sound.
  • the microphone 22 generates data (audio data) that indicates picked-up sound.
  • the body movement sensor 23 is formed by, for example, a load sensor placed inside the backrest or seat base of the driver's seat.
  • the body movement sensor 23 senses a change in the posture of a person in the driver's seat, and generates a sensor signal indicative of the sensed result.
  • the body movement sensor 23 may be formed by other sensors such as an acceleration sensor or an angular velocity sensor.
  • the blood pressure sensor 24 measures the blood pressure of the driver and generates a sensor signal indicative of the measurement result.
  • the blood pressure sensor 24 is configured as a wearable device and attached beforehand to the person responsible for driving in the manual driving mode.
  • the heartbeat sensor 25 measures the heartbeats of the driver, and generates sensor signals indicative of the measurement results.
  • the heartbeat sensor 25 may be a contact sensor attached to a part of the body, such as an ear lobe, or may be a contactless sensor, such as a camera, that extracts a change in the color of the face corresponding to a pulse wave.
  • the vehicle-mounted devices IF 20 is an interface circuit (module) that exchanges signals with various devices in the vehicle using wireless or wired communications.
  • the vehicle-mounted devices IF 20 performs communications in compliance with a predetermined communications standard, such as USB, HDMI (registered trademark), IEEE 1395, Wi-Fi, or Bluetooth (registered trademark).
  • the vehicle-mounted devices IF 20 communicates with an automotive navigation system or a TV system mounted in the vehicle.
  • the vehicle-mounted devices IF 20 also communicates information with, for example, a smartphone 65 owned by the driver.
  • the vehicle-mounted devices IF 20 may communicate with the smartphone 65 wirelessly with an application for communicating with the vehicle-mounted devices IF 20 installed in the smartphone 65, or have a wired connection with the smartphone 65.
  • the smartphone 65 instead of or in addition to the smartphone 65, other types of portable information terminals may be used, such as a handheld game console, a tablet terminal, and a notebook PC.
  • the activity detection section 3 detects the driver's activity based on information from the sensors 21 to 25 of the first sensor group 2 and the vehicle-mounted devices IF 20.
  • the activity detection section 3 may detect the driver's activity periodically. A description of how the activity detection section 3 detects an activity will be given later.
  • the driver conditions detection section 4 detects conditions of the driver related to the driver's consciousness towards driving based on information from the sensors 21 to 25 of the first sensor group 2 and the vehicle-mounted devices IF 20.
  • the driver conditions detection section 4 may detect the conditions periodically.
  • the conditions of the driver detected by the driver conditions detection section 4 may be collectively referred to as "driver conditions".
  • the driver conditions include physiological conditions and psychological conditions.
  • the physiological conditions are based on humans' physiological states, such as drowsiness and fatigue, and the psychological conditions are based on humans' psychological states, such as nervousness and concentration.
  • the driver conditions detection section 4 includes, as an example, a wakefulness level detection part 41, a fatigue level detection part 42, a nervousness level detection part 43, and a concentration level detection part 44, which respectively detect the levels of wakefulness, fatigue, nervousness, and concentration of the driver.
  • the levels of wakefulness and fatigue are each an example of a measure of a physiological condition of the driver, and the levels of nervousness and concentration are each an example of a measure of a psychological condition of the driver. A description of how the driver conditions detection section 4 detects the above measures will be given later.
  • the activity detection section 3 and the driver conditions detection section 4 are implemented to form a detection unit 10 together.
  • the detection unit 10 includes a CPU that may implement predetermined functions in cooperation with software, and thereby implements the functions of the activity detection section 3 and the driver conditions detection section 4.
  • the detection unit 10 also includes an internal memory configured with a ROM, a RAM, and the like, and stores, at any time, the detection results obtained by the activity detection section 3 and the driver conditions detection section 4 in the internal memory.
  • the take-over request method determination unit 5 includes a take-over request method selection section 50 and a storage 55.
  • the take-over request method selection section 50 includes a CPU that may implement predetermined functions in cooperation with software, and thereby implements the functions of a take-over request start timing calculator 51, a take-over request presentation device selector 52, and a take-over request presentation content selector 53 (which will be described later). Using these functions, the take-over request method selection section 50 selects a method of presenting, in the vehicle, a take-over request which informs the driver that the autonomous driving mode is going to be cancelled.
  • the storage 55 is a storage medium in which to store programs and data necessary for implementing the functions of the take-over request method determination unit 5, and is configured with a ROM, a RAM, and the like.
  • the storage 55 retains a take-over request methods database D1.
  • the take-over request methods database D1 is a database in which driver's activities, driver's conditions, and methods of presenting a take-over request are managed in association with one another. Details of the functions of the take-over request method determination unit 5 will be described later.
  • the presentation controller 6 includes a CPU that may implement predetermined functions in cooperation with software, and controls presentation devices 60 capable of presenting information inside the vehicle in their respective ways.
  • the presentation devices 60 include a display 61 of any vehicle-mounted device such as the navigation system, a speaker 62, a driver's seat 63, and the smartphone 65.
  • the presentation controller 6 controls the actuation of, for example, a vibration actuator embedded in the driver's seat 63 or mounted on the seatbelt.
  • the presentation controller 6 controls the smartphone 65 as one of the display presentation devices 60.
  • the presentation controller 6 controls what image to display on the screen of the smartphone 65 or the like by, for example, communicating with the smartphone 65 via the vehicle-mounted devices IF 20.
  • the second sensor group 7 is a group of sensor devices that sense information used to monitor the situation outside the vehicle.
  • the second sensor group 7 includes a camera 71, a radar 72, a GPS 73, and/or the like (see Fig. 9 ).
  • the camera 71 of the second sensor group 7 is placed at a position such as on the exterior surface of the vehicle and is an example of a second camera that captures an image of an environment outside of the vehicle (e.g., surrounding vehicles).
  • the radar 72 may measure the position of and the distance to a vehicle or an obstacle existing near the host vehicle.
  • the GPS 73 receives GPS information indicative of the position of the host vehicle from GPS satellites.
  • the driving mode selection unit 8 includes a CPU that may implement predetermined functions in cooperation with software, and includes a driving information acquisition part 81, a road condition prediction part 82, an autonomous driving continuance determination part 83, an autonomous driving cancellation determination part 84, and an autonomous/manual driving switch part 85. Based on the information obtained by the second sensor group 7, the driving mode selection unit 8 selects either the autonomous driving mode in which the vehicle is driven under autonomous control or the manual driving mode in which the vehicle is driven by the driver.
  • the vehicle controller 80 is configured with a CPU that may implement predetermined functions in cooperation with software, and controls the driving of the vehicle based on the driving mode currently selected. For example, in the autonomous driving mode, the vehicle controller 80 may control an electronic control unit (ECU) and actuators for acceleration, steering, and braking of the vehicle, using the information obtained by the second sensor group 7.
  • ECU electronice control unit
  • each element of the detection unit 10, the take-over request method determination unit 5, the presentation controller 6, the driving mode selection unit 8, the vehicle controller 80, and the like may be configured by dedicated hardware, or may be implemented by execution of a software program appropriate for the element.
  • Each element may be implemented when a program executer, such as a CPU or a processor, reads and executes a software program recorded in a recording medium, such as a hard disk or a semiconductor memory.
  • LSI large scale integration
  • a systematic LSI is an ultra-multifunctional LSI circuit manufactured by integration of multiple components on one chip.
  • a systematic LSI circuit is a computer system including a microprocessor, a ROM, a RAM, and the like. Computer programs are stored in the RAM. The systematic LSI circuit achieves its functions when the microprocessor operates according to the computer programs.
  • IC cards or modules capable of being attached to and detached from the devices.
  • the IC cards or modules are each a computer system including a microprocessor, a ROM, a RAM, and the like.
  • the IC cards or modules may include the above-described ultra-multifunctional LSI circuit.
  • the IC cards or modules achieve their functions when the microprocessor operates according to computer programs.
  • the IC cards or modules may be tamper-resistant.
  • the driving assistance apparatus 1 uses the activity detection section 3 and the driver conditions detection section 4, the driving assistance apparatus 1 of this embodiment keeps monitoring an activity by the driver and conditions of the driver engaging in the activity while the vehicle is driving.
  • the driving mode selection unit 8 constantly determines, based on the road conditions and the like, whether autonomous driving will be continuable for a predetermined period of time (e.g., one hour) or longer from the current time.
  • the take-over request method determination unit 5 determines a method of presenting a notice of the cancellation of the autonomous driving mode by referring to the activity by the driver detected at that time and a detection result of a particular driver condition corresponding to that activity. Then, the presentation controller 6 presents the take-over request according to the determined method, including the timing for starting the presentation, a presentation device to use for the presentation, and the content of the presentation.
  • the "particular driver condition” is a condition (e.g., a concentration level) deemed necessary in making a determination whether the driver can really return from the driving-unrelated activity to manual driving, and is predetermined for each of activities of the driver that may be detected.
  • the take-over request method determination unit 5 changes the timing to start the presentation of a take-over request, the device to use for the presentation (such as one that outputs sound or one that displays a screen), and the content of the presentation, based not only on the level of wakefulness of the driver engaging in the activity, but also on the level of concentration on the activity. For instance, if the driver is watching TV drowsily or is absorbed in playing a video game and extremely concentrated on the screen, it is likely that the driver needs more time to be in a state capable of manual driving than usual. Thus, in this embodiment, the take-over request method determination unit 5 makes the timing to start presenting the take-over request earlier than usual and highlights the take-over request more than usual by selecting such a presentation device and content.
  • the driving assistance apparatus 1 changes the method of presenting a notice of autonomous driving cancellation according to the driver's activity and conditions, helping the driver who has been disengaged from driving in the autonomous driving mode return safely and smoothly to a state capable of manual driving.
  • the following provides detailed descriptions about the operations of the driving assistance apparatus 1 according to this embodiment.
  • Fig. 2 is a functional block exemplifying functions of the activity detection section 3 in the driving assistance apparatus 1.
  • Fig. 3 is a diagram illustrating operation of the activity detection section 3.
  • the activity detection section 3 includes an activity estimator 300 that detects an activity in which the driver is presumably engaging.
  • the activity detection section 3 receives sensor signals from the sensors 21, 22, and 23 of the first sensor group 2 and an output signal from the vehicle-mounted devices IF 20, and sequentially performs detection processing for recognizing the activity in which the driver is engaging while the vehicle is driving. Based on the results of the detection processing, the activity estimator 300 detects an activity by the driver among various kinds of activities, such as texting, playing a video game, watching TV, and so on. Examples of the activities are listed in Fig. 3 .
  • the vehicle-mounted devices IF 20 When a device communicatively connected to the vehicle-mounted devices IF 20, such as an automotive navigation system 61a or the smartphone 65, is operated, the vehicle-mounted devices IF 20 receives operation information including the device being operated and the content of the operation, and generates an output signal indicating the operation information.
  • a device communicatively connected to the vehicle-mounted devices IF 20 such as an automotive navigation system 61a or the smartphone 65.
  • the activity detection section 3 includes an operated-device detector 301 and an operation content detector 302. Based on the output signal from the vehicle-mounted devices IF 20, the operated-device detector 301 detects the device indicated by the operation information included in the output signal, and the operation content detector 302 detects the content of the operation indicated by the operation information.
  • the activity estimator 300 estimates that the driver's activity is "playing a video game with a smartphone".
  • the activity detection section 3 further includes an upper body image detector 303, a body motion recognizer 304, a held-object recognizer 305, a facial image extractor 306, and a viewed-object detector 307.
  • the activity detection section 3 receives image data indicating an image captured by the in-vehicle camera 21, and performs the following image recognition processing using these parts of the activity detection section 3 based on the image captured by the in-vehicle camera 21.
  • the upper body image detector 303 Based on the image data from the in-vehicle camera 21, the upper body image detector 303 detects an image of the upper body of the driver in the captured image. For example, with image data on an image captured when no one is in the driver's seat being stored in the internal memory in advance, the upper body image detector 303 reads the stored image data, compares it with image data on an image which has just been captured, and detects an upper body image based on the different between these pieces of image data.
  • the body motion recognizer 304 recognizes a body motion of the driver engaging in a certain activity based on the upper body image detected. Specifically, for example, the body motion recognizer 304 may detect the positions and postures of the arms and hands based on the contour of the upper body image. Body motions of the driver include touching the screen of a device with a finger, pushing a button on a device, moving a hand and/or an arm to hold an object, and moving the mouth to eat or speak.
  • the held-object recognizer 305 When the body motion of holding an object has been detected, the held-object recognizer 305 performs image processing to recognize the object being held. For example, the held-object recognizer 305 extracts an image of the object being held near the hand from the upper body image, compares it with each piece of image data stored in advance as held-object candidates, such as a smartphone, a game console, and a book, and determines the similarities between the extracted image and the stored image data.
  • held-object candidates such as a smartphone, a game console, and a book
  • the facial image extractor 306 extracts a facial image from the detected upper body image based for example on a feature amount related to human faces. Alternatively, the facial image extractor 306 may extract a facial image from the image data on the entire captured image.
  • the viewed-object detector 307 detects the object at which the driver is looking, based on the orientation of the face detected in the extracted facial image or from the direction of the line of sight estimated from images of the eyes extracted from the facial image. For example, the viewed-object detector 307 detects an object towards which the face or the eyes are directed, by analyzing the entire captured image or estimating the arrangement of devices in the vehicle based on the position of the in-vehicle camera 21.
  • the activity estimator 300 can detect an activity, such as operating a device, watching content on a device (these devices do not have to be communicatively connected to the vehicle-mounted devices IF 20), reading a book, a document, or the like, and eating (see Fig. 3 ).
  • the activity detection section 3 further includes a voice detector 308, a speaker number detector 309, an environmental sound detector 310, and an environmental sound recognizer 311.
  • the activity detection section 3 receives data on sound picked up by the microphone 22, and performs voice recognition processing and environmental sound recognition processing using these parts as follows.
  • the voice detector 308 detects human voice by, for example, extracting a feature amount unique to a human voice from the data on sound picked up by the microphone 22.
  • the speaker number detector 309 detects the number of speakers based for example on the individual differences among the feature amounts of the voices detected. Thereby, when a human voice is detected, the activity detection section 3 can presume that the driver activity is, as illustrated in Fig. 3 , talking on the phone if there is one speaker, and talking with a passenger or passengers if there is more than one speaker.
  • the environmental sound detector 310 detects an environmental sound in the data on the sound picked up by the microphone 22, the environmental sound being determined as not being a human voice based on predetermined feature amounts and the like.
  • the environmental sound recognizer 311 performs processing for identifying the environmental sound detected. For example, the environmental sound recognizer 311 compares the detected environmental sound with each piece of environmental sound data pre-stored as environmental sound candidates, such as music, radio sound, chewing sound, or sound generated by operation of a device such as a shaver.
  • the activity detection section 3 further includes a body movement information detector 313.
  • the body movement information detector 313 detects body movement information on a body movement of the driver based on a sensor signal from the body movement sensor 23.
  • the body movement information detector 313 thus detects a body movement that the driver makes to change their position during a certain one of various activities, and thereby improves the accuracy of the body motion detected to estimate the driver's activity (see Fig. 3 ).
  • driver conditions detection section 4 detects driver conditions.
  • Fig. 4 is a functional block diagram exemplifying functions of the wakefulness level detection part 41 in the driving assistance apparatus 1.
  • the level of wakefulness is a measure of how awake the driver is, as opposed to how drowsy they are. Hence, when drowsy, the driver has a low level of wakefulness.
  • the wakefulness level detection part 41 detects the level of wakefulness of the driver by, for example, analyzing an image captured by the in-vehicle camera 21. As shown in Fig. 4 , the wakefulness level detection part 41 includes a facial image extractor 411, an eye opening degree detector 412, an eye blink detector 413, a head position detector 414, and a wakefulness level estimator 410.
  • the wakefulness level detection part 41 performs image analysis using its parts as follows. First, the facial image extractor 411 acquires image data from the in-vehicle camera 21 at any time, and extracts a facial image of the driver from the captured image. Next, the eye opening degree detector 412 detects the eye opening degree indicating how much the eyelids are open in the extracted facial image. It is likely that the drowsier the driver is, the more the eyelids are closed, making the eye opening degree smaller. Thus, the wakefulness level estimator 410 calculates an estimated level of wakefulness based on the eye opening degree detected: the smaller the eye opening degree, the lower the level of wakefulness.
  • the eye blink detector 413 detects how long the eyelids are closed and how many times the eyes blink based on facial images captured over a predetermined period of time (e.g., one minute).
  • the wakefulness level estimator 410 estimates the level of wakefulness based on the eye-closed time and the number of eye blinks detected: the longer the eye-closed time and/or the smaller the number of eye blinks, the lower the level of wakefulness.
  • the head position detector 414 detects the position of the head of the driver based on the position of the facial image extracted from the image captured by the in-vehicle camera 21.
  • the wakefulness level estimator 410 estimates the level of wakefulness based on detection results of the position of the head over a predetermined period of time (e.g., one minute): the larger the rate of change in the position of the head, the lower the level of wakefulness.
  • the wakefulness level detection part 41 may detect the level of wakefulness by analyzing voice in the vehicle. When the driver is asleep, sleeping sound such as a snore may be observed.
  • the wakefulness level detection part 41 includes a breathing sound detector 415.
  • the breathing sound detector 415 detects breathing sound of the driver from data on sound picked up by the microphone 22, and determines whether the detected breathing sound contains sleeping sound such as a snore.
  • the wakefulness level estimator 410 lowers the estimated level of wakefulness every time it determines that the detected breathing sound contains sleeping sound.
  • the wakefulness level detection part 41 may detect the level of wakefulness based on a measurement result from the heartbeat sensor 25. It is known that in heart rate variability (HRV) containing a high frequency (HF) component (e.g., 0.15 to 0.40 Hz) and a low frequency (LF) component (e.g., 0.04 Hz to 0.15 Hz), the HF component, which is attributable to a parasympathetic activity, is elevated when humans are drowsy.
  • HRV heart rate variability
  • HF high frequency
  • LF low frequency
  • the wakefulness level detection part 41 includes a heart rate variation detector 416 and an LF/HF detector 417.
  • the heart rate variation detector 416 detects heart rate variation based on sensor signals from the heartbeat sensor 25.
  • the LF/HF detector 417 detects the ratio of the LF component to the HF component (LF/HF) in the heart rate variation detected.
  • the wakefulness level estimator 410 estimates the level of wakefulness based on the LF/HF ratio detected: the smaller the LF/HF ratio, the lower the level of wakefulness.
  • the wakefulness level detection part 41 may include a body movement information detector 418.
  • the body movement information detector 418 detects, based on sensor signals from the body movement sensor 23, body movement information indicating that there has been a body movement of the driver.
  • the wakefulness level estimator 410 estimates the level of wakefulness based on body movements detected based on the body movement information: the more the body movements, the higher the level of wakefulness.
  • the wakefulness level detection part 41 may include an operation frequency detector 419.
  • the operation frequency detector 419 detects an operation frequency based on operation information included in output signals from the vehicle-mounted devices IF 20 over the past predetermined time (e.g., one minute).
  • the operation frequency indicates the number of times a device is operated within a predetermined period of time.
  • the wakefulness level estimator 410 estimates the level of wakefulness based on the operation frequency detected: the higher the operation frequency, the higher the level of wakefulness.
  • the wakefulness level estimator 410 estimates the level of wakefulness according to a result obtained by any of the eye opening degree detector 412, the eye blink detector 413, the head position detector 414, the breathing sound detector 415, the LF/HF detector 417, the operation frequency detector 419, and the body movement information detector 418.
  • the present disclosure is not limited to such examples, and the wakefulness level estimator 410 may estimate the level of wakefulness by considering all the results from the detectors comprehensively.
  • Fig. 5 is a functional block diagram exemplifying functions of the fatigue level detection part 42 in the driving assistance apparatus 1.
  • the level of fatigue is a measure of how tired the driver is physically.
  • the fatigue level detection part 42 detects the level of fatigue based on measurement results from the heartbeat sensor 25.
  • the fatigue level detection part 42 includes a heart rate detector 421, a heart rate variation detector 422, an LF/HF detector 423, and a fatigue level estimator 420.
  • the heart rate detector 421 detects the heart rate (HR) of the driver based on sensor signals from the heartbeat sensor 25. When the driver is tired, their heart rate may depart from the average heart rate at rest. In view of this, the fatigue level estimator 420 calculates an estimated level of fatigue based on the difference between the detected heart rate and the preset standard heart rate: the larger the difference, the higher the level of fatigue.
  • the heart rate variation detector 422 and the LF/HF detector 423 of the fatigue level detection part 42 detect a heart rate variation and an LF/HF ratio, respectively, as those of the wakefulness level detection part 41 do.
  • the LF component which is attributable to a sympathetic activity, is thought to be elevated when the driver is tired.
  • the fatigue level estimator 420 estimates the level of fatigue based on the LF/HF ratio detected: the larger the LF/HF ratio, the higher the level of fatigue.
  • the fatigue level detection part 42 may include a body movement information detector 424 that detects body movement information as that of the wakefulness level detection part 41 does.
  • the fatigue level estimator 420 estimates the level of fatigue based on the detected body movement information: the more the driver has been moving, the higher the level of fatigue.
  • the fatigue level detection part 42 may include an operation frequency detector 425 and an incorrect operation detector 426.
  • the operation frequency detector 419 of the wakefulness level detection part 41 does, the operation frequency detector 425 of the fatigue level detection part 42 detects operation frequency based on operation information from the vehicle-mounted devices IF 20.
  • the incorrect operation detector 426 detects incorrect operation frequency based on operation content included in the operation information from the vehicle-mounted devices IF 20.
  • the incorrect operation frequency is the number of incorrect operations performed within a predetermined period of time.
  • the fatigue level estimator 420 estimates the level of fatigue based on the operation frequency and the incorrect operation frequency detected: the lower the operation frequency and/or the higher the detected incorrect operation frequency, the higher the level of fatigue.
  • the fatigue level estimator 420 estimates the level of fatigue according to a result obtained by any of the heart rate detector 421, the heart rate variation detector 422, the LF/HF detector 423, the body movement information detector 424, the operation frequency detector 425, and the incorrect operation detector 426.
  • the present disclosure is not limited to such examples, and the fatigue level estimator 420 may estimate the level of fatigue by considering all the results from the detectors comprehensively. 2-3-3. Methods for Detecting the Level of Nervousness
  • FIG. 6 is a functional block diagram exemplifying functions of the nervousness level detection part 43 in the driving assistance apparatus 1.
  • the level of nervousness is a measure of how nervous (or excited) the driver is.
  • the nervousness level detection part 43 detects the level of nervousness based on a measurement result from the blood pressure sensor 24. As shown in Fig. 6 , the nervousness level detection part 43 includes a blood pressure detector 431 and a nervousness level estimator 430.
  • the blood pressure detector 431 detects the blood pressure of the driver based on a sensor signal from the blood pressure sensor 24. Humans tend to have a higher blood pressure when they are nervous. In view of this, the nervousness level estimator 430 calculates an estimated level of nervousness based on the blood pressure detected: the higher the blood pressure, the higher the level of nervousness.
  • the nervousness level detection part 43 may include a heart rate detector 432, a heart rate variation detector 433, and an LF/HF detector 434 that detect, respectively, a heart rate, a heart rate variation, and an LF/HF ratio as those in the fatigue level detection part 42 do.
  • the nervousness level estimator 430 estimates the level of nervousness based on the heart rate detected: the higher the heart rate, the higher the level of nervousness. Moreover, humans tend to have a heart rate variation with an elevated LF component when they are nervous. In view of this, the nervousness level estimator 430 estimates the level of nervousness based on the LF/HF ratio detected: the larger the LF/HF ratio, the higher the level of nervousness.
  • the nervousness level detection part 43 may include an incorrect operation detector 435 that detects the frequency of incorrect operations as that of the fatigue level detection part 42 does.
  • the nervousness level estimator 430 estimates the level of nervousness based on the incorrect operation frequency detected: the greater the incorrect operation frequency, the higher the level of nervousness.
  • the nervousness level estimator 430 estimates the level of nervousness according to a result obtained by any of the blood pressure detector 431, the heart rate detector 432, the heart rate variation detector 433, the LF/HF detector 434, and the incorrect operation detector 435.
  • the present disclosure is not limited to such examples, and the nervousness level estimator 430 may estimate the level of nervousness by considering all the results from the detectors comprehensively.
  • Fig. 7 is a functional block diagram exemplifying functions of the concentration level detection part 44 in the driving assistance apparatus 1.
  • the level of concentration is a measure of how much the driver is concentrated on the activity in which they are engaging.
  • the concentration level detection part 44 detects the level of concentration by analyzing images captured by the in-vehicle camera 21. As shown in Fig. 7 , the concentration level detection part 44 includes a facial image extractor 441, an eye movement detector 442, and a concentration level estimator 440. The facial image extractor 441 of the concentration level detection part 44 extracts a facial image as that of the wakefulness level detection part 41 does.
  • the eye movement detector 442 identifies the positions of the eyes in the facial images extracted over a predetermined period of time (e.g., one minute) and detects the movement of the eyes, that is, an eye movement rate.
  • a predetermined period of time e.g., one minute
  • the concentration level estimator 440 calculates an estimated level of concentration based on the eye movement rate detected: the lower the eye movement rate, the higher the level of concentration.
  • the concentration level detection part 44 may include an eye opening degree detector 443 that detects how much the eyes are open as that of the wakefulness level detection part 41 does, and/or an eye blink detector 444 that detects the number of eye blinks as that of the wakefulness level detection part 41 does.
  • the concentration level estimator 440 estimates the level of concentration based on the eye opening degree detected and/or the number of eye blinks detected: the larger the eye opening degree and/or the smaller the number of eye blinks, the higher the level of concentration.
  • the concentration level detection part 44 may further include an operation frequency detector 445 that detects the operation frequency as that of the fatigue level detection part 42 does, and/or an incorrect operation frequency detector 446 that detects the incorrect operation frequency as that of the fatigue level detection part 42 does.
  • an operation frequency detector 445 that detects the operation frequency as that of the fatigue level detection part 42 does
  • an incorrect operation frequency detector 446 that detects the incorrect operation frequency as that of the fatigue level detection part 42 does.
  • the concentration level estimator 440 estimates the level of concentration based on the operation frequency detected and/or the incorrect operation frequency detected: the higher the operation frequency and/or the lower the incorrect operation frequency, the higher the level of concentration.
  • the concentration level detection part 44 may further include a heart rate detector 447 that detects heart rate as that of the fatigue level detection part 42 does.
  • a heart rate detector 447 that detects heart rate as that of the fatigue level detection part 42 does.
  • the concentration level estimator 440 estimates the level of concentration based on the heart rate detected: the higher the heart rate, the higher the level of concentration.
  • the concentration level estimator 440 estimates the level of concentration according to a result obtained by any of the facial image extractor 441, the eye movement detector 442, the eye opening degree detector 443, the eye blink detector 444, the operation frequency detector 445, the incorrect operation frequency detector 446, and the heart rate detector 447.
  • the present disclosure is not limited to such examples, and the concentration level estimator 440 may estimate the level of concentration by considering all the results from the detectors comprehensively.
  • Fig. 8 is a flowchart illustrating how the driving assistance apparatus 1 performs the cancellation processing to cancel the autonomous driving mode.
  • the cancellation processing as illustrated in the flowchart of Fig. 8 is performed by the driving assistance apparatus 1 in the autonomous driving mode. During this cancellation processing, the activity detection section 3 and the driver conditions detection section 4 iterate their detection operations described above.
  • the road condition prediction part 82 predicts road conditions based on driving information from the driving information acquisition part 81 (S1). For example, if the host vehicle is travelling on an expressway, the road condition prediction part 82 predicts the time it will take to arrive at the exit of the expressway.
  • the driving information indicates the driving conditions of the host vehicle and the surrounding environment of the host vehicle. A description will be given later of how the driving information is acquired.
  • the autonomous driving continuance determination part 83 determines whether the autonomous driving mode will be continuable for a predetermined period of time or longer from the current time (S2).
  • This predetermined period of time is set with a margin for advancing the timing to start presenting a take-over request to the maximum degree, or in other words, the time considered enough to bring the driver who has been disengaged from driving back to a state capable of manual driving, well in advance of the cancellation of the autonomous driving mode.
  • An example of the predetermined period of time is ten minutes.
  • the driving mode selection unit 8 returns to Step S1 and iterates the processing at a predetermined interval (e.g., one minute).
  • the take-over request method selection section 50 of the take-over request method determination unit 5 acquires information on the activity detected by the activity detection section 3 and information on the driver's conditions detected by the driver conditions detection section 4, these pieces of information being ones detected at the time of the determination. Based on the thus-acquired detection results of the driver's activity and conditions, the take-over request method determination unit 5 determines a take-over request method (S3).
  • the take-over request method specifies how a take-over request is presented inside the vehicle.
  • the take-over request start timing calculator 51 of the take-over request method selection section 50 calculates the timing to start presenting a take-over request, and sets the timing in, for example, the presentation controller 6.
  • the take-over request presentation device selector 52 selects, from the presentation devices 60, a presentation device to use to present the take-over request, and sets the presentation device in the presentation controller 6.
  • the take-over request presentation content selector 53 selects the content to present in the take-over request and sets the presentation content in the presentation controller 6.
  • the presentation controller 6 determines whether the current time has reached the start timing set by the take-over request start timing calculator 51 (S4). The presentation controller 6 iterates this step until the current time reaches the set start timing.
  • the presentation controller 6 When the current time reaches the set start timing (Yes in S4), the presentation controller 6 presents the take-over request according to the take-over request method determined by the take-over request method determination unit 5 (S5). Specifically, the presentation controller 6 controls the presentation of the take-over request so that the take-over request may be presented with the selected content using the selected presentation device.
  • the autonomous driving cancellation determination part 84 of the driving mode selection unit 8 determines whether the driver is ready for manual driving (S6).
  • the autonomous driving cancellation determination part 84 performs this processing by determining, for example, whether the driver has stopped the driving-unrelated activity or whether the driver is looking ahead of the vehicle. This determination may also be made using detection results of physiological/psychological conditions, such as the driver's wakefulness level (by checking whether the wakefulness level is not too low), when appropriate.
  • the take-over request method determination unit 5 determines a take-over request method anew (S3). In this case, the take-over request method is re-set considering the fact that the take-over request is going to be presented a second time.
  • the autonomous/manual driving switch part 85 controls the vehicle controller 80 so that the vehicle controller 80 will cancel the autonomous driving mode and switch the driving mode to the manual driving mode (S7). Then, the vehicle controller 80 cancels the autonomous driving control of the vehicle.
  • the driving mode of the vehicle is switched to the manual driving mode, and the driving assistance apparatus 1 ends the processing illustrated in the flowchart of Fig. 8 .
  • the above-described cancellation processing changes the take-over request method in accordance with an activity in which the driver is engaging in the autonomous driving mode as well as physiological/psychological conditions of the driver engaging in that activity. This enables the driver to safely and smoothly return to manual driving.
  • the presentation controller 6 iterates the determination step S4 until the current time reaches the set start timing.
  • the present disclosure is not limited to this For example, it is possible that the driver's activity and conditions may change between the current time and the set start timing. For this reason, if for example the determination result of Step S4 is "No", the take-over request method determination unit 5 may check new detection results from the activity detection section 3 and the driver conditions detection section 4. If there is any change in the driver's activity or conditions, the take-over request method determination unit 5 may update the take-over request method according to the change.
  • the take-over request method determination unit 5 determines the take-over request method anew (S3). If the determination in Step S6 turns out to be "No" a predetermined number of times or more, the flowchart may be ended. Then, the vehicle controller 80 may control the vehicle so that, for example, the vehicle will be parked in a safe place.
  • Fig. 9 is a functional block diagram exemplifying functions of the driving information acquisition part 81.
  • the driving information acquisition part 81 includes a vehicle travel position calculator 811, a map data storage 812, a vehicle travel information acquirer 813, a surrounding environment information acquirer 814, and a traffic information acquirer 815.
  • the second sensor group 7 includes the camera 71, the radar 72, and the GPS 73. It is assumed that the vehicle is equipped with a wireless communication unit 11 as a communication module capable of wirelessly communicating with a network such as the Internet.
  • the driving information acquisition part 81 acquires GPS information from the GPS 73 of the second sensor group 7.
  • the map data storage 812 has map data stored therein in advance. Based on the GPS information thus acquired, the vehicle travel position calculator 811 calculates the travelling position of the host vehicle on the map data stored in the map data storage 812.
  • the vehicle travel information acquirer 813 acquires vehicle travel information indicating travelling states, such as a travelling speed, of the host vehicle whose driving is controlled by the vehicle controller 80.
  • Step S1 of the flowchart in Fig. 8 the road condition prediction part 82 estimates, based on tinformation of the vehicle position, the map data, and the vehicle travel information, the time it will take for the host vehicle to arrive at the exit of the expressway. Then, in Step S2 of the flowchart in Fig. 8 , the autonomous driving continuance determination part 83 compares the estimated time and the predetermined period of time which has been set in advance (see 2-4), and proceeds to "No" when the estimated time falls below the predetermined time.
  • the surrounding environment information acquirer 814 acquires image data on captured images of the environment outside the host vehicle from the camera 71.
  • the surrounding environment information acquirer 814 may also acquire measurement information from the radar 72.
  • the image data from the camera 71 and the measurement information from the radar 72 are each an example of surrounding environment information including information on vehicles surrounding the host vehicle.
  • the traffic information acquirer 815 acquires traffic information from the wireless communication unit 11.
  • the traffic information indicates real-time traffic conditions, such as the location and size of traffic congestion, which are available on the Internet or the like.
  • the road condition prediction part 82 predicts, for example, the traffic conditions surrounding the host vehicle within the aforementioned predetermined period of time. For instance, the road condition prediction part 82 predicts whether the host vehicle would pass the congested area within the predetermined time. Based on a result of the prediction by the road condition prediction part 82, the autonomous driving continuance determination part 83 determines whether the autonomous driving mode will be continuable for the predetermined period of time or longer from the current time, or in other words, whether the autonomous driving system will face its functional limit. 2-4-2. Processing for Determining the Take-over request method
  • Fig. 10 shows an example of an activities and conditions table D2.
  • the take-over request method determination unit 5 refers to the activities and conditions table D2 and selects, from various measures indicative of driver conditions, a measure used to determine the take-over request method.
  • the activities and conditions table D2 is a data table in which each driver activity is recorded in association with measures of driver conditions which should be considered for the driver activity.
  • detection target activities are each associated with wakefulness level, fatigue level, nervousness level, and concentration level.
  • the detection target activities include texting, using SNS, browsing the Web, playing a video game, watching TV, reading, listening to music, listening to the radio, conversing, talking on the phone, having a meal, and performing personal maintenance (such as shaving or putting on a makeup).
  • the take-over request method selection section 50 selects measures that are assigned "O" in the activities and conditions table D2 for the activity detected by the activity detection section 3.
  • concentration level is assigned "O" for activities such as texting, using SNS, browsing the Web, playing a video game, watching TV, having a meal, and performing personal maintenance. This is because when the driver is concentrated on such activities, it is likely that the driver fixes their eyes to a particular object (such as the game screen if playing a video game) and pays almost no attention to, for example, the environment surrounding the host vehicle. This means it may take a longer time for the driver to return to manual driving.
  • the take-over request method determination unit 5 acquires estimated levels of the respective measures selected as described above based on the activity detected by the activity detection section 3, and determines a take-over request method by referring to the take-over request methods database D1.
  • Fig. 11 shows an example of the take-over request methods database D1.
  • the take-over request methods database D1 manages driver's activities, driver's conditions, and take-over request methods in association with one another.
  • Fig. 11 shows an example of the take-over request methods database D1 where the driver's activities are categorized into two groups: one with texting, using SNS, browsing the Web, playing a video game, and watching TV, and one with reading.
  • the take-over request method determination unit 5 determines a take-over request method based on the levels of wakefulness, fatigue, nervousness, and concentration (see Fig. 10 ).
  • the take-over request methods database D1 exemplified in Fig. 11 has each driver condition ranked “low”, “medium”, or “high”, and manages take-over request methods corresponding to these three driver condition ranks. Specifically, the take-over request method determination unit 5 ranks each of the estimated levels acquired from the driver conditions detection section 4 "low”, “medium”, or “high” by comparing the estimated level with two thresholds levels.
  • the take-over request presentation content selector 53 of the take-over request method determination unit 5 sets the presentation content of the take-over request to be "notify the driver of switch to manual driving".
  • the take-over request presentation device selector 52 selects a display system and a speaker system as presentation devices. Thereby, as shown in Fig. 11 , as a take-over request, this notification content is displayed on a certain screen such as the automotive navigation system 61a, and outputted as audio.
  • the take-over request start timing calculator 51 calculates the timing to start presenting a take-over request for the driver in a normal condition, the timing being a time point which is a predetermined takeover period ahead of the estimated time at which the driving mode switches to manual driving. For example, the takeover period is estimated for each activity as a period of time necessary for the driver to return to manual driving. With reference to Figs. 12 and 13 , a description is given of how the take-over request start timing calculator 51 calculates the timing to start presenting a take-over request.
  • the graph in Fig. 12 depicts the relation between the level of concentration and the timing to start presenting a take-over request in a case where the levels of wakefulness, fatigue, and nervous are ranked "medium”.
  • L1 and L2 are threshold levels used in determining the rank of the level of concentration ("low”, “medium”, or "high”).
  • t0 indicates the timing to start presenting a take-over request to be used if the driver is in the normal condition.
  • the graph in Fig. 13 shows the relation between the level of nervousness and the timing to start presenting a take-over request in a case where the levels of wakefulness, fatigue, and concentration are ranked "medium”.
  • L11 and L12 are threshold levels used in determining the rank of the level of nervousness out of the three ranks.
  • the storage 55 of the take-over request method determination unit 5 has stored therein information D31 in Fig. 12 and information D32 in Fig. 13 for each of the ranks of driver condition levels: "low”, “medium”, and "high".
  • the take-over request start timing calculator 51 refers to the information D31, like the graph in Fig. 12 , and calculates the start timing suitable for the estimated level of concentration detected by the driver conditions detection section 4.
  • the level of concentration is ranked "high”
  • the higher the level of concentration the earlier the start timing is than the normal timing t0. Consequently, the more the driver is concentrated on their current activity, such as playing a video game, the earlier the take-over request is presented, helping the driver to return to manual driving safely.
  • the take-over request start timing calculator 51 refers to the information D32, like the graph in Fig. 13 , and calculates the start timing suitable for the estimated level of nervousness. Consequently, the more nervous the driver is, the earlier the take-over request is presented, helping the driver to return to manual driving safely.
  • the take-over request presentation content selector 53 sets the presentation content of the take-over request to "give a warning prompting the driver to stop the activity".
  • the take-over request presentation device selector 52 determines how to present the take-over request having the warning content, such as displaying the notice on a particular screen of a display system and outputting warning audio from a speaker system, as shown in Fig. 11 .
  • the take-over request presentation device selector 52 determines to vibrate the seat to shake the body of the driver using an actuator system as a presentation device.
  • the take-over request presentation device selector 52 selects the device whose screen is being watched by the driver (e.g., the smartphone used for texting or the in-vehicle TV used for watching TV) as a display system used as the presentation device, as shown in Fig. 11 . If the activity in which the driver is engaging is reading, the screen of the automotive navigation system 61a is selected as a display system used as the presentation device in this embodiment, as shown in Fig. 11 .
  • the take-over request presentation content selector 53 sets the presentation content of the take-over request to "give a warning prompting the driver to wake up”.
  • the take-over request presentation device selector 52 determines how to present the take-over request having such a content, such as displaying the notice on a particular screen of a display system, outputting warning audio or an alarm from a speaker system, vibrating the seat using an actuator system, and/or ejecting cold air from the air conditioner, as shown in Fig. 11 .
  • the take-over request start timing calculator 51 calculates the start timing for the take-over request based for example on the estimated level of wakefulness, so that the take-over request will be presented earlier than the normal timing.
  • the driving assistance apparatus 1 assists a transition from the autonomous driving mode in which the vehicle is driven under autonomous control to the manual driving mode in which the vehicle is driven by the driver.
  • the driving assistance apparatus 1 includes the activity detection section 3, the driver conditions detection section 4, and the take-over request method determination unit 5.
  • the activity detection section 3 detects an activity in which the driver is engaging while the vehicle is driving in the autonomous driving mode.
  • the driver conditions detection section 4 detects a plurality of conditions of the driver while the vehicle is driving.
  • the take-over request method determination unit 5 determines how to present a take-over request in the vehicle to inform the driver that the autonomous driving mode is going to be cancelled.
  • the take-over request method determination unit 5 sets a take-over request method, which is a method of presenting a take-over request, in the presentation controller 6 based on the driver's activity detected by the activity detection section 3 and on the driver's conditions detected by the driver conditions detection section 4.
  • This driving assistance apparatus 1 sets the take-over request method based on an activity by the driver and conditions of the driver performing that activity. This helps the driver return to manual driving in a transition from the autonomous driving mode to the manual driving mode.
  • the device to use for the presentation and the timing to start the presentation can be changed appropriately depending not only on the driver's activity, but also on the physiological and psychological conditions of the driver engaging in that activity. For instance, when the driver is extremely concentrated on an activity such as playing a video game, it is likely that it will take a longer time for the driver to be completely ready for manual driving, than for the driver not concentrated on the activity.
  • the driving assistance apparatus 1 is configured to be able to change the timing to present the take-over request according to the driver's level of concentration on playing the video game, which is detected by the driver conditions detection section 4, and therefore allows the driver to be completely ready for manual driving.
  • the take-over request method determination unit 5 changes the take-over request method based on the detection result of a certain condition of the driver engaging in the detected activity (e.g., when the level of concentration is high), and sets the thus-changed take-over request method in the presentation controller 6. This enables the driver engaging in a driving-unrelated activity to return to manual driving safely and smoothly.
  • the driver conditions detection section 4 detects the levels of a plurality of measures indicative of conditions of the driver.
  • the take-over request method determination unit 5 sets the take-over request method based on the detected levels of a certain combination of measures among the plurality of measures which is suitable for the driver's detected activity (see Fig. 10 ). In this way, a take-over request method is appropriately set based on the levels of measures, which are one of various combinations of measures which is suitable for the activity in which the driver is engaging.
  • the plurality of conditions include a psychological condition indicated by the level of concentration of the driver. This enables the take-over request method to be set with the driver's level of concentration taken into account.
  • the driver conditions detection section 4 detects the level of concentration of the driver.
  • the take-over request method determination unit 5 sets the take-over request method so that the presentation content of the take-over request may be highlighted according to the detected level of concentration. This way, the presentation content of the take-over request is highlighted more when the driver's level of concentration on the particular activity is higher. This helps the driver to return to manual driving.
  • the take-over request method determination unit 5 sets the take-over request method so that the timing to start presenting the take-over request will be advanced according to the detected level of concentration. This way, the presentation of the take-over request starts earlier when the driver is over-concentrated on a particular activity. This helps the driver to return to manual driving even more.
  • the plurality of conditions include a physiological condition of the driver.
  • the take-over request method can be set with a physiological condition of the driver, such as drowsiness, taken into account.
  • the setting the take-over request method includes setting at least one of the timing to start presenting the take-over request, the content presented in the take-over request, and the device to use to present the take-over request. This way, the take-over request method can be set from various perspectives.
  • the driving assistance apparatus 1 further includes the first sensor group 2 and the vehicle-mounted devices IF 20 as a sensor unit to sense information on the driver while the vehicle is driving.
  • the activity detection section 3 detects the activity by the driver based on the sensed information.
  • the driver conditions detection section 4 detects the plurality of conditions based on the sensed information. Such monitoring of the driver using the sensor unit enables accurate detection of the driver's activity and conditions.
  • the sensor unit of the driving assistance apparatus 1 includes at least one of the in-vehicle camera 21 that captures an image of the inside of the vehicle, the microphone 22 that picks up voice in the vehicle, the vehicle-mounted devices IF 20 that receives information from devices in the vehicle, the heartbeat sensor 25, the blood pressure sensor 24, and the body movement sensor 23. These various sensor devices allow monitoring of the driver.
  • the sensor unit of the driving assistance apparatus 1 may include any combination of the above sensor devices, or other sensor devices such as a perspiration sensor.
  • the activity detection section 3 and the driver conditions detection section 4 may detect the driver's activity and conditions using information sensed by any device included in the sensor unit.
  • the driving assistance apparatus 1 further includes the driving information acquisition part 81 and the autonomous driving continuance determination part 83.
  • the driving information acquisition part 81 acquires driving information, which indicates driving conditions of the vehicle.
  • the autonomous driving continuance determination part 83 determines, in the autonomous driving mode, whether the autonomous driving mode will be continuable for a predetermined period of time or longer.
  • the take-over request method determination unit 5 sets the take-over request method. This way, when it is determined based on the driving information that the autonomous driving mode will not be continuable for the predetermined period of time or longer, the take-over request method can be set ahead of time before the autonomous driving mode is actually cancelled.
  • the driving information includes information on the vehicle speed, information on the vehicle position, and surrounding environment information, which indicates the environment surrounding the vehicle.
  • the take-over request method determination unit 5 can thus determine the continuance of the autonomous driving mode based on various pieces of information.
  • the surrounding environment information includes information on the situation of any other vehicle around the vehicle, which allows the determination on the continuance of the autonomous driving mode to be made according to the situation of the surrounding vehicles.
  • the vehicle is equipped with the camera 71 that captures an image of the outside of the vehicle.
  • the driving information acquisition part 81 acquires surrounding environment information from the camera 71, which allows the determination on the continuance of the autonomous driving mode to be made according to any other vehicle in the images captured by the camera 71.
  • the driving assistance apparatus 1 assists a transition from the autonomous driving mode in which the vehicle is driven under autonomous control to the manual driving mode in which the vehicle is driven by the driver.
  • the method includes detecting, by the driving assistance apparatus 1, an activity in which the driver is engaging while the vehicle is driving.
  • the method includes detecting, by the driving assistance apparatus 1, a plurality of conditions of the driver while the vehicle is driving.
  • the method includes determining, by the driving assistance apparatus 1, a take-over request method, which is a method of presenting information in the vehicle to inform the driver that the autonomous driving mode is going to be canceled.
  • the determining a take-over request method includes setting the take-over request method based on the detected activity and conditions of the driver before the autonomous driving mode is cancelled.
  • This driving assistance method helps the driver return to manual driving in a transition from the autonomous driving mode to the manual driving mode.
  • the present disclosure includes the various methods described above.
  • these methods may be a computer program executed by a computer, or digital signals forming the computer program,
  • the computer programs or the digital signals may be recorded in a computer-readable recording medium, such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a Blu-ray Disc (BD, registered trademark), a USB memory device, a memory card, such as an SD card, or a semiconductor memory.
  • a computer-readable recording medium such as a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a Blu-ray Disc (BD, registered trademark), a USB memory device, a memory card, such as an SD card, or a semiconductor memory.
  • BD Blu-ray Disc
  • An aspect of the present disclosure may be the digital signals recorded in any of these recording media.
  • the computer program or the digital signals may be transmitted using a telecommunications line, a wireless or wired communications line, a network such as the Internet, data broadcasting, or the like.
  • One aspect of the present disclosure may be a computer system including a microprocessor and a memory, the memory having the computer program stored therein, the microprocessor being operated according to the computer program.
  • the above-described apparatus may be implemented by an independent computer system by transfer of the program or the digital signals recorded in the recording medium or by transfer of the program or the digital signals over the network.
  • the functional blocks in the block diagrams are just an example in terms of how they are divided. Two or more functional blocks may be implemented as one functional block, or one functional block may be divided into two or more functional blocks. One or more functions of a certain functional block may be implemented by another functional block. Similar functions of different functional blocks may be processed in parallel or by time division by single hardware or software.
  • the driving assistance apparatus has been described above based on the embodiment, but the present disclosure is not limited to the embodiment.
  • the one or more aspects also include various modifications of the embodiment that may be conceived of by those skilled in the art and modes formed in combination with a constituent of a different embodiment, as long as such modifications or modes do not depart from the gist of the present disclosure.
  • the driving assistance apparatus and the driving assistance method according to the present disclosure are applicable to an autonomous driving system that automates driving of the vehicle.

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
EP17173443.7A 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren Active EP3264211B1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP20212082.0A EP3809227B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2016127972A JP6778872B2 (ja) 2016-06-28 2016-06-28 運転支援装置及び運転支援方法

Related Child Applications (2)

Application Number Title Priority Date Filing Date
EP20212082.0A Division EP3809227B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren
EP20212082.0A Division-Into EP3809227B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren

Publications (2)

Publication Number Publication Date
EP3264211A1 true EP3264211A1 (de) 2018-01-03
EP3264211B1 EP3264211B1 (de) 2021-01-20

Family

ID=58992684

Family Applications (2)

Application Number Title Priority Date Filing Date
EP17173443.7A Active EP3264211B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren
EP20212082.0A Active EP3809227B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP20212082.0A Active EP3809227B1 (de) 2016-06-28 2017-05-30 Fahrassistenzvorrichtung und fahrassistenzverfahren

Country Status (4)

Country Link
US (3) US10449856B2 (de)
EP (2) EP3264211B1 (de)
JP (1) JP6778872B2 (de)
CN (1) CN107539318B (de)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3552913A3 (de) * 2018-04-11 2020-06-10 Hyundai Motor Company Vorrichtung und verfahren zur steuerung für die ermöglichung eines autonomen systems in einem fahrzeug
US10836394B2 (en) 2018-04-11 2020-11-17 Hyundai Motor Company Apparatus and method for lane change control
US10843710B2 (en) 2018-04-11 2020-11-24 Hyundai Motor Company Apparatus and method for providing notification of control authority transition in vehicle
CN111994087A (zh) * 2020-09-02 2020-11-27 中国第一汽车股份有限公司 一种辅助驾驶方法、系统、车辆及介质
US11077854B2 (en) 2018-04-11 2021-08-03 Hyundai Motor Company Apparatus for controlling lane change of vehicle, system having the same and method thereof
US11084491B2 (en) 2018-04-11 2021-08-10 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11084490B2 (en) 2018-04-11 2021-08-10 Hyundai Motor Company Apparatus and method for controlling drive of vehicle
US11173910B2 (en) 2018-04-11 2021-11-16 Hyundai Motor Company Lane change controller for vehicle system including the same, and method thereof
US11173912B2 (en) 2018-04-11 2021-11-16 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11334067B2 (en) 2018-04-11 2022-05-17 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11351989B2 (en) 2018-04-11 2022-06-07 Hyundai Motor Company Vehicle driving controller, system including the same, and method thereof
US11529956B2 (en) 2018-04-11 2022-12-20 Hyundai Motor Company Apparatus and method for controlling driving in vehicle
US11541889B2 (en) 2018-04-11 2023-01-03 Hyundai Motor Company Apparatus and method for providing driving path in vehicle
US11548509B2 (en) 2018-04-11 2023-01-10 Hyundai Motor Company Apparatus and method for controlling lane change in vehicle
US11597403B2 (en) 2018-04-11 2023-03-07 Hyundai Motor Company Apparatus for displaying driving state of vehicle, system including the same and method thereof
EP4119411A4 (de) * 2020-04-01 2023-07-26 Huawei Technologies Co., Ltd. Verfahren und vorrichtung zum automatischen fahren

Families Citing this family (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11823055B2 (en) * 2019-03-31 2023-11-21 Affectiva, Inc. Vehicular in-cabin sensing using machine learning
EP3378722B1 (de) * 2015-11-19 2024-02-21 Sony Group Corporation Fahrassistenzvorrichtung und fahrassistenzverfahren und sich bewegender körper
JP6778872B2 (ja) 2016-06-28 2020-11-04 パナソニックＩｐマネジメント株式会社 運転支援装置及び運転支援方法
MX2019001528A (es) * 2016-08-09 2019-07-04 Nissan Motor Metodo de control y dispositivo de control de vehiculo de conduccion automatica.
JP6365621B2 (ja) * 2016-10-19 2018-08-01 マツダ株式会社 運転支援装置
WO2018109868A1 (ja) * 2016-12-14 2018-06-21 本田技研工業株式会社 車両制御装置
CN109952227B (zh) * 2016-12-21 2021-10-08 本田技研工业株式会社 车辆控制系统、车辆控制方法及存储介质
JP6686869B2 (ja) * 2016-12-22 2020-04-22 株式会社デンソー 運転交代制御装置、及び運転交代制御方法
DE102017201405B4 (de) * 2017-01-30 2018-12-13 Audi Ag Verfahren zum Betreiben eines Kraftfahrzeugs
JP6863011B2 (ja) * 2017-03-31 2021-04-21 トヨタ自動車株式会社 操舵制御装置
JP6631577B2 (ja) * 2017-04-05 2020-01-15 株式会社デンソー 運転交代制御システム、運転交代制御プログラム、及び運転交代制御方法
US10906554B2 (en) * 2017-05-23 2021-02-02 Magna Electronics Inc. Autonomous driving system
JP6920112B2 (ja) * 2017-06-15 2021-08-18 株式会社デンソーテン 運転支援装置および運転支援方法
US10462281B2 (en) * 2017-06-30 2019-10-29 Intel Corporation Technologies for user notification suppression
KR101989523B1 (ko) * 2017-07-07 2019-06-14 엘지전자 주식회사 차량에 구비된 차량 제어 장치 및 차량의 제어방법
JP6652539B2 (ja) * 2017-10-12 2020-02-26 矢崎総業株式会社 自動運転時情報伝達方法および車載情報提示装置
DE102017219955A1 (de) * 2017-11-09 2019-05-09 Continental Automotive Gmbh System zum automatisierten Fahren mit Unterstützung eines Fahrers beim Ausüben einer fahrfremden Tätigkeit
US10368333B2 (en) * 2017-11-20 2019-07-30 Google Llc Dynamically adapting provision of notification output to reduce user distraction and/or mitigate usage of computational resources
US10642266B2 (en) * 2017-12-28 2020-05-05 Automotive Research & Testing Center Safe warning system for automatic driving takeover and safe warning method thereof
US11061399B2 (en) * 2018-01-03 2021-07-13 Samsung Electronics Co., Ltd. System and method for providing information indicative of autonomous availability
JP6969451B2 (ja) * 2018-03-08 2021-11-24 株式会社オートネットワーク技術研究所 車載制御装置、制御プログラム及び機器制御方法
CN112055865A (zh) * 2018-03-15 2020-12-08 Arm有限公司 用于行为和/或生物状态处理的系统、装置和/或过程
US11259729B2 (en) 2018-03-15 2022-03-01 Arm Ltd. Systems, devices, and/or processes for behavioral and/or biological state processing
US10373466B1 (en) 2018-03-15 2019-08-06 Arm Ltd. Systems, devices, and/or processes for tracking behavioral and/or biological state
WO2019176037A1 (ja) * 2018-03-15 2019-09-19 三菱電機株式会社 危険警告システム及び危険警告方法
JP7055043B2 (ja) * 2018-03-16 2022-04-15 本田技研工業株式会社 車両用制御装置及び車両
JP7118529B2 (ja) * 2018-03-29 2022-08-16 矢崎総業株式会社 車内監視モジュール、及び、監視システム
WO2019188398A1 (ja) * 2018-03-30 2019-10-03 ソニーセミコンダクタソリューションズ株式会社 情報処理装置、移動装置、および方法、並びにプログラム
CN108674415B (zh) * 2018-04-18 2021-04-02 北京汽车集团有限公司 车辆控制方法和装置
CN111989729B (zh) * 2018-04-20 2022-12-06 索尼半导体解决方案公司 信息处理装置、移动装置、信息处理系统、方法和程序
WO2019208450A1 (ja) * 2018-04-27 2019-10-31 パナソニックＩｐマネジメント株式会社 運転支援装置、運転支援方法及びプログラム
GB2574877B (en) * 2018-06-22 2020-10-28 Jaguar Land Rover Ltd Apparatus and methods for controlling a transition between driving modes
US10884410B2 (en) * 2018-06-29 2021-01-05 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for determining whether a vehicle is capable of navigating an intersection in an autonomous driving mode
CN108973678A (zh) * 2018-07-29 2018-12-11 合肥市智信汽车科技有限公司 一种汽车自动巡航控制系统
CN110871810A (zh) * 2018-08-21 2020-03-10 上海博泰悦臻网络技术服务有限公司 车辆、车机设备及其基于驾驶模式的行车信息提示方法
US10800409B2 (en) * 2018-09-04 2020-10-13 Caterpillar Paving Products Inc. Systems and methods for operating a mobile machine using detected sounds
US10650623B2 (en) * 2018-09-18 2020-05-12 Avinew, Inc. Detecting of automatic driving
JP7180228B2 (ja) * 2018-09-20 2022-11-30 いすゞ自動車株式会社 車両用監視装置
JP7070298B2 (ja) * 2018-09-28 2022-05-18 株式会社Ｊｖｃケンウッド 業務支援システムおよび業務支援方法
CN113016016A (zh) * 2018-11-26 2021-06-22 三菱电机株式会社 信息提示控制装置、信息提示装置和信息提示控制方法、以及程序和记录介质
CN109795505A (zh) * 2018-12-10 2019-05-24 北京百度网讯科技有限公司 自动驾驶判别方法、装置、计算机设备及存储介质
JP7156011B2 (ja) * 2018-12-26 2022-10-19 トヨタ自動車株式会社 情報提示装置
US11235776B2 (en) * 2019-01-31 2022-02-01 Toyota Motor Engineering & Manufacturing North America, Inc. Systems and methods for controlling a vehicle based on driver engagement
US10528047B1 (en) * 2019-01-31 2020-01-07 Chongqing Jinkang New Energy Automobile Co., Ltd. Method and system for monitoring user activity
US11294372B2 (en) 2019-02-15 2022-04-05 International Business Machines Corporation Driving mode decision support
JP2020157832A (ja) * 2019-03-25 2020-10-01 本田技研工業株式会社 車両の制御システム、車両における報知方法、及びプログラム
CN116370792A (zh) * 2019-03-29 2023-07-04 松下知识产权经营株式会社 专注度测量装置
KR20210003343A (ko) * 2019-07-01 2021-01-12 현대자동차주식회사 차량의 안전 장비 제어 장치 및 방법
US11414093B2 (en) 2019-08-30 2022-08-16 Toyota Research Institute, Inc. System and method for transitioning a vehicle from an autonomous mode in response to a handover event
US11608088B2 (en) * 2019-10-14 2023-03-21 Continental Automotive Systems, Inc. Optics based detection of hands on-off and hand gesture based function selection for human driver
CN112758098B (zh) * 2019-11-01 2022-07-22 广州汽车集团股份有限公司 基于驾驶员状态等级的车辆驾驶权限接管控制方法及装置
WO2021090402A1 (ja) * 2019-11-06 2021-05-14 日本電気株式会社 ストレス推定装置、ストレス推定方法、及び、記録媒体
WO2021106633A1 (ja) * 2019-11-27 2021-06-03 株式会社デンソー Ｈｍｉ制御装置、ｈｍｉ制御方法、ｈｍｉ制御プログラム、および運転制御装置
CN112959999A (zh) * 2019-11-27 2021-06-15 深圳市大富科技股份有限公司 汽车辅助驾驶的方法以及汽车辅助驾驶装置
JP7180661B2 (ja) * 2019-11-27 2022-11-30 株式会社デンソー Ｈｍｉ制御装置、ｈｍｉ制御方法、ｈｍｉ制御プログラム、および運転制御装置
CN114787848A (zh) * 2019-12-16 2022-07-22 松下知识产权经营株式会社 信息处理方法及信息处理系统
CN111942396B (zh) * 2020-02-17 2021-12-24 采埃孚汽车系统(上海)有限公司 自动驾驶控制装置、方法及自动驾驶系统
DE102020202388A1 (de) 2020-02-25 2021-08-26 Volkswagen Aktiengesellschaft Verfahren zum Einstellen einer Signalvorrichtung zur Ausgabe von Signalen an einen Fahrer und ein System
DE102020104856A1 (de) 2020-02-25 2021-08-26 Bayerische Motoren Werke Aktiengesellschaft Vorrichtung und Verfahren zur Steuerung der Rückholbereitschaft eines Fahrers eines Fahrzeugs
JP7402084B2 (ja) 2020-03-05 2023-12-20 本田技研工業株式会社 乗員行動判定装置
WO2021229636A1 (ja) * 2020-05-11 2021-11-18 日本電信電話株式会社 識別装置、識別方法及び識別プログラム
EP3915851B1 (de) * 2020-05-28 2024-05-01 Zenuity AB System und verfahren zur schätzung der übernahmezeit
JP7400672B2 (ja) * 2020-09-14 2023-12-19 トヨタ自動車株式会社 車両制御装置
CN114379563B (zh) * 2020-10-21 2023-08-08 广州汽车集团股份有限公司 一种安全驾驶控制方法及其系统、存储介质
CN112677981B (zh) * 2021-01-08 2022-06-07 浙江三一装备有限公司 用于作业机械安全驾驶的智能辅助方法及装置
JP2022150368A (ja) * 2021-03-26 2022-10-07 パナソニックＩｐマネジメント株式会社 支援装置
CN113222295B (zh) * 2021-06-07 2022-05-27 吉林大学 一种面向l3级自动驾驶汽车控制权切换状态下的接管时间预测方法
WO2022258150A1 (en) 2021-06-08 2022-12-15 Lotus Tech Innovation Centre Gmbh Computer-implemented method of assisting a driver, computer program product, driving assistance system, and vehicle
CN114162130B (zh) * 2021-10-26 2023-06-20 东风柳州汽车有限公司 驾驶辅助模式切换方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150094899A1 (en) * 2013-10-01 2015-04-02 Volkswagen Ag Method for Driver Assistance System of a Vehicle
JP2015210660A (ja) 2014-04-25 2015-11-24 日産自動車株式会社 情報呈示装置及び情報呈示方法
JP2015230573A (ja) 2014-06-05 2015-12-21 アルパイン株式会社 車両運転支援装置、方法およびプログラム
DE102014011264A1 (de) * 2014-07-28 2016-01-28 GM Global Technology Operations LLC (n. d. Ges. d. Staates Delaware) Verfahren zur Berechnung einer Rückholzeit
DE102014225680A1 (de) * 2014-12-12 2016-06-16 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zur Übernahmeaufforderung einer Fahraufgabe an den Fahrzeugführer eines Fahrzeugs im Rahmen des automatischen Fahrens des Fahrzeugs

Family Cites Families (122)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR940001633B1 (ko) * 1990-01-17 1994-02-28 미쯔비시 덴끼 가부시끼가이샤 주행 제어장치
US5572449A (en) * 1994-05-19 1996-11-05 Vi&T Group, Inc. Automatic vehicle following system
US5469356A (en) * 1994-09-01 1995-11-21 Caterpillar Inc. System for controlling a vehicle to selectively allow operation in either an autonomous mode or a manual mode
US6720920B2 (en) * 1997-10-22 2004-04-13 Intelligent Technologies International Inc. Method and arrangement for communicating between vehicles
JP3358403B2 (ja) * 1995-09-11 2002-12-16 トヨタ自動車株式会社 隊列走行制御装置
JP3633707B2 (ja) * 1996-03-08 2005-03-30 日産ディーゼル工業株式会社 車群走行制御装置
US6125321A (en) * 1996-06-07 2000-09-26 Toyota Jidosha Kabushiki Kaisha Motor vehicle drive system controller and automatic drive controller
JP3732292B2 (ja) * 1996-11-27 2006-01-05 本田技研工業株式会社 車群走行制御システム
DE19849583B4 (de) * 1997-10-27 2009-06-18 Nissan Motor Co., Ltd., Yokohama-shi System und Verfahren für das Steuern eines Abstandes zwischen Fahrzeugen
JP2000085407A (ja) * 1998-07-17 2000-03-28 Denso Corp 車間制御装置及び記録媒体
JP2003072488A (ja) * 2001-08-31 2003-03-12 Sony Corp 車載装置、車両及び車両情報の処理方法
JP2004017889A (ja) * 2002-06-19 2004-01-22 Advics:Kk 自動ブレーキ装置
US6882923B2 (en) * 2002-10-17 2005-04-19 Ford Global Technologies, Llc Adaptive cruise control system using shared vehicle network data
WO2004038335A1 (ja) * 2002-10-22 2004-05-06 Hitachi, Ltd. 通信型ナビゲーションシステムにおける地図データ配信方法
JP2004217175A (ja) * 2003-01-17 2004-08-05 Toyota Motor Corp 車間距離制御装置
US20050022606A1 (en) * 2003-07-31 2005-02-03 Partin Dale L. Method for monitoring respiration and heart rate using a fluid-filled bladder
DE10343683A1 (de) * 2003-09-20 2005-04-21 Daimler Chrysler Ag Informationssystem für Kraftfahrzeuge
US7266430B2 (en) * 2003-12-22 2007-09-04 International Business Machines Corporation Medical applications in telematics
US10227063B2 (en) * 2004-02-26 2019-03-12 Geelux Holdings, Ltd. Method and apparatus for biological evaluation
JP4735310B2 (ja) * 2005-04-15 2011-07-27 株式会社デンソー 走行支援装置
US20070233337A1 (en) * 2005-09-14 2007-10-04 Plishner Paul J Semi-autonomous guidance system for a vehicle
DE112006002894B4 (de) * 2005-10-21 2021-11-11 Deere & Company Vernetztes Vielzweck-Roboterfahrzeug
US7624943B2 (en) * 2006-03-22 2009-12-01 The Boeing Company Multi-mode unmanned and manned vehicle systems and methods
US20080258890A1 (en) * 2006-05-22 2008-10-23 Todd Follmer System and Method for Remotely Deactivating a Vehicle
US9067565B2 (en) * 2006-05-22 2015-06-30 Inthinc Technology Solutions, Inc. System and method for evaluating driver behavior
US7554435B2 (en) * 2006-09-07 2009-06-30 Nissan Technical Center North America, Inc. Vehicle on-board unit
US7692548B2 (en) * 2006-09-12 2010-04-06 Deere & Company Method and system for detecting operator alertness
US8532862B2 (en) * 2006-11-29 2013-09-10 Ryan A. Neff Driverless vehicle
US7652583B2 (en) * 2007-03-20 2010-01-26 Deere & Company Method and system for maintaining operator alertness
US7839292B2 (en) * 2007-04-11 2010-11-23 Nec Laboratories America, Inc. Real-time driving danger level prediction
JP4893507B2 (ja) * 2007-07-04 2012-03-07 オムロン株式会社 脇見検出装置および方法、並びに、プログラム
US20090051510A1 (en) * 2007-08-21 2009-02-26 Todd Follmer System and Method for Detecting and Reporting Vehicle Damage
US7719431B2 (en) * 2007-10-05 2010-05-18 Gm Global Technology Operations, Inc. Systems, methods and computer products for drowsy driver detection and response
US20090157461A1 (en) * 2007-12-12 2009-06-18 Honeywell International Inc. Vehicle deployment planning system
US8401736B2 (en) * 2008-06-20 2013-03-19 Toyota Jidosha Kabushiki Kaisha Driving assistance apparatus and driving assistance method
US20100007479A1 (en) * 2008-07-08 2010-01-14 Smith Matthew R Adaptive driver warning methodology
TWI335280B (en) * 2008-10-14 2011-01-01 Univ Nat Taiwan Image security warning system for vehicle use
US8126642B2 (en) * 2008-10-24 2012-02-28 Gray & Company, Inc. Control and systems for autonomously driven vehicles
JP4973687B2 (ja) * 2009-05-13 2012-07-11 トヨタ自動車株式会社 走行支援装置
US8384534B2 (en) * 2010-01-14 2013-02-26 Toyota Motor Engineering & Manufacturing North America, Inc. Combining driver and environment sensing for vehicular safety systems
US8618922B2 (en) * 2010-03-30 2013-12-31 GM Global Technology Operations LLC Method and system for ensuring operation of limited-ability autonomous driving vehicles
US10074024B2 (en) * 2010-06-07 2018-09-11 Affectiva, Inc. Mental state analysis using blink rate for vehicles
US9292471B2 (en) * 2011-02-18 2016-03-22 Honda Motor Co., Ltd. Coordinated vehicle response system and method for driver behavior
US8698639B2 (en) * 2011-02-18 2014-04-15 Honda Motor Co., Ltd. System and method for responding to driver behavior
WO2012129425A2 (en) * 2011-03-23 2012-09-27 Tk Holdings Inc. Driver assistance system
US9014915B2 (en) * 2011-07-25 2015-04-21 GM Global Technology Operations LLC Active safety control for vehicles
US8554468B1 (en) * 2011-08-12 2013-10-08 Brian Lee Bullock Systems and methods for driver performance assessment and improvement
US20140240115A1 (en) * 2011-09-26 2014-08-28 Toyota Jidosha Kabushiki Kaisha Driving assistance system for vehicle
DE102011117297A1 (de) * 2011-11-01 2013-05-02 Volkswagen Aktiengesellschaft Verfahren zum Betreiben eines Fahrerassistenzsystems und dazugehöriges Fahrerassistenzsystem
KR20130050113A (ko) * 2011-11-07 2013-05-15 현대자동차주식회사 위험단계별 경고기능을 갖는 차량 운행보조 시스템 및 그 방법
DE102012002695A1 (de) * 2012-02-14 2013-08-14 Wabco Gmbh Verfahren zur Ermittlung einer Notbremssituation eines Fahrzeuges
US8718861B1 (en) * 2012-04-11 2014-05-06 Google Inc. Determining when to drive autonomously
DE102012206725A1 (de) * 2012-04-24 2013-10-24 Ford Global Technologies, Llc Verfahren und Vorrichtung zum Versetzen eines autonom fahrenden Kraftfahrzeugs in einen sicheren Zustand
US8825258B2 (en) * 2012-11-30 2014-09-02 Google Inc. Engaging and disengaging for autonomous driving
JP5765326B2 (ja) * 2012-12-19 2015-08-19 株式会社デンソー 車車間通信装置および隊列走行制御装置
KR102017884B1 (ko) * 2013-02-06 2019-10-22 한국전자통신연구원 운전자 집중도 분석 장치 및 방법
US9801580B2 (en) * 2013-03-22 2017-10-31 Toyota Jidosha Kabushiki Kaisha Driving assistance device, driving assistance method, information-providing device, information-providing method, navigation device and navigation method
US9342074B2 (en) * 2013-04-05 2016-05-17 Google Inc. Systems and methods for transitioning control of an autonomous vehicle to a driver
EP2817787A4 (de) * 2013-04-15 2015-10-21 Flextronics Ap Llc Erfassung und anzeige eines fahrzeugeindringungsalarms
US8874301B1 (en) * 2013-07-09 2014-10-28 Ford Global Technologies, Llc Autonomous vehicle with driver presence and physiological monitoring
KR101470190B1 (ko) * 2013-07-09 2014-12-05 현대자동차주식회사 자율주행 시스템의 고장 처리 장치 및 그 방법
US9454150B2 (en) * 2013-07-17 2016-09-27 Toyota Motor Engineering & Manufacturing North America, Inc. Interactive automated driving system
EP2848488B2 (de) * 2013-09-12 2022-04-13 Volvo Car Corporation Verfahren und Einrichtung zur Übergabewarnung in einem Fahrzeug mit autonomer Fahrfähigkeit
DE102013110909A1 (de) * 2013-10-01 2015-04-02 Scania Cv Ab Vorrichtung zum automatischen Fahren eines Fahrzeugs
US9141112B1 (en) * 2013-10-16 2015-09-22 Allstate Insurance Company Caravan management
DE102013222048B4 (de) * 2013-10-30 2023-10-05 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zum Aktivieren oder Deaktivieren einer automatischen Fahrfunktion
DE102013019141A1 (de) * 2013-11-15 2015-05-21 Audi Ag Fahrmoduswechsel beim Fahrerassistenzsystem
CN114089813A (zh) * 2013-11-29 2022-02-25 普罗克西有限公司 穿戴式计算装置
DE102013020933A1 (de) * 2013-12-11 2015-06-11 Daimler Ag Verfahren zum automatischen Betrieb eines Fahrzeuges
US9650051B2 (en) * 2013-12-22 2017-05-16 Lytx, Inc. Autonomous driving comparison and evaluation
US9355423B1 (en) * 2014-01-24 2016-05-31 Allstate Insurance Company Reward system related to a vehicle-to-vehicle communication system
US9720411B2 (en) * 2014-02-25 2017-08-01 Ford Global Technologies, Llc Autonomous driving sensing system and method
US9511764B2 (en) * 2014-02-28 2016-12-06 Ford Global Technologies, Llc Semi-autonomous mode control
EP2915718B1 (de) * 2014-03-04 2018-07-11 Volvo Car Corporation Vorrichtung und Verfahren zur kontinuierlichen Herstellung einer Grenze für autonome Antriebsverfügbarkeit und Kraftfahrzeug mit einer solchen Vorrichtung
US9734685B2 (en) * 2014-03-07 2017-08-15 State Farm Mutual Automobile Insurance Company Vehicle operator emotion management system and method
WO2015151243A1 (ja) * 2014-04-02 2015-10-08 日産自動車株式会社 車両用情報呈示装置
WO2015162784A1 (ja) * 2014-04-25 2015-10-29 日産自動車株式会社 情報呈示装置及び情報呈示方法
US9141109B1 (en) * 2014-05-06 2015-09-22 Toyota Motor Engineering & Manufacturing North America, Inc. Automated driving safety system
US10185997B1 (en) * 2014-05-20 2019-01-22 State Farm Mutual Automobile Insurance Company Accident fault determination for autonomous vehicles
JP6289284B2 (ja) * 2014-06-20 2018-03-07 ルネサスエレクトロニクス株式会社 半導体装置及び制御方法
JP6375754B2 (ja) * 2014-07-25 2018-08-22 アイシン・エィ・ダブリュ株式会社 自動運転支援システム、自動運転支援方法及びコンピュータプログラム
JP6201927B2 (ja) * 2014-08-01 2017-09-27 トヨタ自動車株式会社 車両制御装置
JP6135618B2 (ja) * 2014-08-08 2017-05-31 トヨタ自動車株式会社 車両制御装置
EP3000651B1 (de) * 2014-09-24 2018-02-21 Volvo Car Corporation System und Verfahren zur Sitzretraktion während eines autonomen Fahrmodus
US20160092825A1 (en) * 2014-09-29 2016-03-31 The Boeing Company Predictive fatigue risk management
US10007263B1 (en) * 2014-11-13 2018-06-26 State Farm Mutual Automobile Insurance Company Autonomous vehicle accident and emergency response
US9483059B2 (en) * 2014-11-26 2016-11-01 Toyota Motor Engineering & Manufacturing North America, Inc. Method to gain driver's attention for autonomous vehicle
JP6323318B2 (ja) * 2014-12-12 2018-05-16 ソニー株式会社 車両制御装置および車両制御方法、並びにプログラム
US9402577B2 (en) * 2014-12-23 2016-08-02 Automotive Research & Test Center Driver's fatigue detection system and method
JP6260544B2 (ja) * 2015-01-19 2018-01-17 トヨタ自動車株式会社 自動運転装置
US11107031B2 (en) * 2015-02-18 2021-08-31 Ryder Integrated Logistics, Inc. Vehicle fleet control systems and methods
JP6191633B2 (ja) * 2015-02-20 2017-09-06 トヨタ自動車株式会社 運転支援装置
TWI565611B (zh) * 2015-03-26 2017-01-11 神達電腦股份有限公司 行車安全檢測方法、系統、行車電腦及智慧型穿戴裝置
US9884631B2 (en) * 2015-06-04 2018-02-06 Toyota Motor Engineering & Manufacturing North America, Inc. Transitioning between operational modes of an autonomous vehicle
US9802638B1 (en) * 2015-06-19 2017-10-31 Waymo Llc Removable manual controls for an autonomous vehicle
EP3109119B1 (de) * 2015-06-23 2022-08-10 Volvo Car Corporation Verfahren und anordnung zur ermöglichung von sekundären aufgaben beim halbautomatischen fahren
US9493118B1 (en) * 2015-06-24 2016-11-15 Delphi Technologies, Inc. Cognitive driver assist with variable warning for automated vehicles
JP6381826B2 (ja) * 2015-10-30 2018-08-29 三菱電機株式会社 車両情報表示制御装置
KR20170054186A (ko) * 2015-11-09 2017-05-17 현대자동차주식회사 자율주행차량 제어 장치 및 그 방법
JP6641916B2 (ja) * 2015-11-20 2020-02-05 オムロン株式会社 自動運転支援装置、自動運転支援システム、自動運転支援方法および自動運転支援プログラム
DE102015224555A1 (de) * 2015-12-08 2017-06-08 Robert Bosch Gmbh Verfahren zum Betreiben eines Fahrzeugs
KR102464926B1 (ko) * 2016-02-11 2022-11-08 삼성전자주식회사 운송 기기를 제어하는 제어 장치 및 이의 운송 기기 제어 방법
US20170247040A1 (en) * 2016-02-25 2017-08-31 Ford Global Technologies, Llc Autonomous vehicle control transitioning
US9828001B2 (en) * 2016-04-10 2017-11-28 Toyota Motor Engineering & Manufacturing North America, Inc. Confidence icons for apprising a driver of confidence in an autonomous operation of a vehicle
US10430745B2 (en) * 2016-05-03 2019-10-01 Azuga, Inc. Method and apparatus for evaluating driver performance and determining driver rewards
US10065651B2 (en) * 2016-05-10 2018-09-04 Samsung Electronics Co., Ltd Electronic device and method for determining a state of a driver
US10317900B2 (en) * 2016-05-13 2019-06-11 GM Global Technology Operations LLC Controlling autonomous-vehicle functions and output based on occupant position and attention
EP3465371A4 (de) * 2016-05-31 2019-12-18 Peloton Technology Inc. Zustandsmaschine eines kolonnensteuergeräts
US10059346B2 (en) * 2016-06-07 2018-08-28 Ford Global Technologies, Llc Driver competency during autonomous handoff
US9956963B2 (en) * 2016-06-08 2018-05-01 GM Global Technology Operations LLC Apparatus for assessing, predicting, and responding to driver fatigue and drowsiness levels
US9878657B2 (en) * 2016-06-15 2018-01-30 Denso International America, Inc. Projected laser lines/graphics onto the road for indicating truck platooning/warning to other drivers of presence of truck platoon
JP6778872B2 (ja) * 2016-06-28 2020-11-04 パナソニックＩｐマネジメント株式会社 運転支援装置及び運転支援方法
US10209708B2 (en) * 2016-07-28 2019-02-19 Lytx, Inc. Determining driver engagement with autonomous vehicle
US10710594B2 (en) * 2016-12-28 2020-07-14 Faurecia Automotive Seating, Llc Occupant-status prediction system
US10133270B2 (en) * 2017-03-28 2018-11-20 Toyota Research Institute, Inc. Electronic control units, vehicles, and methods for switching vehicle control from an autonomous driving mode
US10471963B2 (en) * 2017-04-07 2019-11-12 TuSimple System and method for transitioning between an autonomous and manual driving mode based on detection of a drivers capacity to control a vehicle
US10308242B2 (en) * 2017-07-01 2019-06-04 TuSimple System and method for using human driving patterns to detect and correct abnormal driving behaviors of autonomous vehicles
US10948911B2 (en) * 2017-10-31 2021-03-16 Denso International America, Inc. Co-pilot
US10229592B1 (en) * 2017-11-07 2019-03-12 Mohamed Roshdy Elsheemy Method on-board vehicles to predict a plurality of primary signs of driving while impaired or driving while distracted
KR102019568B1 (ko) * 2017-12-19 2019-11-04 현대자동차주식회사 자율주행차량의 스티어링 휠 파지 감지 시스템 및 방법
JP6900915B2 (ja) * 2018-01-24 2021-07-07 トヨタ自動車株式会社 自動運転システム
US11454970B2 (en) * 2018-05-21 2022-09-27 Cummins Inc. Adjustment of autonomous vehicle control authority

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150094899A1 (en) * 2013-10-01 2015-04-02 Volkswagen Ag Method for Driver Assistance System of a Vehicle
JP2015210660A (ja) 2014-04-25 2015-11-24 日産自動車株式会社 情報呈示装置及び情報呈示方法
JP2015230573A (ja) 2014-06-05 2015-12-21 アルパイン株式会社 車両運転支援装置、方法およびプログラム
DE102014011264A1 (de) * 2014-07-28 2016-01-28 GM Global Technology Operations LLC (n. d. Ges. d. Staates Delaware) Verfahren zur Berechnung einer Rückholzeit
DE102014225680A1 (de) * 2014-12-12 2016-06-16 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zur Übernahmeaufforderung einer Fahraufgabe an den Fahrzeugführer eines Fahrzeugs im Rahmen des automatischen Fahrens des Fahrzeugs

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11173912B2 (en) 2018-04-11 2021-11-16 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11772677B2 (en) 2018-04-11 2023-10-03 Hyundai Motor Company Apparatus and method for providing notification of control authority transition in vehicle
EP3552913A3 (de) * 2018-04-11 2020-06-10 Hyundai Motor Company Vorrichtung und verfahren zur steuerung für die ermöglichung eines autonomen systems in einem fahrzeug
US11334067B2 (en) 2018-04-11 2022-05-17 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11077854B2 (en) 2018-04-11 2021-08-03 Hyundai Motor Company Apparatus for controlling lane change of vehicle, system having the same and method thereof
US11084491B2 (en) 2018-04-11 2021-08-10 Hyundai Motor Company Apparatus and method for providing safety strategy in vehicle
US11084490B2 (en) 2018-04-11 2021-08-10 Hyundai Motor Company Apparatus and method for controlling drive of vehicle
US11173910B2 (en) 2018-04-11 2021-11-16 Hyundai Motor Company Lane change controller for vehicle system including the same, and method thereof
US10843710B2 (en) 2018-04-11 2020-11-24 Hyundai Motor Company Apparatus and method for providing notification of control authority transition in vehicle
US10836394B2 (en) 2018-04-11 2020-11-17 Hyundai Motor Company Apparatus and method for lane change control
US11529956B2 (en) 2018-04-11 2022-12-20 Hyundai Motor Company Apparatus and method for controlling driving in vehicle
US11351989B2 (en) 2018-04-11 2022-06-07 Hyundai Motor Company Vehicle driving controller, system including the same, and method thereof
US11541889B2 (en) 2018-04-11 2023-01-03 Hyundai Motor Company Apparatus and method for providing driving path in vehicle
US11550317B2 (en) 2018-04-11 2023-01-10 Hyundai Motor Company Apparatus and method for controlling to enable autonomous system in vehicle
US11548509B2 (en) 2018-04-11 2023-01-10 Hyundai Motor Company Apparatus and method for controlling lane change in vehicle
US11548525B2 (en) 2018-04-11 2023-01-10 Hyundai Motor Company Apparatus and method for providing notification of control authority transition in vehicle
US11597403B2 (en) 2018-04-11 2023-03-07 Hyundai Motor Company Apparatus for displaying driving state of vehicle, system including the same and method thereof
EP4119411A4 (de) * 2020-04-01 2023-07-26 Huawei Technologies Co., Ltd. Verfahren und vorrichtung zum automatischen fahren
CN111994087A (zh) * 2020-09-02 2020-11-27 中国第一汽车股份有限公司 一种辅助驾驶方法、系统、车辆及介质

Also Published As

Publication number Publication date
US20200001714A1 (en) 2020-01-02
US20170368936A1 (en) 2017-12-28
JP2018005343A (ja) 2018-01-11
US11027608B2 (en) 2021-06-08
EP3809227B1 (de) 2022-07-27
EP3264211B1 (de) 2021-01-20
US20210268902A1 (en) 2021-09-02
CN107539318B (zh) 2022-04-05
CN107539318A (zh) 2018-01-05
JP6778872B2 (ja) 2020-11-04
US10449856B2 (en) 2019-10-22
EP3809227A1 (de) 2021-04-21

Similar Documents

Publication Publication Date Title
US11027608B2 (en) Driving assistance apparatus and driving assistance method
US11738757B2 (en) Information processing device, moving apparatus, method, and program
JP7299840B2 (ja) 情報処理装置および情報処理方法
US10786193B2 (en) System and method for assessing arousal level of driver of vehicle that can select manual driving mode or automated driving mode
US11623647B2 (en) Driver and vehicle monitoring feedback system for an autonomous vehicle
WO2020100539A1 (ja) 情報処理装置、移動装置、および方法、並びにプログラム
JP5326521B2 (ja) 覚醒状態判断装置及び覚醒状態判断方法
WO2019208450A1 (ja) 運転支援装置、運転支援方法及びプログラム
JP2019021229A (ja) 車両制御装置および車両制御方法
JP4867215B2 (ja) 生理・心理状態判定装置、生理・心理状態判定方法、リファレンスデータ生成装置、及びリファレンスデータ生成方法。
JP2007122579A (ja) 車両制御装置
US20230054024A1 (en) Information processing apparatus, information processing system, information processing method, and information processing program
WO2020145161A1 (ja) 情報処理装置、移動装置、および方法、並びにプログラム
CN113491519A (zh) 基于情感-认知负荷的数字助理
US11430231B2 (en) Emotion estimation device and emotion estimation method
CN111413961B (zh) 控制装置以及计算机可读存储介质
US11383640B2 (en) Techniques for automatically reducing annoyance levels of drivers when using driver monitoring systems
US11912267B2 (en) Collision avoidance system for vehicle interactions
JP7238193B2 (ja) 車両制御装置および車両制御方法
KR102613180B1 (ko) 차량 및 그 제어방법
JP2021189704A (ja) 制御装置およびプログラム

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180618

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20191104

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20200731

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017031566

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1356961

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210215

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20210120

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1356961

Country of ref document: AT

Kind code of ref document: T

Effective date: 20210120

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210421

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210520

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210420

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210420

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210520

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017031566

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

26N No opposition filed

Effective date: 20211021

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20210530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210530

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210530

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210520

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20170530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210120

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230519

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210120