WO2018168097A1 - Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme permettant de déterminer un état de conduite - Google Patents

Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme permettant de déterminer un état de conduite Download PDF

Info

Publication number
WO2018168097A1
WO2018168097A1 PCT/JP2017/042561 JP2017042561W WO2018168097A1 WO 2018168097 A1 WO2018168097 A1 WO 2018168097A1 JP 2017042561 W JP2017042561 W JP 2017042561W WO 2018168097 A1 WO2018168097 A1 WO 2018168097A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
driving
state determination
operation mode
driver
Prior art date
Application number
PCT/JP2017/042561
Other languages
English (en)
Japanese (ja)
Inventor
智浩 籔内
啓 菅原
岡地　一喜
充恵 鵜野
光司 滝沢
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Publication of WO2018168097A1 publication Critical patent/WO2018168097A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • B60R21/0132—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over responsive to vehicle motion parameters, e.g. to vehicle longitudinal or transversal deceleration or speed value
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T7/00—Brake-action initiating means
    • B60T7/12—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60T—VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL; ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
    • B60T7/00—Brake-action initiating means
    • B60T7/12—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger
    • B60T7/14—Brake-action initiating means for automatic initiation; for initiation not subject to will of driver or passenger operated upon collapse of driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/09—Taking automatic action to avoid collision, e.g. braking and steering
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/12—Limiting control by the driver depending on vehicle state, e.g. interlocking means for the control input for preventing unsafe operation
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems

Definitions

  • the present invention relates to a driving state determination device, a driving state determination method, and a program for driving state determination that determine whether driving is appropriate based on a driver's state.
  • the present invention has been made paying attention to the above circumstances, and intends to provide a driving state determination device, a driving state determination method, and a program for driving state determination that can accurately determine driving suitability. .
  • a first aspect of the present invention includes an information acquisition unit that acquires information related to a steering direction of a vehicle and a gaze direction of a driver of the vehicle, and the steering direction during execution of the manual operation mode. And an operation state determination unit that determines whether or not the manual operation mode is appropriate based on the relationship between the line-of-sight direction and a signal output unit that outputs a signal corresponding to the determination result.
  • the driving state determination unit determines that the manual driving mode is inappropriate when a difference between the steering direction and the line-of-sight direction is larger than a reference angle. It is determined to be.
  • the driving state determination unit is configured such that the time when the difference between the steering direction and the line-of-sight direction is larger than a reference angle is longer than a reference time. It is determined that the manual operation mode is inappropriate.
  • the driving state determination unit determines whether the manual driving mode is appropriate based on a relationship among the steering direction, the line-of-sight direction, and the vehicle speed of the vehicle. Judgment is made.
  • the driving state determination unit is configured such that the difference between the steering direction and the line-of-sight direction is greater than a reference angle and the vehicle speed is higher than a reference speed. It is determined that the manual operation mode is inappropriate.
  • the information acquisition unit relates to information related to object detection and recognition of the object by the driver. Information is acquired, and the driving state determination unit determines that the manual driving mode is inappropriate when the object is detected and the object is unrecognized.
  • the signal output unit is configured to determine the manual operation mode based on the inappropriate determination of the manual operation mode.
  • a control signal for switching to the automatic operation mode is output.
  • the signal output unit sets the automatic operation mode based on the inappropriate determination of the manual operation mode.
  • a control signal for executing and decelerating or stopping the vehicle is outputted.
  • a ninth aspect of the present invention is based on an information acquisition process for acquiring information related to the steering direction of the vehicle and the gaze direction of the driver of the vehicle, and the relationship between the steering direction and the gaze direction during execution of the manual operation mode.
  • An operation state determination method comprising: an operation state determination process for determining suitability of the manual operation mode; and a signal output process for outputting a signal corresponding to the determination result.
  • a tenth aspect of the present invention is a program for operation switching that causes a computer to function as each unit included in the operation state determination device according to any one of the first to eighth aspects.
  • the driving state determination device can determine the suitability of the manual operation mode based on the relationship between the steering direction and the line-of-sight direction during execution of the manual operation mode. For example, it is possible to accurately determine whether driving is appropriate or not, assuming that the state in which the driver's attention is lacking from the relationship between the steering direction and the line-of-sight direction during execution of the manual driving mode is the state in which the manual driving mode is inappropriate. For example, when it is determined that the manual operation mode is in an inappropriate state, a signal corresponding to the determination result indicating that the manual operation mode is inappropriate can be output. It is also possible to switch the mode to the automatic driving mode, and it is possible to improve safety by reducing the occurrence of vehicle contact due to driver's carelessness.
  • the second aspect of the present invention when the difference between the steering direction and the line-of-sight direction is larger than the reference angle, that is, when the steering direction and the line-of-sight direction are deviated by a predetermined amount or more, There is a high possibility of lack of attention.
  • Such a state can be determined that the manual operation mode is inappropriate. For example, it is possible to switch the manual operation mode to the automatic operation mode based on a signal corresponding to the determination result indicating that the manual operation mode is inappropriate. It is possible to prevent the operation mode from switching to the automatic operation mode.
  • the third aspect of the present invention when the difference between the steering direction and the line-of-sight direction is longer than the reference time, that is, the steering direction and the line-of-sight over a period longer than the predetermined time.
  • the direction deviates more than a predetermined amount, there is a high possibility that the driver's attention is lacking.
  • Such a state can be determined that the manual operation mode is inappropriate. For example, it is possible to switch the manual operation mode to the automatic operation mode based on a signal corresponding to the determination result indicating that the manual operation mode is inappropriate, but by setting an appropriate reference angle and reference time, It is possible to prevent the manual operation mode from being switched to the automatic operation mode excessively.
  • the driving state determination device can determine the suitability of the manual driving mode based on the relationship between the steering direction, the line-of-sight direction, and the vehicle speed during execution of the manual driving mode. For example, it is determined with high accuracy whether or not the driver is not aware of a state lacking the driver's attention from the relationship between the steering direction, the line-of-sight direction, and the vehicle speed during execution of the manual driving mode. be able to.
  • the vehicle speed of the vehicle is higher than the predetermined speed and the steering direction
  • the direction of the line of sight is deviated by a predetermined amount or more
  • Such a state can be determined that the manual operation mode is inappropriate.
  • the driving state determination device when the object is detected and the object is unrecognized, that is, even though there is an object that should originally be directed to the line of sight, the driving state determination device When the line of sight is not suitable for the driver, there is a high possibility that the driver's attention is lacking. Such a state can be determined that the manual operation mode is inappropriate.
  • the operation state determination device can output a control signal for switching the manual operation mode to the automatic operation mode based on the inappropriate determination of the manual operation mode. Based on this signal, it is possible to switch the manual operation mode to the automatic operation mode, and it is possible to improve the safety by reducing the occurrence of vehicle contact due to the driver's carelessness.
  • the driving state determination device can output a control signal for decelerating or stopping the vehicle by executing the automatic driving mode based on the inappropriate determination of the manual driving mode. it can. Based on this signal, the automatic driving mode can be executed to decelerate or stop the vehicle. As a result, it is possible to improve the safety by reducing the occurrence of vehicle contact or the like due to the driver's carelessness.
  • the operating state determination method can obtain the same effects as those of the first aspect described above. That is, the driving state determination method can accurately determine driving suitability. It is also possible to improve the safety by reducing the occurrence of vehicle contact due to the driver's carelessness.
  • the program for determining the driving state can obtain the same effects as those of the first to eighth aspects described above. That is, the program for determining the driving state can accurately determine whether the driving is appropriate. It is also possible to improve the safety by reducing the occurrence of vehicle contact due to the driver's carelessness.
  • FIG. 1 is a diagram illustrating an overall configuration of a vehicle including a driving state determination device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing the configuration of the operating state determination device according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing a configuration of the information detection unit according to the embodiment of the present invention.
  • FIG. 4 is a diagram for explaining the difference between the steering direction and the line-of-sight direction based on the traveling direction of the vehicle.
  • FIG. 5 is a diagram for explaining the difference between the object direction and the line-of-sight direction based on the traveling direction of the vehicle.
  • FIG. 6 is a flowchart illustrating an example of a flow from the determination of suitability for manual operation mode to the signal output by the operation state determination device.
  • FIG. 1 is a diagram illustrating an overall configuration of a vehicle including a driving state determination device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing the configuration of the operating state determination device according
  • FIG. 7 is a flowchart showing a first example of manual operation mode suitability determination by the operating state determination device.
  • FIG. 8 is a flowchart showing a second example of determination of suitability for manual operation mode by the operation state determination device.
  • FIG. 9 is a flowchart showing a third example of manual operation mode suitability determination by the operating state determination device.
  • FIG. 10 is a flowchart showing a fourth example of manual operation mode suitability determination by the operating state determination device.
  • FIG. 1 is a diagram showing an overall configuration of a vehicle 1 provided with a driving state determination device 2 according to an embodiment of the present invention.
  • the driving state determination device 2 is mounted on a vehicle 1 such as a passenger car.
  • the configuration of the driving state determination device 2 will be described later.
  • the vehicle 1 may be, for example, any one of an automobile, a bus, a truck, a train, and the like, or a vehicle on which a driver (hereinafter also referred to as a driver) other than these rides.
  • the vehicle 1 includes, as basic equipment, a power unit 3 including a power source and a transmission, a steering device 4 equipped with a steering wheel 5, and further includes a manual operation mode and an automatic operation mode as operation modes. .
  • An engine and / or a motor is used as the power source.
  • the manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example.
  • the manual driving mode for example, an operation mode for driving the vehicle 1 based only on the driving operation of the driver and an operation for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driving operation of the driver. Mode is included.
  • the driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example.
  • the driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included.
  • a driver's accelerator operation for example, operation of an accelerator pedal
  • brake operation for example, operation of a brake pedal
  • manual steering manual operation of steering
  • speed manual speed adjustment
  • Adjustment manual operation is also included.
  • manual steering the vehicle 1 is steered mainly by the driver's operation of the steering wheel 5.
  • the speed of the vehicle 1 is adjusted mainly by the driver's accelerator operation or brake operation.
  • the driving operation support control does not include control for forcibly intervening in the driving operation of the driver to automatically drive the vehicle 1. That is, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle 1 within a preset allowable range, but the driving of the vehicle 1 is forced under a certain condition (for example, lane departure of the vehicle 1). Interventive control is not included.
  • the automatic operation mode is a mode for realizing an operation state in which the vehicle 1 is automatically driven along the road on which the vehicle 1 is traveling, for example.
  • the automatic driving mode includes, for example, a driving state in which the vehicle 1 is automatically driven toward a preset destination without driving by the driver.
  • the automatic driving mode it is not always necessary to automatically control all of the vehicle 1, and the driving state in which the driving operation of the driver is reflected in the traveling of the vehicle 1 within a preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in the traveling of the vehicle 1 under certain conditions, while reflecting the driving operation of the driver in the traveling of the vehicle 1 within a preset allowable range.
  • the vehicle 1 further includes an outside camera 6, a steering sensor 7, an accelerator pedal sensor 8, a brake pedal sensor 9, a GPS receiver 10, a gyro sensor 11, a vehicle speed sensor 12, a navigation device 13, an automatic An operation control device 14, a driver camera 15, and an audio output device 16 are provided.
  • the outside camera 6 is installed at an arbitrary position of the vehicle 1 so that the outside of the vehicle 1 can be photographed. Although one vehicle exterior camera 6 is shown in FIG. 1, the vehicle 1 may include a plurality of vehicle exterior cameras that capture different directions.
  • the outside camera 6 continuously captures the driving environment in the vicinity of the vehicle 1.
  • the vehicle exterior camera 6 is activated in response to the start of operation of the vehicle 1 and continuously captures the outside of the vehicle 1.
  • the vehicle exterior camera 6 outputs the captured image (hereinafter also referred to as vehicle exterior image data) to the driving state determination device 2 and the automatic driving control device 14.
  • the steering sensor 7 detects a steering angle.
  • the steering sensor 7 outputs the detection result to the driving state determination device 2 and the automatic driving control device 14.
  • the accelerator pedal sensor 8 detects an operation amount of the accelerator pedal.
  • the accelerator pedal sensor 8 outputs the detection result to the driving state determination device 2 and the automatic driving control device 14.
  • the brake pedal sensor 9 detects the operation amount of the brake pedal.
  • the brake pedal sensor 9 outputs the detection result to the driving state determination device 2 and the automatic driving control device 14.
  • the GPS receiver 10 receives current position information of the vehicle 1.
  • the GPS receiver 10 outputs the current position information to the navigation device 13 and the automatic driving control device 14.
  • the gyro sensor 11 detects the behavior of the vehicle 1.
  • the gyro sensor 11 outputs the detection result to the navigation device 13 and the automatic operation control device 14.
  • the vehicle speed sensor 12 detects the speed of the vehicle 1.
  • the vehicle speed sensor 12 outputs the detection result to the navigation device 13 and the automatic driving control device 14.
  • the navigation device 13 is an example of a video display device that includes a display 131 that displays video.
  • the navigation device 13 stores map information.
  • the navigation device 13 extracts route information from the current position to the destination by using information on the destination input by the driver or the like, map information, and current position information from the GPS receiver 10.
  • the navigation device 13 displays route information on the display 131.
  • the navigation device 13 can also display information other than route information on the display 131.
  • the navigation device 13 outputs the route information to the driving state determination device 2 and the automatic driving control device 14.
  • the navigation device 13 extracts the vehicle traveling state using the current position information from the GPS receiver 10, the detection result from the gyro sensor 11, and the detection result from the vehicle speed sensor 12.
  • the vehicle traveling state includes information such as a current position, a traveling direction, a traveling speed, a traveling distance, and a traveling time.
  • the navigation device 13 displays the traveling state of the vehicle on the display 131.
  • the navigation device 13 outputs the vehicle running state to the driving state determination device 2 and the automatic driving control device 14.
  • the route information described above may include not only information on the route from the current position to the destination, but also information on the road environment from the current position to the destination. Some examples of information about the road environment will be described.
  • the information on the road environment may include information on the type of road that passes from the current position to the destination.
  • the types of roads are classified into, for example, roads in which human traffic is restricted or roads in which human traffic is not restricted.
  • the road where the passage of people is restricted is, for example, an expressway.
  • the highway can also be called a motorway.
  • the road where the passage of people is not restricted is, for example, a general road.
  • the information on the road environment may include information on the speed limit of the road that passes from the current position to the destination.
  • the information regarding the road environment may include position information of an installation on the road passing from the current position to the destination.
  • the installed object is, for example, a sign, but may be an object installed on the road.
  • the information regarding the road environment may include position information of a building near the road passing from the current position to the destination. Note that the route information may include information other than the above example as information on the road environment.
  • the first road and the second road are as follows.
  • the second road is, for example, a road environment in which the driver needs more concentration and attention than the first road for driving the vehicle 1.
  • the second road may be a road having a different road environment from the first road as described above, and is not limited to this example.
  • the second road becomes a road in a road environment in which the driver needs more concentration and attention to drive the vehicle 1 than the first road
  • the present invention is not limited thereto. It is not a thing.
  • the first road is a highway and the second road is a general road. On highways, there is usually no human jump out. On the other hand, people can jump out on ordinary roads.
  • the highway is a road having a straight section longer than a general road. Therefore, a general road requires more concentration and attention to drive the vehicle 1 than a highway, regardless of the driving mode.
  • the first road is a road section including an intersection where a signal is installed among general roads
  • the second road is a road section including an intersection where no signal is installed among general roads.
  • the configuration of the automatic operation control device 14 will be described.
  • the automatic operation control device 14 automatically controls traveling of the vehicle 1 when the operation mode is the automatic operation mode.
  • the automatic operation control device 14 sets the operation mode to the automatic operation mode based on various information or various instructions (including control signals) to automatically control the traveling of the vehicle 1 or changes the operation mode from the automatic operation mode to the manual operation mode.
  • the automatic control of traveling of the vehicle 1 is stopped by switching.
  • the navigation device 13 receives an instruction input of the automatic driving mode from the driver, instructs the automatic driving control device 14 to the automatic driving mode, the automatic driving control device 14 sets the driving mode to the automatic driving mode, and the vehicle 1 run is automatically controlled.
  • the navigation device 13 receives the setting information of the automatic driving section corresponding to the stored map information, and detects that the vehicle 1 has entered the automatic driving section based on the current position information from the GPS receiver 10.
  • the automatic operation mode is instructed, and the automatic operation control device 14 sets the operation mode to the automatic operation mode, and automatically controls the traveling of the vehicle 1.
  • the automatic driving control device 14 includes an outside image data from the outside camera 6, a detection result from the steering sensor 7, a detection result from the accelerator pedal sensor 8, a detection result from the brake pedal sensor 9, and a GPS receiver 10. Current position information, a detection result from the gyro sensor 11, a detection result from the vehicle speed sensor 12, and route information from the navigation device 13 are acquired. For example, the automatic driving control device 14 automatically controls the travel of the vehicle 1 based on these information and traffic information acquired by road-to-vehicle communication.
  • Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed).
  • Automatic steering is an operating state in which the steering device 4 is automatically controlled.
  • Automatic steering includes LKAS (Lane Keeping Assist System).
  • the LKAS automatically controls the steering device 4 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation.
  • Even when LKAS is being executed, the driver's steering operation may be reflected in the steering of the vehicle within a range (allowable range) in which the vehicle 1 does not deviate from the traveling lane.
  • automatic steering is not limited to LKAS.
  • Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled.
  • Automatic speed adjustment includes ACC (Adaptive Cruise Control). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle.
  • the automatic operation control device 14 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed.
  • the automatic driving control device 14 can operate the driver's accelerator operation (for example, accelerator) up to a preset maximum permissible speed (for example, the maximum speed legally determined on the traveling road) even when ACC is being executed.
  • the vehicle 1 can be accelerated according to the pedal operation.
  • the automatic speed adjustment is not limited to ACC but also includes CC (Cruise Control).
  • the configuration of the driver camera 15 will be described.
  • the driver camera 15 is installed, for example, at a position that is in front of the driver as on a dashboard.
  • the driver camera 15 is an example of a sensor that monitors a driver.
  • the driver camera 15 is activated in response to the start of driving of the vehicle 1 and continuously captures a predetermined range including the driver's face.
  • the driver camera 15 outputs the captured image (hereinafter referred to as driver image data) to the driving state determination device 2.
  • Driver image data is an example of monitoring data used to detect the state of the driver.
  • the driver's condition includes, for example, driver's forward gaze, drowsiness, sleep, looking aside, putting on and taking off clothes, telephone operation, leaning on the window / armrest, driving disturbance by passengers and pets, onset of illness, backward, prone And at least one of indicators such as eating, drinking, smoking, dizziness, abnormal behavior, car navigation / audio operation, wearing / removing glasses / sunglasses, photography and visual recognition.
  • the visual recognition is an index of how much the driver visually recognizes the object.
  • the state of the driver may include an index other than the index exemplified here.
  • the audio output device 16 includes a speaker 161.
  • the audio output device 16 outputs various information as audio.
  • a configuration of the above-described operation state determination device 2 will be described.
  • the driving state determination device 2 acquires information on the state of the vehicle and the state of the driver, detects the state of the vehicle and the state of the driver, determines propriety of driving based on the state of the vehicle and the state of the driver, and determines A signal corresponding to the result is output.
  • the driving state determination device 2 estimates the driving concentration level of the driver based on the above-described driver state, and determines whether the driver is in a state suitable for driving the vehicle 1 based on the driving concentration level. You may do it.
  • the driving concentration degree is a degree suitable for the driver to drive the vehicle 1. As the driving concentration increases, the driver becomes more suitable for driving the vehicle 1. Conversely, as the degree of driving concentration decreases, the driver becomes more unsuitable for driving the vehicle 1.
  • FIG. 2 is a block diagram illustrating a configuration of the driving state determination device 2 as an example.
  • the operating state determination device 2 includes an input / output interface unit 21, a storage unit 22, and a control unit 23.
  • the input / output interface unit 21 includes an outside camera 6, a steering sensor 7, an accelerator pedal sensor 8, a brake pedal sensor 9, a GPS receiver 10, a navigation device 13, an automatic driving control device 14, a driver camera 15, and a voice output device.
  • Each of 16 is connected to the control unit 23.
  • the configuration of the storage unit 22 will be described.
  • the storage unit 22 is a non-volatile memory that can be written and read at any time, such as a solid state drive (SSD) and a hard disk drive (HDD).
  • the storage unit 22 includes a monitoring data storage unit 221 and a vehicle exterior image data storage unit 222.
  • the monitoring data storage unit 221 stores driver image data that the control unit 23 acquires from the driver camera 15.
  • the vehicle exterior image data storage unit 222 stores vehicle exterior image data that the control unit 23 acquires from the vehicle exterior camera 6.
  • the configuration of the control unit 23 will be described.
  • the control unit 23 includes a processor 231 and a memory 232.
  • the processor 231 is, for example, a CPU (Central Processing Unit) that constitutes a computer.
  • the configuration of each unit included in the processor 231 will be described later.
  • the memory 232 is a non-volatile memory or a RAM (Random-Access Memory).
  • the memory 232 includes a program that causes the processor 231 to perform processing of each unit included in the processor 231.
  • the program can also be referred to as an instruction for operating the processor 231.
  • the program is stored in the storage unit 22 and is read from the storage unit 22 to the memory 232.
  • the program in the memory 232 is read by the processor 231.
  • One embodiment may be realized by a program.
  • the processor 231 includes a monitoring data acquisition unit 2311, a vehicle exterior image data acquisition unit 2312, a vehicle information acquisition unit 2313, a route information acquisition unit 2314, a current position information acquisition unit 2315, an information detection unit 2316, and a signal output unit 2318. With. Each unit may be distributed among one or more processors.
  • the monitoring data acquisition unit 2311 acquires driver image data from the driver camera 15 via the input / output interface unit 21.
  • the monitoring data acquisition unit 2311 stores the driver image data in the monitoring data storage unit 221.
  • the vehicle exterior image data acquisition unit 2312 acquires vehicle exterior image data from the vehicle exterior camera 6 via the input / output interface unit 21.
  • the vehicle exterior image data acquisition unit 2312 stores the vehicle exterior image data in the vehicle exterior image data storage unit 222.
  • Vehicle information acquisition unit 2313 acquires information related to vehicle operation. For example, the vehicle information acquisition unit 2313 acquires information regarding the steering angle from the steering sensor 7, information regarding the operation amount of the accelerator pedal from the accelerator pedal sensor 8, and information regarding the operation amount of the brake pedal from the brake pedal sensor 9. The vehicle information acquisition unit 2313 acquires information related to the steering direction based on information related to the steering angle from the steering sensor 7. Furthermore, the vehicle information acquisition unit 2313 may acquire information on the current position, traveling direction, traveling speed, traveling distance, traveling time, and the like as the vehicle traveling state from the navigation device 13. In addition, the vehicle information acquisition unit 2313 may acquire information on the behavior of the vehicle 1 from the gyro sensor 11 and information on the speed of the vehicle 1 from the vehicle speed sensor 12. The vehicle information acquisition unit 2313 outputs the acquired information to the information detection unit 2316.
  • the route information acquisition unit 2314 acquires route information from the navigation device 13 via the input / output interface unit 21.
  • the route information acquisition unit 2314 outputs the route information to the information detection unit 2316.
  • the current position information acquisition unit 2315 acquires current position information from the GPS receiver 10 via the input / output interface unit 21.
  • the current position information acquisition unit 2315 outputs the current position information to the information detection unit 2316.
  • the information detection unit 2316 detects information related to the state of the vehicle based on the information acquired by the vehicle information acquisition unit 2313. For example, the information detection unit 2316 detects information related to the steering direction of the vehicle 1, detects the relationship between the steering direction and the elapsed time, and detects the vehicle speed of the vehicle 1. In addition, the information detection unit 2316 detects information related to the state of the driver from the driver image data stored in the monitoring data storage unit 221. For example, the information detection unit 2316 detects information related to the driver's line-of-sight direction and line-of-sight angle as the driver's state. The angle between the traveling direction of the vehicle 1 and the line-of-sight direction of the driver of the vehicle 1 is defined as the line-of-sight angle.
  • the information detection unit 2316 uses, for example, at least one of image data outside the vehicle, route information, and current position information, and information related to detection of the object, and the object by the driver. Information on cognition of may be detected.
  • the information regarding the object includes information regarding whether an object that the driver may visually recognize or an object that the driver may visually recognize and that the vehicle 1 may contact may be detected, and It is information about the object angle.
  • An angle between the traveling direction of the vehicle 1 and the object direction from the vehicle 1 to the object is defined as an object angle.
  • the object angle will be described in detail later.
  • the recognition of the object corresponds to the visual recognition described above as the driver's state.
  • the information detection unit 2316 may acquire driver image data from the monitoring data acquisition unit 2311 without going through the monitoring data storage unit 221.
  • the storage unit 22 may not include the monitoring data storage unit 221.
  • the information detection unit 2316 outputs information related to the vehicle state and the driver state to the driving state determination unit 2317.
  • the driving state determination unit 2317 determines whether driving is appropriate based on information from the information detection unit 2316. For example, the driving state determination unit 2317 determines whether the manual driving mode is appropriate based on the relationship between the steering direction and the line-of-sight direction. The driving state determination unit 2317 may determine whether or not the manual driving mode is appropriate based on the relationship between the steering direction, the line-of-sight direction, and the vehicle speed of the vehicle. In addition, the driving state determination unit 2317 includes information related to detection of an object that the driver may visually recognize or an object that the driver may visually recognize and that the vehicle 1 may contact. Whether or not the manual driving mode is appropriate may be determined based on information regarding recognition of the object by the person. The appropriateness determination of the manual operation mode by the operation state determination unit 2317 will be described in detail later.
  • the driving state determination unit 2317 may estimate the driving concentration level of the driver based on the driver's state detected by the information detection unit 2316. Since the driver's state is detected from the driver image data as described above, it can be said that the driving state determination unit 2317 estimates the driver's degree of driving concentration from the driver image data.
  • the driving state determination unit 2317 estimates a driving concentration level corresponding to each of one or more indexes included in the driver's state. For example, the driving state determination unit 2317 estimates a driving concentration level using sleepiness as an index, and also estimates a driving concentration level using a side look as an index. Note that the driving state determination unit 2317 may estimate, for example, one driving concentration degree by comprehensively determining a plurality of indexes included in the driver's state.
  • the driving state determination unit 2317 can estimate the driving concentration degree by a numerical value such as a ratio.
  • the numerical value estimated by the driving state determination unit 2317 may be increased as the driving concentration level is increased, or may be decreased as the driving concentration level is increased.
  • the estimation of the driving concentration by the driving state determination unit 2317 may be performed using an AI (Artificial Intelligence) function such as machine learning or deep learning.
  • AI Artificial Intelligence
  • the driving state determination unit 2317 can estimate the driver's state with high accuracy by using the past estimation result for estimation of the current driving concentration degree, for example.
  • the driving state determination unit 2317 compares the driving concentration degree with a reference.
  • the driving concentration degree for each of the plurality of indices may be compared with a reference.
  • the information detection unit 2316 outputs the comparison result to the signal output unit 2318. Note that the reference may be arbitrarily changed.
  • the signal output unit 2318 outputs a signal to each unit via the input / output interface unit 21.
  • the signal output unit 2318 outputs a signal according to the determination result of driving suitability to the automatic driving control device 14.
  • the automatic driving control device 14 may control driving based on a signal corresponding to the determination result of driving suitability. The output of the signal according to the determination result of driving suitability and the operation control based on the signal according to the determination result will be described in detail later.
  • the signal output unit 2318 may output a signal corresponding to the determination result of driving suitability to at least one of the navigation device 13 and the audio output device 16. For example, the signal output unit 2318 outputs a warning signal according to the inappropriate driving determination.
  • the navigation device 13 displays a message for giving a warning to the driver on the display 131 based on the warning signal.
  • the voice output device 16 outputs a message for giving attention to the driver from the speaker 161 based on the warning signal.
  • the signal output unit 2318 outputs a first warning signal according to the inappropriate driving determination during execution of the automatic operation mode, and the second according to the inappropriate driving determination during execution of the manual operation mode.
  • a warning signal may be output.
  • At least one of the navigation device 13 and the voice output device 16 outputs a message such as “switching to manual operation mode is not accepted because the state is inappropriate for driving” based on the first warning signal. To do.
  • at least one of the navigation device 13 and the audio output device 16 is based on the second warning signal “switching the manual operation mode to the automatic operation mode because it is inappropriate for driving”, or “ “The vehicle is stopped (or decelerated) because it is inappropriate for driving.” Is output.
  • the driver can recognize from the message that the driver is not in a state suitable for driving the vehicle 1.
  • the signal output unit 2318 may output a warning signal to a device other than the navigation device 13 and the audio output device 16.
  • the signal output unit 2318 may output a warning signal to a device that gives an external stimulus such as vibration to the driver.
  • the signal output unit 2318 may output a control signal for switching the manual operation mode to the automatic operation mode to the automatic operation control device 14 based on the inappropriate determination of the manual operation mode.
  • the signal output unit 2318 may output a control signal for decelerating or stopping the vehicle by executing the automatic operation mode based on the inappropriate determination of the manual operation mode.
  • the signal output unit 2318 may determine whether or not to output a warning signal for giving attention to the driver to the external device based on the comparison result from the driving state determination unit 2317. For example, the signal output unit 2318 may output a warning signal to the driver when the driving concentration estimated by the driving state determination unit 2317 is lower than the reference. The signal output unit 2318 outputs a warning signal to at least one of the navigation device 13 and the audio output device 16, for example. The signal output unit 2318 can output a warning signal when one or more driving concentration degrees among a plurality of driving concentration degrees estimated from a plurality of indices are lower than a reference. The signal output unit 2318 may output a warning signal when a predetermined number or more of the driving concentration degrees estimated from the plurality of indices is lower than the reference.
  • FIG. 3 is a block diagram illustrating a configuration of the information detection unit 2316.
  • the information detection unit 2316 includes a local information detection unit 23161, a global information detection unit 23162, and a driver information detection unit 23163.
  • the local information detection unit 23161 detects the state of at least one of the organs included in the driver's face in the driver image data.
  • the organs included in the face are, for example, the eyes, mouth, nose and ears, but other organs may be used.
  • the local information detection unit 23161 detects the eye state
  • the local information detection unit 23161 detects, for example, the eye open / closed degree, the line-of-sight direction, the face direction, and the like of the driver.
  • Local information detection unit 23161 outputs a detection result (hereinafter also referred to as local information) to driver information detection unit 23163.
  • the global information detection unit 23162 detects at least one of the global states of the driver in the driver image data.
  • the global state is, for example, the movement and posture of the driver, but may be other than these.
  • the global information detection unit 23162 outputs a detection result (hereinafter also referred to as global information) to the driver information detection unit 23163.
  • the driver information detection unit 23163 detects the above-described driver state using the local information from the local information detection unit 23161 and the global information from the global information detection unit 23162.
  • the information detection unit 2316 can detect various driver states by combining local information and global information, for example.
  • the information detection unit 2316 can detect the traveling direction of the vehicle based on the traveling information of the vehicle 1 and can detect the driver's line-of-sight direction and line-of-sight angle based on the traveling direction of the vehicle.
  • the information detection unit 2316 can detect visual recognition using monitoring data and target position information.
  • the information detection unit 2316 can detect visual recognition using the image data outside the vehicle in addition to the driver image data as follows.
  • the information detection unit 2316 extracts a target for detecting visual recognition from the image data outside the vehicle.
  • the information detection unit 2316 divides the image data outside the vehicle into a plurality of areas, and extracts a target from the divided plurality of areas. Thereby, the position (direction) of the object can be detected.
  • the target is, for example, an installation such as a sign and a guardrail, and a building, but is not particularly limited as long as the driver can visually recognize the object.
  • the information detection unit 2316 can extract the extracted object from the pattern matching between the extracted object and the registered dangerous object. It may be determined whether or not is a dangerous article.
  • the information detection unit 2316 detects the driver's line of sight and the face direction from the driver image data captured at substantially the same timing as when the image data outside the vehicle from which the target was extracted was captured.
  • the driver's line of sight and face orientation are detected by the local information detector 23161 as described above.
  • the information detection unit 2316 detects visual recognition using at least one of the driver's line of sight and face orientation and target position information. For example, you may make it detect visual recognition, when the time when at least one of a driver
  • the information detection unit 2316 can detect visual recognition using the route information and the current position information in addition to the driver image data as follows.
  • the information detection unit 2316 extracts a target located near the vehicle 1 with reference to the route information and the current position information.
  • the target is, for example, an installation such as a sign or a building, but is not particularly limited as long as the driver can visually recognize it.
  • the information detection unit 2316 detects the driver's line of sight and the direction of the face from the driver image data captured at approximately the same timing as the vehicle 1 passes through the vicinity of the target.
  • the information detection unit 2316 detects visual recognition using at least one of the driver's line of sight and face orientation and target position information.
  • the information detection unit 2316 may obtain the target position and the timing at which the vehicle 1 passes through the vicinity of the target through road-to-vehicle communication. In this case, the information detection unit 2316 detects the driver's line of sight and the direction of the face from the driver image data captured at substantially the same timing as the vehicle 1 passes through the vicinity of the target. The information detection unit 2316 detects visual recognition using at least one of the driver's line of sight and face orientation and target position information.
  • the information detection unit 2316 may use a message displayed on the display 131 of the navigation device 13 as a target. In this case, the information detection unit 2316 detects the driver's line of sight and the face direction from the driver image data captured at substantially the same timing as when the message is displayed on the display 131. The information detection unit 2316 detects visual recognition using at least one of the driver's line of sight and face orientation and target position information.
  • the information detection unit 2316 can appropriately detect the driver's state using visual recognition as an index.
  • the information detection unit 2316 may use an object located in the vicinity of the front, rear, left, or right of the vehicle 1.
  • the information detection unit 2316 preferably uses an object located near the left side or the right side rather than the front side of the vehicle 1. If the object is located on the front side of the vehicle 1, the driver's line of sight and face do not move much. On the other hand, if the target is located near the left side or the right side of the vehicle 1, the driver's line of sight and face move to the left or right side. Therefore, the information detection unit 2316 can appropriately detect visual recognition.
  • the steering direction, steering angle, line-of-sight direction, line-of-sight angle, object direction, object angle, etc. will be described.
  • the range of the steering direction is from + X degrees to -X degrees with the traveling direction of the vehicle 1 as a reference (0 degrees).
  • the range of the line-of-sight direction is + Y degrees to -Y degrees with reference to the traveling direction of the vehicle 1.
  • FIG. 4 is a diagram for explaining the difference between the steering direction and the line-of-sight direction.
  • the angle between the traveling direction of the vehicle 1 and the steering direction is the steering angle
  • the angle between the traveling direction of the vehicle 1 and the line-of-sight direction of the driver of the vehicle 1 is the line-of-sight angle.
  • the angle between the steering direction and the line-of-sight direction is a difference
  • the difference is also the sum of the steering angle and the line-of-sight angle.
  • the driving state determination unit 2317 determines that the driver is looking aside when the difference is larger than the reference angle, and can determine that the manual driving mode is inappropriate based on this.
  • FIG. 5 is a diagram for explaining the difference between the object direction and the line-of-sight direction.
  • the angle between the traveling direction of the vehicle 1 and the object direction from the vehicle 1 to the object is the object angle
  • the angle between the traveling direction of the vehicle 1 and the line of sight of the driver of the vehicle 1 is the line of sight. It is a horn.
  • the angle between the object direction and the line-of-sight direction is a difference
  • the difference is also the sum of the object angle and the line-of-sight angle.
  • the driving state determination unit 2317 determines that the object is unrecognized when the difference is larger than the reference angle, and can determine that the manual driving mode is inappropriate based on this.
  • FIG. 6 is a flowchart showing an example of a flow from manual operation mode suitability determination to signal output by the driving state determination device.
  • the vehicle information acquisition unit 2313 and the information detection unit 2316 acquire information such as the steering direction of the vehicle 1 (step S11). It should be noted that the monitoring data acquisition unit 2311, the outside image data acquisition unit 2312, and the information detection unit 2316 depend on the state of the driver of the vehicle 1 (the driver's Information on the detection of the object, the recognition of the object, the traveling of the vehicle, and the like.
  • step S12 determines that the automatic driving mode is being executed (step S12, NO)
  • the operation state determination unit 2317 determines whether the manual operation mode is appropriate based on the acquired information (step S13). For example, the driving state determination unit 2317 determines that the manual driving mode is inappropriate or appropriate based on the acquired information. Note that whether or not the manual operation mode is appropriate will be described in detail later.
  • the signal output unit 2318 outputs a signal according to the suitability determination of the manual operation mode by the operation state determination unit 2317 to each unit (step S14). For example, the signal output unit 2318 outputs a signal indicating that the manual operation mode by the driving state determination unit 2317 is appropriate or a signal indicating that it is inappropriate to the navigation device 13 or the automatic operation control device 14.
  • the automatic operation control device 14 may switch the manual operation mode to the automatic operation mode based on a signal indicating that the manual operation mode is inappropriate. Further, the automatic driving control device 14 may execute the automatic driving mode based on a signal indicating that the manual driving mode is inappropriate to decelerate or stop the vehicle.
  • the signal output unit 2318 may output a control signal for switching the manual operation mode to the automatic operation mode based on the inappropriate determination of the manual operation mode to the automatic operation control device 14.
  • the automatic operation control device 14 switches the manual operation mode to the automatic operation mode based on this control signal.
  • the signal output unit 2318 may output a control signal for executing the automatic driving mode based on the inappropriate determination of the manual driving mode to decelerate or stop the vehicle to the automatic driving control device 14.
  • the automatic driving control device 14 executes the automatic driving mode based on this control signal to decelerate or stop the vehicle.
  • FIG. 7 is a flowchart showing a first example of determination of suitability for manual operation mode by the automatic operation control device 14.
  • the monitoring data acquisition unit 2311, the vehicle information acquisition unit 2313, and the information detection unit 2316 acquire information regarding the steering direction and the line-of-sight direction.
  • the driving state determination unit 2317 determines whether the manual driving mode is appropriate based on the relationship between the steering direction and the line-of-sight direction. For example, when the difference (difference angle) between the steering direction and the line-of-sight direction is larger than the reference angle (step S1311, YES), the driving state determination unit 2317 determines that the manual driving mode is inappropriate (step S1312).
  • the steering direction and the line-of-sight direction are deviated by a predetermined amount or more, it can be determined that the possibility of the side-view driving is high and the manual operation mode is inappropriate. If the steering direction is right and the line-of-sight direction is left, and the steering direction and the line-of-sight direction are inconsistent, it can be determined that the manual operation mode is inappropriate.
  • the driving state determination unit 2317 may determine that the manual driving mode is appropriate when the difference between the steering direction and the line-of-sight direction is equal to or smaller than the reference angle (step S1311, NO).
  • any angle can be set as the reference angle.
  • the determination standard becomes strict
  • a wide angle for example, 50 degrees or more
  • FIG. 8 is a flowchart showing a second example of determination of suitability for manual operation mode by the automatic operation control device 14.
  • the monitoring data acquisition unit 2311, the vehicle information acquisition unit 2313, and the information detection unit 2316 acquire information regarding the steering direction and the line-of-sight direction.
  • the information regarding the steering direction and the line-of-sight direction is information indicating the steering direction and the line-of-sight direction corresponding to the time change. That is, it is possible to determine whether or not the difference between the steering direction and the line-of-sight direction is larger than the reference angle and the state is longer than the reference time from the information regarding the steering direction and the line-of-sight direction.
  • the driving state determination unit 2317 When the difference between the steering direction and the line-of-sight direction is larger than the reference angle (step S1321, YES) and the state is longer than the reference time (step S1322, YES), the driving state determination unit 2317 has an inappropriate manual driving mode. Is determined (step S1323). That is, it is determined that the manual operation mode is inappropriate when the steering direction and the line-of-sight direction are displaced by a predetermined amount or more over a longer time than the predetermined time.
  • the driving state determination unit 2317 has an appropriate manual driving mode. It may be determined that That is, if the steering direction and the line-of-sight direction are not deviated by a predetermined amount or more, and if the time is not more than a predetermined time even if they deviate by a predetermined amount or more, it is determined that the manual operation mode is appropriate. Good.
  • An arbitrary time can be set as the reference time. For example, if the reference time is shortened (for example, less than 1.5 seconds), the criterion becomes strict, and if it is long (1.5 seconds or more), the criterion becomes loose.
  • FIG. 9 is a flowchart showing a third example of determination of suitability for manual operation mode by the automatic operation control device 14.
  • the monitoring data acquisition unit 2311, the vehicle information acquisition unit 2313, and the information detection unit 2316 acquire information regarding the steering direction, the line-of-sight direction, and the vehicle speed.
  • the driving state determination unit 2317 determines whether or not the manual driving mode is appropriate based on the relationship among the steering direction, the line-of-sight direction, and the vehicle speed of the vehicle. For example, the driving state determination unit 2317 detects that the vehicle 1 is traveling from the information regarding the vehicle speed (step S1331, YES), and the difference between the steering direction and the line-of-sight direction is larger than the reference angle (step S1332, YES).
  • step S1334 When the vehicle speed is faster than the reference speed (step S1333, YES), it is determined that the manual operation mode is inappropriate (step S1334).
  • a predetermined speed for example, not in a state where the vehicle is slowing down at about 1 to 10 km
  • the steering direction and the line-of-sight direction are deviated by a predetermined amount or more
  • the manual operation mode is inappropriate. judge.
  • the driving state determination unit 2317 also determines that the vehicle 1 is not traveling (step S1331, NO), or the difference between the steering direction and the line-of-sight direction is equal to or smaller than the reference angle (step S1332, NO), or the vehicle speed is the reference speed. In the following cases (step S1333, NO), it may be determined that the manual operation mode is appropriate. That is, when the vehicle 1 is stopped, the steering direction and the line-of-sight direction are not deviated by a predetermined amount or more, or when the speed of the vehicle in which a deviation of the predetermined amount or more has occurred is equal to or less than the predetermined speed, the manual operation mode May be determined to be appropriate.
  • FIG. 10 is a flowchart illustrating a fourth example of the manual operation mode suitability determination by the automatic operation control device 14.
  • the monitoring data acquisition unit 2311, the vehicle information acquisition unit 2313, and the information detection unit 2316 acquire information related to object detection and information related to object recognition from the driving state determination device 2. Based on the acquired information, the driving state determination unit 2317 detects an object that the driver may visually recognize or an object that the driver may visually recognize and that the vehicle 1 may contact. If detected (step S1341, YES) and the object is unrecognized (step S1342, YES), it is determined that the manual operation mode is inappropriate (step S1343).
  • running state determination part 2317 may determine with manual driving mode being appropriate, when the object is not detected (step S1341, NO). In addition, even when the object is detected (step S1341, YES), the driving state determination unit 2317 determines that the manual operation mode is appropriate when the object is recognized (step S1342, NO). May be. Examples of an object that the driver may visually recognize, or an object that the driver may visually recognize and that the vehicle 1 may come into contact with include a fallen object on a road, a sign, or a guardrail. There is. It can be determined from the traveling state of the vehicle 1 whether or not there is a possibility of contact with a fallen object, a sign, a guardrail, or the like on the road.
  • the signal output unit 2318 may output a signal indicating that the driving state is inappropriate.
  • the automatic operation control device 14 can determine the suitability of the manual operation mode based on the relationship between the steering direction and the line-of-sight direction during execution of the manual operation mode. For example, it is possible to accurately determine whether driving is appropriate or not, assuming that the state in which the driver's attention is lacking from the relationship between the steering direction and the line-of-sight direction during execution of the manual driving mode is the state in which the manual driving mode is inappropriate.
  • the steering direction does not correspond to the line-of-sight direction
  • the steering direction when the steering direction is right, but when the line of sight is sent to the left, or when the difference between the steering direction and the line-of-sight direction is greater than the reference value Therefore, it is possible to determine that such a state is an inappropriate state of the manual operation mode.
  • a signal corresponding to the determination result indicating that the manual operation mode is inappropriate can be output. It is also possible to switch the mode to the automatic driving mode, and it is possible to improve safety by reducing the occurrence of vehicle contact due to driver's carelessness.
  • the difference between the steering direction and the line-of-sight direction is larger than the reference angle, that is, when the steering direction and the line-of-sight direction are deviated by a predetermined amount or more, there is a high possibility that the driver's attention is lacking.
  • Such a state can be determined that the manual operation mode is inappropriate.
  • the manual operation mode is inappropriate. For example, it is possible to switch the manual operation mode to the automatic operation mode based on a signal corresponding to the determination result indicating that the manual operation mode is inappropriate, but by setting an appropriate reference angle and reference time, For example, it is possible to prevent the manual operation mode from being excessively switched to the automatic operation mode in a case that does not correspond to the side-viewing operation.
  • the suitability of the manual operation mode can be determined based on the relationship between the steering direction, the line-of-sight direction, and the vehicle speed during execution of the manual operation mode. For example, it is determined with high accuracy whether or not the driver is not aware of a state lacking the driver's attention from the relationship between the steering direction, the line-of-sight direction, and the vehicle speed during execution of the manual driving mode. be able to. In intersections, narrow streets, or places with many people, there are cases where you drive slowly while looking around. By using the relationship between the steering direction, the line-of-sight direction, and the vehicle speed as the determination criterion, it is possible to prevent the manual operation mode from being excessively switched to the automatic operation mode in such a case.
  • the difference between the steering direction and the line-of-sight direction is larger than the reference angle and the vehicle speed is faster than the reference speed, that is, if the vehicle speed is faster than a predetermined speed and the steering direction and the line-of-sight direction are different from each other by a predetermined amount.
  • a state can be determined that the manual operation mode is inappropriate. For example, it is possible to switch the manual operation mode to the automatic operation mode based on a signal corresponding to the determination result indicating that the manual operation mode is inappropriate, but by setting an appropriate reference angle and reference speed, It is possible to prevent the manual operation mode from being switched to the automatic operation mode excessively.
  • the manual operation mode may be determined to be inappropriate because it is dangerous.
  • the object is detected in the steering direction by combining the steering direction, the line-of-sight direction, and the object detection information, and the steering direction does not correspond to the line-of-sight direction, it is dangerous and the manual operation mode is inappropriate. You may make it determine. Also, by combining the steering direction, line-of-sight direction, object detection, and speed information, an object is detected in the steering direction, and the steering direction corresponds to the line-of-sight direction. Therefore, the manual operation mode may be determined to be inappropriate.
  • the manual operation mode can be continued when the driver is slowing down while paying attention to the surroundings.
  • the object detection, object recognition, and speed information if the object is detected, the object is not recognized, and the speed is higher than the reference speed, it is dangerous and the manual operation mode is determined to be inappropriate. It may be. When the object is detected, the object is not recognized, and the speed is equal to or lower than the reference speed, it is possible that the risk is low and the manual operation mode is not determined to be inappropriate.
  • the manual operation mode may be determined to be inappropriate because of danger. Further, it may be determined whether or not the manual operation mode is inappropriate by combining a plurality of object detection and speed information. If an object (for example, another vehicle) approaches from the right side of the vehicle, an object (for example, a guard rail) is on the left side of the vehicle, and the speed is higher than the reference speed, it is determined that the manual operation mode is inappropriate because it is dangerous. When an object approaches from the right side of the vehicle and there is an object on the left side of the vehicle and the speed is equal to or lower than the reference speed, it is possible that the risk is low and the manual operation mode may not be determined as inappropriate.
  • the manual operation mode may be determined whether or not the manual operation mode is inappropriate based on the concentration of the driver. For example, when the driver's concentration level is lower than the reference, it is dangerous and the manual driving mode may be determined to be inappropriate. Further, it may be determined whether or not the manual operation mode is inappropriate based on the driver's concentration level and the vehicle speed. For example, when the driver's concentration is lower than the reference and the speed is equal to or higher than the reference speed, the manual operation mode may be determined to be inappropriate because it is dangerous.
  • the manual operation mode may be determined whether or not the manual operation mode is inappropriate based on the degree of concentration and the road environment. For example, the manual driving mode may be determined to be inappropriate because it is dangerous if the second road has low concentration and requires concentration and attention.
  • the driving state determination device 2 can obtain the degree of concentration corresponding to various indexes. For example, it has been described that the degree of concentration using a side look as an index can be obtained. For example, instead of the information on the steering direction and the line-of-sight direction, a degree of concentration using a side look as an index may be used.
  • the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage.
  • various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, you may combine the component covering different embodiment suitably.
  • the above-described embodiment may be realized by a storage medium such as a ROM (Read Only Memory) that stores a program that causes the processors 141 and 231 to function the processes of the units included in the processors 141 and 231.
  • a storage medium such as a ROM (Read Only Memory) that stores a program that causes the processors 141 and 231 to function the processes of the units included in the processors 141 and 231.
  • (Appendix 2) With at least one processor, Obtaining information about the steering direction of the vehicle and the gaze direction of the driver of the vehicle; Determining the suitability of the manual operation mode based on the relationship between the steering direction and the line-of-sight direction during execution of the manual operation mode; An operation state determination method that switches the manual operation mode to an automatic operation mode based on the inappropriate determination of the manual operation mode.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Human Computer Interaction (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)

Abstract

La présente invention peut déterminer, avec une précision élevée, le caractère approprié/inapproprié de la conduite. Le dispositif de détermination d'état de conduite selon l'invention comprend : une unité d'acquisition d'informations qui acquiert des informations concernant l'orientation de direction d'un véhicule et la direction du regard d'un conducteur du véhicule; une unité de détermination d'état de conduite qui détermine, sur la base de la relation entre l'orientation de direction et la direction du regard pendant l'exécution d'un mode de conduite manuelle, si le mode de conduite manuelle est approprié ou non; et une unité d'émission de signal qui émet un signal en fonction du résultat de détermination.
PCT/JP2017/042561 2017-03-14 2017-11-28 Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme permettant de déterminer un état de conduite WO2018168097A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-048144 2017-03-14
JP2017048144A JP6631569B2 (ja) 2017-03-14 2017-03-14 運転状態判定装置、運転状態判定方法及び運転状態判定のためのプログラム

Publications (1)

Publication Number Publication Date
WO2018168097A1 true WO2018168097A1 (fr) 2018-09-20

Family

ID=63522960

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/042561 WO2018168097A1 (fr) 2017-03-14 2017-11-28 Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme permettant de déterminer un état de conduite

Country Status (2)

Country Link
JP (1) JP6631569B2 (fr)
WO (1) WO2018168097A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020066568A1 (fr) * 2018-09-26 2020-04-02 日本電気株式会社 Dispositif d'aide à la conduite, procédé d'aide à la conduite et support d'enregistrement

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3882883A4 (fr) 2018-11-13 2021-12-29 Sony Group Corporation Dispositif de traitement d'informations, procédé de traitement d'informations, et programme
JP2021006967A (ja) * 2019-06-28 2021-01-21 ソニーセミコンダクタソリューションズ株式会社 情報処理装置と情報処理方法および運転支援システム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010066968A (ja) * 2008-09-10 2010-03-25 Toyota Motor Corp 視線状態検出装置
JP2012173803A (ja) * 2011-02-17 2012-09-10 Panasonic Corp 安全運転支援装置及び安全運転支援方法
JP2016151828A (ja) * 2015-02-16 2016-08-22 トヨタ自動車株式会社 衝突回避装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6520668B2 (ja) * 2015-02-09 2019-05-29 株式会社デンソー 車両用表示制御装置及び車両用表示ユニット
JP6686869B2 (ja) * 2016-12-22 2020-04-22 株式会社デンソー 運転交代制御装置、及び運転交代制御方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010066968A (ja) * 2008-09-10 2010-03-25 Toyota Motor Corp 視線状態検出装置
JP2012173803A (ja) * 2011-02-17 2012-09-10 Panasonic Corp 安全運転支援装置及び安全運転支援方法
JP2016151828A (ja) * 2015-02-16 2016-08-22 トヨタ自動車株式会社 衝突回避装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020066568A1 (fr) * 2018-09-26 2020-04-02 日本電気株式会社 Dispositif d'aide à la conduite, procédé d'aide à la conduite et support d'enregistrement
JP2020052643A (ja) * 2018-09-26 2020-04-02 日本電気株式会社 運転支援装置、運転支援方法、プログラム
JP7206750B2 (ja) 2018-09-26 2023-01-18 日本電気株式会社 運転支援装置

Also Published As

Publication number Publication date
JP6631569B2 (ja) 2020-01-15
JP2018151900A (ja) 2018-09-27

Similar Documents

Publication Publication Date Title
CN111361552B (zh) 自动驾驶系统
KR102051142B1 (ko) 차량용 운전자 위험 지수 관리 시스템 및 그 방법
JP2020125111A (ja) 車両制御装置及び車両
CN110192084B (zh) 自动驾驶辅助装置、方法及程序
CN106462554B (zh) 汽车车道秩序系统、方法及设备
US10232772B2 (en) Driver assistance system
JP6990020B2 (ja) 車両の追従発進制御装置
US9908410B2 (en) Method and device for warning the driver of a motor vehicle in the event of lack of attention
WO2018168052A1 (fr) Dispositif de détermination de niveau de concentration, procédé de détermination de niveau de concentration et programme de détermination de niveau de concentration
WO2018168121A1 (fr) Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme pour la détermination d'un état de conduite
WO2018168051A1 (fr) Dispositif de détermination de degré de concentration, procédé de détermination de degré de concentration et programme de détermination de degré de concentration
JP6269360B2 (ja) 運転支援システム及び運転支援方法
US11745745B2 (en) Systems and methods for improving driver attention awareness
US20190243361A1 (en) Drive switching determination apparatus, drive switching determination method, and program for drive switching determination
WO2018168049A1 (fr) Dispositif de détermination de degré de concentration, procédé de détermination de degré de concentration, et programme de détermination de degré de concentration
WO2018168097A1 (fr) Dispositif de détermination d'état de conduite, procédé de détermination d'état de conduite, et programme permettant de déterminer un état de conduite
JP7409265B2 (ja) 車載表示装置、方法およびプログラム
JP2021046182A (ja) 自動運転車
JP6790522B2 (ja) 車両の制御装置
WO2018168099A1 (fr) Dispositif de détermination de degré de concentration, procédé de détermination de degré de concentration et programme pour déterminer le degré de concentration
JP2015106395A (ja) 信号情報提示装置
US20190147264A1 (en) Concentration determination apparatus, concentration determination method, and program for concentration determination
US10766412B1 (en) Systems and methods for notifying other road users of a change in vehicle speed
WO2018168046A1 (fr) Dispositif de détermination de niveau de concentration, procédé de détermination de niveau de concentration, et programme de détermination de niveau de concentration
WO2018168048A1 (fr) Dispositif de détermination de degré de concentration, procédé de détermination de degré de concentration, et programme permettant de déterminer un degré de concentration

Legal Events

Date Code Title Description
DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17900902

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17900902

Country of ref document: EP

Kind code of ref document: A1