WO2018163474A1 - 運転モード切替制御装置、方法及びプログラム - Google Patents

運転モード切替制御装置、方法及びプログラム Download PDF

Info

Publication number
WO2018163474A1
WO2018163474A1 PCT/JP2017/033149 JP2017033149W WO2018163474A1 WO 2018163474 A1 WO2018163474 A1 WO 2018163474A1 JP 2017033149 W JP2017033149 W JP 2017033149W WO 2018163474 A1 WO2018163474 A1 WO 2018163474A1
Authority
WO
WIPO (PCT)
Prior art keywords
preparation period
driver
operation mode
information
unit
Prior art date
Application number
PCT/JP2017/033149
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
初美 青位
相澤　知禎
岡地　一喜
啓 菅原
充恵 鵜野
光司 滝沢
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Priority to DE112017007208.5T priority Critical patent/DE112017007208T5/de
Publication of WO2018163474A1 publication Critical patent/WO2018163474A1/ja

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0053—Handover processes from vehicle to occupant
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0053—Handover processes from vehicle to occupant
    • B60W60/0054—Selection of occupant to assume driving tasks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping

Definitions

  • the present invention relates to an operation mode switching control device, method, and program for switching a vehicle operation mode between a manual operation mode and an automatic operation mode.
  • the automatic driving mode includes, for example, navigation system information using GPS (Global Positioning System), traffic information acquired by road-to-vehicle communication, and information on a surrounding monitoring system that monitors the position and movement of surrounding people and vehicles.
  • GPS Global Positioning System
  • the vehicle can be automatically driven by controlling a power unit, a steering device, a brake, and the like (see, for example, Patent Document 1).
  • the automatic driving mode can be expected to reduce the burden on the driving operation of the driver and alleviate traffic congestion, it is practically difficult to automate all the driving operations from the start to the end of driving. For this reason, it is required to appropriately switch from the automatic operation mode to the manual operation mode.
  • the operation mode from the automatic operation mode to the manual operation mode confirm that the driver is ready for manual operation prior to switching and ask the driver to prepare for manual operation.
  • the preparation period when switching from the automatic operation mode to the manual operation mode is set as a preparation period for one driver to perform manual operation from now on. Therefore, when the above change is made, the two drivers share the preparation period, and the preparation period may not be sufficient for the driver in charge of manual driving.
  • the present invention has been made paying attention to the above circumstances, and ensures a sufficient driver preparation period even when a driver change occurs when switching from the automatic operation mode to the manual operation mode. It is an object of the present invention to provide an operation mode switching control device, method and program capable of performing the above.
  • a first aspect of the present invention is an operation mode switching control device that performs switching control of a vehicle operation mode between a manual operation mode and an automatic operation mode.
  • a first preparation period is set when switching to the manual operation mode, a preparation period control unit that counts the first preparation period, and first preparation period information for informing the remaining time of the first preparation period
  • a preparation period information generation unit that generates the output, an output unit that outputs the first preparation period information, and a determination unit that determines whether or not a driver change has occurred in the first preparation period. It has.
  • the preparation period control unit sets a second preparation period and counts the second preparation period, and the preparation period
  • the information generation unit generates second preparation period information for informing the remaining time of the second preparation period, and the output unit outputs the second preparation period information.
  • the preparation period is set and counted by the preparation period control unit, and the first time for informing the remaining time of the preparation period
  • the preparation period information is generated by the preparation period information generation unit and output from a predetermined output unit.
  • the preparation period control unit sets and counts a new second preparation period and informs the remaining time of the second preparation period.
  • Second preparation period information is generated by the preparation period information generation unit and output from a predetermined output unit. Therefore, it is possible to prevent the situation where two drivers share the limited preparation period when switching from the automatic operation mode to the manual operation mode, and a certain preparation period is always provided for the driver after the change. Can be secured. Thereby, the new driver after the change can make sufficient preparations for the manual driving operation and can perform the manual driving with a psychological margin. As a result, a safe and comfortable driving environment can be provided.
  • video data about a driver acquired by the monitoring camera before the counting unit starts counting the first preparation period and the monitoring camera in the first preparation period are used.
  • the similarity is calculated using the acquired video data about the driver, and the determination as to whether or not a driver change has occurred is executed based on the calculated similarity. is there.
  • the determination unit acquires the video data about the driver acquired by the monitoring camera before starting the counting of the first preparation period and the monitoring camera in the first preparation period.
  • the similarity is calculated using the image data of the driver, and a determination is made as to whether or not a driver change has occurred based on the calculated similarity. Therefore, based on the unique information such as the driver's face, it can be reliably determined whether or not a driver change has occurred before and after the start of the latest preparation period.
  • the determination unit determines whether or not a driver change has occurred in the second preparation period, and the determination unit determines whether or not a driver change has occurred in the second preparation period.
  • the preparation period control unit sets a third preparation period, counts the third preparation period, and the preparation period information generation unit determines the remaining of the third preparation period. Preparation period information for informing time is generated, and the output unit outputs the third preparation period information.
  • a new third preparation period is set and the third preparation period is counted.
  • Preparation period information for informing the remaining time of the new third preparation period is generated, and the third preparation period information is output from a predetermined output unit.
  • video data about a driver acquired by the monitoring camera in the first preparation period and a driver acquired by the monitoring camera in the second preparation period by the determination unit. Similarity is calculated using video data for and the determination as to whether or not a driver change has occurred in the second preparation period is performed based on the calculated similarity. Is.
  • the video data about the driver acquired by the monitoring camera in the first preparation period and the driver acquired by the monitoring camera in the second preparation period by the determination unit.
  • the degree of similarity is calculated using the video data and a determination is made as to whether or not a driver change has occurred based on the calculated degree of similarity. Therefore, based on the unique information such as the driver's face, it can be reliably determined whether or not a driver change has occurred before and after the start of the latest preparation period.
  • an operation mode switching signal output unit that outputs a signal for switching the automatic operation mode to the manual operation mode when the latest preparation period has elapsed. is there.
  • the operation mode switching signal output unit outputs a signal for switching the automatic operation mode to the manual operation mode when the latest preparation period has elapsed. For this reason, it is possible to switch the automatic operation mode to the manual operation mode after reliably providing a certain preparation period to the new driver after the change.
  • the length of the preparation period newly set according to a predetermined standard is adjusted by the preparation period information generation unit.
  • the newly set preparation period can be adjusted to a predetermined length according to the current driver state, and the appropriate preparation period can be set flexibly. Can improve safety.
  • the preparation period information generation unit further generates information notifying that the driver change has been confirmed, and the output unit outputs information notifying that the driver change has been confirmed. It is what you do.
  • information notifying that the change of the driver has been confirmed is generated and output from the predetermined output unit. Accordingly, the driver quickly and easily grasps that the preparation period is newly set and the remaining time of the preparation period, and that the preparation period is newly set is due to the change of the driver. be able to.
  • the operation mode when switching from the automatic operation mode to the manual operation mode, even when a driver change occurs, the operation mode can sufficiently ensure the driver's preparation period.
  • a switching control device, method, and program can be provided.
  • the flowchart which shows the procedure and control content of the preparation period control by the operation mode switching control apparatus shown in FIG. The conceptual diagram for demonstrating the content of the preparation period control by the operation mode switching control apparatus shown in FIG.
  • FIG. 1 is a diagram showing an overall configuration of an automatic driving control system including an operation mode switching control device according to an embodiment of the present invention, and this automatic driving control system is mounted on a vehicle 1 such as a passenger car.
  • the vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, a steering device 3 equipped with a steering wheel 4, and further includes a manual operation mode and an automatic operation mode as operation modes. .
  • An engine and / or a motor is used as the power source.
  • the manual driving mode is a mode in which the vehicle 1 is driven mainly by a driving operation manually performed by a driver (hereinafter also referred to as a driver).
  • the manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.
  • the driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example.
  • the driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation) is also included.
  • a driver's accelerator operation for example, operation of an accelerator pedal
  • brake operation for example, operation of a brake pedal
  • manual steering manual operation of steering
  • speed manual speed adjustment
  • Adjustment manual operation is also included.
  • manual steering the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4.
  • the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.
  • the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle.
  • the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.
  • the automatic operation mode is a mode that realizes an operation state in which the vehicle automatically travels along the road on which the vehicle travels, for example.
  • the automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver.
  • the automatic driving mode it is not always necessary to automatically control all of the vehicle, and the driving state in which the driving operation of the driver is reflected in the driving of the vehicle within the preset allowable range is also included in the automatic driving mode. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.
  • reference numeral 5 denotes an automatic operation control device for executing operation control in the automatic operation mode.
  • the automatic driving control device 5 acquires sensing data from the steering sensor 11, the accelerator pedal sensor 12, the brake pedal sensor 13, the GPS receiver 14, the gyro sensor 15, and the vehicle speed sensor 16, respectively. These sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, information obtained by a peripheral monitoring system that monitors the position and movement of surrounding people and vehicles Based on this, the traveling of the vehicle 1 is automatically controlled.
  • Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed).
  • Automatic steering is an operating state in which the steering device 3 is automatically controlled.
  • Automatic steering includes LKA (Lane Keeping Assist).
  • LKA Li Keeping Assist
  • the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation.
  • the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range).
  • automatic steering is not limited to LKA.
  • Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled.
  • Automatic speed adjustment includes ACC (Adaptive Cruise Control). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle.
  • the automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed.
  • the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC.
  • the vehicle can be accelerated according to the pedal operation.
  • the automatic speed adjustment is not limited to ACC but also includes CC (Cruise Control).
  • the automatic operation control system of this embodiment is an operation mode switching control device 6 and a driver camera 7 as a first monitoring sensor as devices for switching between the manual operation mode and the automatic operation mode. And a torque sensor 8 as a second monitoring sensor and an audio output device 9.
  • the driver camera 7 is installed at a position in front of the driver such as on the dashboard, for example, and images the driver and outputs the video signal to the driving mode switching control device 6.
  • the torque sensor 8 detects torque generated when the driver operates the steering wheel 4, and outputs a detection signal to the operation mode switching control device 6.
  • the audio output device 9 has a speaker and a display, and outputs the audio signal of the message output from the operation mode switching control device 6 from the speaker and displays the display signal of the message on the display.
  • the operation mode switching control device 6 controls the switching of the operation modes as a whole, and is configured as follows.
  • FIG. 2 is a block diagram showing the functional configuration.
  • the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.
  • the input / output interface unit 62 receives the video signal and the torque detection signal output from the driver camera 7 and the torque sensor 8, respectively, and converts them into digital data. At the same time, the message output from the control unit 61 is converted into an audio signal and a display signal and output to the audio output unit 9. Further, the operation mode switching control signal output from the control unit 61 is output to the automatic operation control device 5.
  • the storage unit 63 includes, as a storage medium, a nonvolatile memory such as SSD (Solid State Drive) or HDD (Hard Disk Drive) that can be written and read at any time, and a volatile memory such as RAM (Random Access Memory).
  • a driver monitoring video storage unit 631, a driver state storage unit 632, and a preparation period table storage unit 633 are provided.
  • the driver monitoring video storage unit 631 stores the driver monitoring video acquired over time by the driver monitoring video acquisition control unit 611 in association with time information.
  • the driver state storage unit 632 stores information indicating the determination result generated by the driver state determination unit 612 in association with a time stamp indicating the determination timing.
  • the preparation period table storage unit 633 stores a preparation period table created in advance.
  • This preparation period table defines, for example, preparation periods classified for each driver state, such as “normal state”, “tension state”, and “attentiveness distraction state”.
  • the control unit 61 has a CPU (Central Processing Unit) and a program memory that constitute a computer, and includes a driver monitoring video acquisition control unit 611 and a driver state determination unit 612 as control functions necessary for carrying out the present embodiment.
  • the driver monitoring video acquisition control unit 611 fetches the digital data (driver monitoring video data) of the driver video signal output from the driver camera 7 from the input / output interface unit 62, and stores the fetched driver monitoring video data in the storage unit. 63 stored in the driver monitoring video storage unit 631. Further, the driver monitoring video acquisition control unit 611 creates time series local facial video data and time series global motion video data from the time series driver monitoring video data sequentially output from the driver camera 7.
  • the driver monitoring video storage unit 631 of the storage unit 63 can also be stored.
  • the driver state determination unit 612 reads driver monitoring video data from the driver monitoring video storage unit 631 at a preset time interval. Each time the driver monitoring video data is read, a process is performed to determine whether or not the driver can manually perform a driving operation based on the driver monitoring video data. For example, it is determined whether or not the driver is in a sleep state by checking whether or not the driver has closed his / her eyes using local facial image data. Then, the information indicating the determination result is stored in the driver state storage unit 632 in association with the time stamp indicating the determination timing.
  • the driver state determination unit 612 uses a plurality of feature points that are reflected in local facial image data acquired over time by using a method such as ASM (Active Shape Model) or AAM (Active Appearance Model). (For example, each organ of the driver's face, nose, mouth, etc.) is detected, and their position is calculated.
  • the driver state determination unit 612 calculates a face shape model for local face image data at each time by a three-dimensional fitting process using the calculated positions of the plurality of feature points. Further, the driver state determination unit 612 calculates the similarity between the face shape model corresponding to the reference time and the face shape model corresponding to another time, and compares the obtained similarity with a preset threshold value. Thus, it is determined whether or not a driver change has occurred.
  • ASM Active Shape Model
  • AAM Active Appearance Model
  • the preparation period control unit 613 performs control related to the preparation period that is set for the driver to prepare when switching from the automatic operation mode to the manual operation mode. For example, the preparation period control unit 613 performs a predetermined preparation period when the driver state determination unit 612 determines that the driver can perform a driving operation in the manual operation mode during the operation control period in the automatic operation mode. Set. The preparation period control unit 613 sets a new preparation period when the driver state determination unit 612 determines that a driver change has occurred during the set preparation period.
  • the preparation period information generation unit 614 provides information (preparation period information) for informing the driver of at least how many seconds the remaining preparation period is regarding the preparation period set or reset by the preparation period control unit 613. And output to the audio output unit 9 and the display unit 10 via the input / output interface unit 62.
  • the operation mode switching signal output unit 615 outputs a control signal for switching from the automatic operation mode to the manual operation mode to the automatic operation control device 5 when the preparation period has elapsed.
  • FIG. 3 is a flowchart showing the procedure and control contents of the preparation period control by the operation mode switching control device 6.
  • FIG. 4 is a conceptual diagram for explaining the contents of the preparation period control by the operation mode switching control device 6. The operation of the operation mode switching control device 6 will be described according to FIG. 3 with reference to FIG.
  • the driver camera 7 is activated, and continuously captures a predetermined range including the driver's face and outputs the video signal.
  • the operation mode switching control device 6 first receives the digital data (driver monitoring video data) of the video signal output from the driver camera 7 from the input / output interface unit 62 under the control of the driver monitoring video acquisition control unit 611.
  • the captured driver monitoring video data is stored in the driver monitoring video storage unit 631 of the storage unit 63 (step S12).
  • the imaging of the driver may be intermittently performed at a cycle shorter than a time interval for determining the state of the driver described later.
  • the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. In this way, it is possible to save the storage capacity of the driver monitoring video storage unit 631 by reducing the information amount of the monitoring video data.
  • step S13 driver monitoring video data is read from the driver monitoring video storage unit 631. Then, it is determined from each of the read driver monitoring video data whether or not the driver can manually perform the driving operation (step S14).
  • the driver's eye open state, blinking frequency, or eye movement is detected to recognize the driver's arousal level. Then, by comparing this arousal level with a threshold value, it is determined whether or not the driver can perform a driving operation manually.
  • the driver state determination unit 612 associates the information indicating the determination result with information indicating the determination timing, for example, time stamp information, in the driver state storage unit 632. Store (step S15).
  • the driver state determination unit 612 determines whether or not to switch the current automatic operation mode to the manual operation mode at a predetermined timing (step S16).
  • the timing of this determination is typically tens of seconds before reaching the exit of the expressway when the vehicle is driving on the expressway in the automatic driving mode, and the external sensor of the vehicle is normal due to bad weather etc.
  • the timing at which it is determined that the operation of the automatic operation mode is stopped and the continuation of the automatic operation mode is stopped may be mentioned.
  • step S11 to S16 the process from step S11 to S16 is repeatedly executed without proceeding to step S17.
  • the driver state determination unit 612 determines that the current automatic operation mode is to be switched to the manual operation mode, a control signal for instructing control related to the preparation period associated with switching from the automatic operation mode to the manual operation mode is provided. Then, it is output from the driver state determination unit 612 to the preparation period control unit 613.
  • the preparation period control unit 613 receives a control signal from the driver state determination unit 612, the preparation period control unit 613 sets the preparation period to 60 seconds, for example, and starts countdown (step S17).
  • the preparation period information generation unit 614 generates preparation period information for notifying the driver of at least how many seconds remain in the preparation period, and outputs the preparation period information to the display unit 10 via the input / output interface unit 62 (step S18). .
  • FIG. 5 shows an example of the preparation period information generated and output in step S18 and displayed on the display unit 10.
  • the driver can visually recognize how many seconds from the automatic operation mode to the manual operation mode by the displayed preparation period information. Moreover, you may make it output the warning sound for notifying that the timing of switching to the manual operation mode from the automatic operation mode is near, the audio
  • the driver can recognize from the preparation period information that the switch to the manual operation mode is about to be performed, and thus can start preparation so as to correspond to the manual operation mode.
  • the driver state determination unit 612 changes the driver in the preparation period. Whether or not has occurred is determined (step S19).
  • the driver state determination unit 612 obtains a local facial image acquired at a predetermined reference time (in the example of FIG. 4, the latest time tk before the start of the preparation period set in step S17). Each feature such as eyes, nose and mouth of the driver's face as feature points in the data is detected, and the spatial position of each organ is calculated. In addition, the driver state determination unit 612 operates as a feature point reflected in each local face image data acquired at each time of the preparation period (in the example of FIG. 4, time tk + 1, tk + 2,). Each organ such as eyes, nose and mouth of the person's face is detected and the spatial position of each organ is calculated.
  • a predetermined reference time in the example of FIG. 4, the latest time tk before the start of the preparation period set in step S17.
  • Each feature such as eyes, nose and mouth of the driver's face as feature points in the data is detected, and the spatial position of each organ is calculated.
  • the driver state determination unit 612 calculates a face shape model for local face image data at a predetermined reference time and each time of the preparation period by a three-dimensional fitting process using the calculated positions of a plurality of feature points. To do.
  • the driver state determination unit 612 calculates the similarity between the face shape model corresponding to the reference time and the face shape model corresponding to each time in the preparation period.
  • the similarity calculation is not particularly limited, and for example, a general geometric pattern matching method can be used.
  • the driver state determination unit 612 compares the similarity at each time in the preparation period with a predetermined threshold value. As a result, when the similarity at a predetermined time (in the example of FIG. 4, the time tl ⁇ 1) is equal to or smaller than a predetermined threshold, the driver state determination unit 612 indicates that “the driver change at the predetermined time is It is determined that it has occurred (Y in step S19), and a control signal for notifying the occurrence of a driver change is output to the preparation period control unit 613.
  • the preparation period control unit 613 When the preparation period control unit 613 receives the control signal from the driver state determination unit 612, the preparation period control unit 613 sets a new preparation period as, for example, 60 seconds, and starts a countdown (step S20).
  • the preparation period information generation unit 614 generates information (new preparation period information) for notifying the driver of at least how many seconds the newly set preparation period is remaining, and via the input / output interface unit 62 It outputs to the display part 10 (step S18 which passed through step S20).
  • FIG. 6 shows an example of new preparation period information generated and output in step S18 and displayed on the display unit 10.
  • a message is added regarding the occurrence of a driver change and therefore the establishment of a new preparation period.
  • the driver can quickly and easily visually confirm that the preparation period has been newly secured in accordance with the driver change, in addition to how many seconds the automatic operation mode is switched to the manual operation mode. can do.
  • the driver state determination unit 612 applies local facial image data acquired at a new reference time (in the example of FIG. 4, the latest time tl before the start of a new preparation period).
  • a new reference time in the example of FIG. 4, the latest time tl before the start of a new preparation period.
  • Each feature such as eyes, nose, mouth, etc. of the driver's face as a feature point is detected, and the spatial position of each organ is calculated.
  • the driver state determination unit 612 is a feature point that is reflected in each local face image data acquired at each time of the new preparation period (in the example of FIG. 4, time tl + 1, tl + 2,). The driver's face, eyes, nose, mouth and other organs are detected and their respective spatial positions are calculated.
  • the driver state determination unit 612 performs a face shape model on local face image data at each time of a new reference time and a new preparation period by a three-dimensional fitting process using the calculated positions of a plurality of feature points. Calculate The driver state determination unit 612 calculates the similarity between the face shape model corresponding to the new reference time and the face shape model corresponding to each time of the new preparation period.
  • the driver state determination unit 612 compares the similarity at each time in the new preparation period with a predetermined threshold. As a result, when the similarity at a predetermined time (in the example of FIG. 4, time tm ⁇ 1) is equal to or less than a predetermined threshold, the driver state determination unit 612 indicates that “the driver change at the predetermined time is It is determined that it has occurred "(Y in step S19 via step S20), and a control signal for notifying the occurrence of a driver change is output to the preparation period control unit 613.
  • a predetermined time in the example of FIG. 4, time tm ⁇ 1
  • the driver state determination unit 612 indicates that “the driver change at the predetermined time is It is determined that it has occurred "(Y in step S19 via step S20), and a control signal for notifying the occurrence of a driver change is output to the preparation period control unit 613.
  • the preparation period control unit 613 When the preparation period control unit 613 receives the control signal from the driver state determination unit 612, the preparation period control unit 613 newly sets a new preparation period, for example, as 60 seconds, and starts countdown (step S20).
  • the preparation period information generation unit 614 generates information (further new preparation period information) for informing the driver of at least how many seconds remain in the newly set preparation period. To the display unit 10 (step S18 via step S20).
  • step S19 if the similarity at each time within the preparation period does not fall below the predetermined threshold, the driver state determination unit 612 determines that “no driver change has occurred” (in step S19). N).
  • the preparation period control unit 613 continues to count down the preparation period set in step S17 (step S21).
  • the preparation period control unit 613 determines whether or not the countdown of the latest preparation period has been completed, and if not completed, repeats the processing from step S19 to S22 (N in step S22). Also. If the driver change occurs before the countdown of the latest preparation period is completed, the processing from steps S18 to S20 is repeatedly executed at each change.
  • the operation mode switching control apparatus 6 When receiving the control signal for notifying that the preparation period has elapsed from the preparation period control unit 613, the operation mode switching control apparatus 6 sends a control signal for switching from the automatic operation mode to the manual operation mode to the automatic operation control apparatus 5. Output.
  • the preparation period information generation unit 614 generates information (switching completion information) for notifying that the switching from the automatic operation mode to the manual operation mode is completed as a result of the preparation period having elapsed, and the input / output interface unit 62 Is output to the display unit 10 (step S24).
  • the driver grasps that the switching from the automatic operation mode to the manual operation mode is completed based on the switching completion information, and thereafter, the driving control according to the manual operation of the driver is performed.
  • the preparation period control unit 613 sets and counts the preparation period and informs the remaining time of the preparation period.
  • the preparation period information is generated by the preparation period information generation unit 614 and output from the sound output unit 9 and the display unit 10.
  • the driver state determination unit 612 determines that a driver change has occurred
  • the preparation period control unit 613 sets and counts a new preparation period and informs the remaining time of the new preparation period.
  • new preparation period information is generated by the preparation period information generation unit 614 and output from the audio output device 9 and the display unit 10.
  • the driver state determination unit 612 uses the video data about the driver acquired by the monitoring camera before starting the counting of the preparation period set in step S17, and the driving acquired by the monitoring camera in the new preparation period.
  • the similarity is calculated using the video data of the driver, and a determination is made as to whether or not a driver change has occurred based on the calculated similarity. Therefore, based on the unique information such as the driver's face, it can be reliably determined whether or not a driver change has occurred before and after the start of the latest preparation period.
  • a new preparation period is set, the new preparation period is counted, and preparation period information for informing the remaining time is generated. And output from the audio output device 9 and the display unit 10. As a result, it is possible to determine whether or not a driver change has occurred even during the newly set preparation period, and every time the driver changes until the automatic operation mode is switched to the manual operation mode. It is possible to reliably set a new preparation period for a new driver.
  • the driver state determination unit 612 further calculates video data about the driver acquired by the monitoring camera before starting the counting of a new preparation period, and about the driver acquired by the monitoring camera in a new preparation period.
  • the similarity is calculated using the video data, and a determination is made as to whether or not a driver change has occurred based on the calculated similarity. Therefore, based on the unique information such as the driver's face, it can be reliably determined whether or not a driver change has occurred before and after the start of the latest preparation period.
  • the operation mode switching signal output unit 615 outputs a signal for switching the automatic operation mode to the manual operation mode when the latest preparation period has elapsed, to the automatic operation control device 5. For this reason, it is possible to switch the automatic operation mode to the manual operation mode after reliably providing a certain preparation period to the new driver after the change.
  • the preparation period control unit 613 can adjust the newly set preparation period to a predetermined length according to the current driver state, and can set an appropriate preparation period flexibly. , Can increase safety.
  • information notifying that the driver change has been confirmed is generated by the preparation period information generation unit 614 and output from a predetermined output unit. Accordingly, the driver quickly and easily grasps that the preparation period is newly set and the remaining time of the preparation period, and that the preparation period is newly set is due to the change of the driver. be able to.
  • the latest monitoring video data acquired before the start of the latest preparation period is employed as the monitoring video data serving as a reference for determining the occurrence of the driver change.
  • this is merely an example, and other monitoring video data can be adopted as the reference monitoring video data.
  • criteria for selecting the monitoring video data as a reference include monitoring video data that has been successfully detected at the position of each feature point in the face area, and monitoring video data that is as close in time as possible to the start time of the latest preparation period. It is mentioned.
  • the monitoring video data used as the reference need not be limited to the monitoring video data at a specific time, and monitoring video data at a plurality of times is selected and averaged using the face shape pattern for each monitoring video data. It is also possible to use the calculated one for similarity calculation.
  • operator's change was demonstrated as a uniform time (same 60 seconds as the preparation period before being newly set).
  • the driver state determination unit 612 is, for example, a global operation image (related to the driver) acquired over time by the driver monitoring image acquisition control unit 611, a biological signal (for example, a pulse wave sensor or Based on the driver's pulse wave signal or heartbeat signal detected by the heartbeat sensor, the signal indicating the vertical movement of the diaphragm detected by the pressure sensor), etc., the driver's state is determined.
  • the preparation period control unit 613 prepares the preparation period table storage unit 633.
  • the preparation period corresponding to the current driver's situation is set with reference to the table stored in (1).
  • the preparation period considering the new driver condition can be set flexibly, and a safer and more comfortable driving environment can be provided.
  • the preparation period control unit 613 starts setting the preparation period and counting it down.
  • the trigger for setting the preparation period and starting the countdown by the preparation period control unit 613 is not limited to this example.
  • an instruction input that the driver himself “switches to the manual operation mode” by a predetermined operation may be used as a trigger, or an override operation may be performed as a trigger.
  • the vehicle type, the function of the automatic operation control device, the control function and control procedure of the operation mode switching control device, and the control contents can be variously modified and implemented without departing from the gist of the present invention.
  • the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention in the implementation stage.
  • the embodiments may be appropriately combined as much as possible, and in that case, the combined effect can be obtained.
  • the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements.
  • An operation mode switching control device for switching and controlling a vehicle operation mode between a manual operation mode and an automatic operation mode, Having at least one hardware processor;
  • the at least one hardware processor comprises: When it is determined that the driver is in a state where the driver can perform the driving operation in the manual operation mode during the operation control period in the automatic operation mode, the first preparation period is set, and the first preparation period is counted.
  • An operation mode switching control method executed by an apparatus for switching and controlling a vehicle operation mode between a manual operation mode and an automatic operation mode, Using at least one hardware processor, A control process of setting a first preparation period when switching from the automatic operation mode to the manual operation mode, and counting the first preparation period; Generating a first preparation period information for informing the remaining time of the first preparation period; Outputting the first preparation period information; In the first preparation period, a determination process for determining whether or not a driver change has occurred; Run When it is determined that a driver change has occurred in the determination process, A control step of setting a second preparation period and counting the second preparation period; Generating a second preparation period information for informing the remaining time of the second preparation period; An output process of outputting the second preparation period information; The operation mode switching control method for executing

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Traffic Control Systems (AREA)
  • Auxiliary Drives, Propulsion Controls, And Safety Devices (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
PCT/JP2017/033149 2017-03-09 2017-09-13 運転モード切替制御装置、方法及びプログラム WO2018163474A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
DE112017007208.5T DE112017007208T5 (de) 2017-03-09 2017-09-13 Fahrmodusumschaltsteuereinrichtung, -verfahren und -programm

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017-045243 2017-03-09
JP2017045243A JP2018149822A (ja) 2017-03-09 2017-03-09 運転モード切替制御装置、方法及びプログラム

Publications (1)

Publication Number Publication Date
WO2018163474A1 true WO2018163474A1 (ja) 2018-09-13

Family

ID=63449045

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/033149 WO2018163474A1 (ja) 2017-03-09 2017-09-13 運転モード切替制御装置、方法及びプログラム

Country Status (3)

Country Link
JP (1) JP2018149822A (de)
DE (1) DE112017007208T5 (de)
WO (1) WO2018163474A1 (de)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111915159A (zh) * 2020-07-15 2020-11-10 北方工业大学 一种基于动态时间预算的个性化接管预警方法和系统
CN113173179A (zh) * 2021-06-09 2021-07-27 中国第一汽车股份有限公司 一种驾驶模式切换提示方法、装置及车辆

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7264086B2 (ja) 2020-02-27 2023-04-25 トヨタ自動車株式会社 車両制御装置及び車両制御方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008191871A (ja) * 2007-02-02 2008-08-21 Toyota Motor Corp 飲酒運転防止支援システム
JP2008239071A (ja) * 2007-03-28 2008-10-09 Toyota Motor Corp 運転管理装置
US20160231743A1 (en) * 2013-10-01 2016-08-11 Volkswagen Ag Method for a driver assistance system of a vehicle
JP2017013605A (ja) * 2015-06-30 2017-01-19 株式会社デンソー 車載システム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008191871A (ja) * 2007-02-02 2008-08-21 Toyota Motor Corp 飲酒運転防止支援システム
JP2008239071A (ja) * 2007-03-28 2008-10-09 Toyota Motor Corp 運転管理装置
US20160231743A1 (en) * 2013-10-01 2016-08-11 Volkswagen Ag Method for a driver assistance system of a vehicle
JP2017013605A (ja) * 2015-06-30 2017-01-19 株式会社デンソー 車載システム

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111915159A (zh) * 2020-07-15 2020-11-10 北方工业大学 一种基于动态时间预算的个性化接管预警方法和系统
CN111915159B (zh) * 2020-07-15 2023-09-12 北方工业大学 一种基于动态时间预算的个性化接管预警方法和系统
CN113173179A (zh) * 2021-06-09 2021-07-27 中国第一汽车股份有限公司 一种驾驶模式切换提示方法、装置及车辆
CN113173179B (zh) * 2021-06-09 2023-02-21 中国第一汽车股份有限公司 一种驾驶模式切换提示方法、装置及车辆

Also Published As

Publication number Publication date
JP2018149822A (ja) 2018-09-27
DE112017007208T5 (de) 2019-12-12

Similar Documents

Publication Publication Date Title
WO2018070064A1 (ja) 運転モード切替制御装置、方法およびプログラム
JP6519435B2 (ja) 報知管理装置及び報知管理方法
JP6690582B2 (ja) 運転モード切替制御装置、方法およびプログラム
JP6447647B2 (ja) 運転モード切替制御装置、方法およびプログラム
CN110192084B (zh) 自动驾驶辅助装置、方法及程序
JP6690581B2 (ja) 運転モード切替制御装置、方法およびプログラム
CN111295699B (zh) 辅助方法以及利用该辅助方法的辅助系统、辅助装置
JP2017157196A (ja) 運転者監視システム
CN109843690B (zh) 驾驶模式切换控制装置、系统以及方法
WO2018163474A1 (ja) 運転モード切替制御装置、方法及びプログラム
GB2545978A (en) Vehicle control based on connectivity of a portable device
JP6856086B2 (ja) 報知管理装置及び報知管理プログラム
WO2018163552A1 (ja) 運転モード切替制御装置、方法およびプログラム
WO2018163490A1 (ja) 視認支援装置、方法およびプログラム
JP6648722B2 (ja) 故障判定装置、方法およびプログラム
WO2018163454A1 (ja) 運転モード切替制御装置、方法およびプログラム
WO2017150466A1 (ja) 運転者監視システム
JP2019079217A (ja) 運転支援装置、及び運転支援方法
WO2018163493A1 (ja) 監視装置、方法、及びプログラム
WO2018163536A1 (ja) 運転者身体コンディション回復支援装置、方法およびプログラム
WO2018168046A1 (ja) 集中度判定装置、集中度判定方法及び集中度判定のためのプログラム
WO2018168050A1 (ja) 集中度判定装置、集中度判定方法及び集中度判定のためのプログラム
JP2017220171A (ja) 安全運転支援装置及び安全運転支援プログラム
WO2018163497A1 (ja) 覚醒支援装置、方法およびプログラム
WO2018159013A1 (ja) 覚醒支援装置、方法およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17899962

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17899962

Country of ref document: EP

Kind code of ref document: A1