WO2018163552A1

WO2018163552A1 - Drive mode switching control device, method and program

Info

Publication number: WO2018163552A1
Application number: PCT/JP2017/044488
Authority: WO
Inventors: 初美青位; 相澤　知禎; 岡地　一喜; 啓菅原; 充恵鵜野; 光司滝沢
Original assignee: オムロン株式会社
Priority date: 2017-03-09
Filing date: 2017-12-12
Publication date: 2018-09-13
Also published as: JP6627810B2; JP2018151684A

Abstract

The purpose of this invention is to shorten the time from an override operation to the completion of switching, and adopts the following measures to achieve this purpose. This drive mode switching control device determines, at an arbitrarily-defined time interval, whether the driver is in a state in which he or she is capable of driving operation in manual driving mode. When an override operation is detected by an operation detection unit 613, the drive mode switching control device outputs a switching signal which switches self-driving mode to manual driving mode if the state of the driver is such that he or she is capable of performing driving operation in manual driving mode, on the basis of determination results corresponding to the time immediately prior to the time when the override operation was detected.

Description

Operation mode switching control device, method and program

The present invention relates to an operation mode switching control device, method and program for switching a vehicle operation mode between a manual operation mode and an automatic operation mode.

In recent years, as a driving mode of a vehicle, in addition to a manual driving mode in which the vehicle is driven based on a driving operation of the driver, an automatic driving mode in which the vehicle is driven along a predetermined route without depending on the driving operation of the driver. Development is underway. Autonomous driving mode includes, for example, navigation system information using the Global Positioning System (GPS), traffic information acquired by road-to-vehicle communication, and information on the surrounding monitoring system that monitors the position and movement of people and vehicles in the vicinity. Originally, the vehicle can be automatically driven by controlling a power unit, a steering device, a brake, and the like.

By the way, although the automatic driving mode can be expected to reduce the burden on the driving operation of the driver and alleviate traffic congestion, it is practically difficult to automate all the driving operations from the start to the end of driving. For this reason, it is required to appropriately switch from the automatic operation mode to the manual operation mode. As a trigger for switching the automatic operation mode to the manual operation mode, for example, there is a driving operation (hereinafter referred to as an override operation) by the driver during the automatic driving (see, for example, JP-A-2016-151815). When an override operation is detected, it is necessary to detect the state of the driver and determine that it is in a state where a manual driving operation is possible prior to switching.

However, the override operation as described above is usually an emergency operation that is consciously performed by the driver. Here, in the case of an emergency operation, the driver's state is detected and determined prior to the switching even though the driver desires a quick switching. For this reason, it takes a long time to switch from the emergency operation to the manual operation mode, and there is a concern that the driving operation by the driver is delayed. In addition, the driver feels uneasy before switching to the manual operation mode, which is very undesirable.

This invention is intended to provide an operation mode switching control device, method, and program capable of shortening the time from the override operation to the manual operation mode.

In order to solve the above-mentioned problem, a first aspect of the present invention is an operation mode switching control device for switching a vehicle operation mode from an automatic operation mode to a manual operation mode for monitoring a driver's state. An acquisition unit that acquires sensing data representing the driver's state from a monitoring sensor, and whether or not the driver's state is in a state where a driving operation in the manual driving mode can be performed based on the acquired sensing data Is detected at an arbitrary time interval, and the override operation by the driver is detected based on a determination unit that holds the determination result and a detection signal output from an in-vehicle sensor that can detect the driving operation by the driver. When an override operation is detected by the operation detection unit and the operation detection unit, among the determination results held in the determination unit, A switching signal for switching the automatic operation mode to the manual operation mode when the state of the driver is in a state where the operation operation by the manual operation mode can be performed based on a determination result corresponding to the state immediately before the bar ride operation is detected. And a switching signal output unit for outputting.

According to the first aspect of the present invention, it is determined at any time interval whether or not the driver is in a state where the driving operation in the manual operation mode can be performed, and an override operation is detected by the operation detection unit. And the automatic driving mode is changed to the manual driving mode when the driver is in a state where the driving operation can be performed in the manual driving mode based on a determination result corresponding immediately before the override operation is detected. A switching signal for switching is output. For this reason, since it is not necessary to detect and determine the state of the driver after the override operation, the time until the switching signal from the override operation to the manual operation mode is output can be shortened.

According to a second aspect of the present invention, the determination unit includes a concentration level detection unit and a concentration level determination unit, and the concentration level detection unit determines the concentration level of the driver with respect to driving based on the acquired sensing data. Whether or not the state of the driver is in a state where the driver can perform the driving operation in the manual driving mode based on the latest concentration degree detected by the detecting unit. The determination is made at every time interval.

According to the second aspect of the present invention, the degree of concentration of the driver with respect to driving is periodically detected, and based on the detected latest degree of concentration, the driver's state is a driving operation in the manual driving mode. It is determined at each time interval whether or not it is in a state where it can be performed. For this reason, prior to switching, it is possible to check the determination result indicating whether or not the driver is in a state where the driver can perform the driving operation depending on whether or not the latest concentration of the driver is high.

In a third aspect of the present invention, the concentration determination unit executes the determination based on a plurality of past concentrations including the latest concentration detected by the concentration detection unit. It is.

According to the third aspect of the present invention, the determination is executed based on a plurality of past concentrations including the detected latest concentration. For this reason, prior to the switching, it is possible to check a determination result indicating whether or not the driver is in a state where the driver can perform the driving operation according to a plurality of degrees of concentration in the past past of the driver.

According to a fourth aspect of the present invention, the determination is executed by the concentration determination unit based on a tendency that the plurality of concentrations vary from the past to the present.

According to the fourth aspect of the present invention, the determination is performed based on a tendency that the plurality of concentration levels fluctuate from the past to the present. Therefore, prior to switching, whether or not the driver is in a state where the driver can perform the driving operation based on a tendency that the plurality of degrees of concentration of the driver increase, substantially stabilize, or decrease from the past to the present. It is possible to confirm the determination result shown. For example, when the driver's concentration degree tends to increase or is almost stable, a determination result indicating that the driving operation can be performed can be confirmed. When the driver's concentration tends to decrease, a determination result indicating that the driver cannot perform the driving operation can be confirmed.

According to a fifth aspect of the present invention, the determination unit includes an estimation unit and a state determination unit, the estimation unit estimates the driver's state based on the acquired sensing data, and the state determination unit. Thus, based on the state estimated by the estimation unit, it is determined at each time interval whether or not the driver is in a state where the driving operation can be performed in the manual driving mode.

According to the fifth aspect of the present invention, the state of the driver is estimated, and it is determined at each time interval whether or not the estimated state of the driver is in a state where the driving operation in the manual driving mode can be performed. . For this reason, prior to switching, it is possible to confirm a determination result indicating whether or not the estimated state is in a state where a driving operation can be performed.

According to a sixth aspect of the present invention, the state determination unit executes the determination based on a tendency that the estimated state varies from the past to the present.

According to the sixth aspect of the present invention, the determination is executed based on a tendency that the estimated state varies from the past to the present. For this reason, prior to switching, the driver's state indicates whether or not the driver's state is ready for driving based on the tendency of the driver's state to change from a certain state to a state suitable for driving from the past to the present. The determination result can be confirmed. For example, when the driver's state tends to approach a state suitable for driving or is generally stable in a state suitable for driving, the determination result indicating that the driver can perform a driving operation can be confirmed. When the driver's state tends to approach a state unsuitable for driving or tends to be substantially stable in a state unsuitable for driving, it is possible to confirm the determination result that the driver is not in a state capable of performing a driving operation.

According to a seventh aspect of the present invention, the determination unit further includes an adjustment unit, and the adjustment unit adjusts the time interval based on the tendency.

According to the seventh aspect of the present invention, the time interval of the determination by the determination unit is adjusted based on the tendency that the driver's concentration or state varies. For this reason, for example, when there is a tendency to change so as to obtain a determination result indicating that the driving operation in the manual operation mode can be performed, the determination time interval is adjusted to be longer. On the other hand, for example, when there is a tendency to fluctuate so that a determination result indicating that the driving operation in the manual operation mode cannot be performed is obtained, the determination time interval is adjusted to be short.

That is, according to each aspect of the present invention, it is possible to provide an operation mode switching control device, method, and program capable of shortening the time from the override operation to the manual operation mode.

FIG. 1 is a diagram showing an overall configuration of an automatic operation control system including an operation mode switching control device according to a first embodiment of the present invention. FIG. 2 is a block diagram showing a functional configuration of the operation mode switching control apparatus according to the first embodiment of the present invention. FIG. 3 is a flowchart showing the procedure and control contents of the operation mode switching control by the operation mode switching control device shown in FIG. FIG. 4 is a flowchart showing the procedure for determining whether or not manual operation is possible and the control content in the flow shown in FIG. FIG. 5 is a flowchart showing another procedure and control contents of the flow shown in FIG. FIG. 6 is a flowchart showing the procedure and control contents of a specific example of the flow shown in FIG. FIG. 7 is a flowchart showing the procedure and control contents of a further specific example of the flow shown in FIG. FIG. 8 is a flowchart showing a procedure for determining whether or not manual operation is possible by the operation mode switching control device according to the second embodiment of the present invention and the control contents. FIG. 9 is a flowchart showing the procedure and control contents of a specific example of the flow shown in FIG. FIG. 10 is a flowchart showing a procedure and control contents of a further specific example of the flow shown in FIG.

Embodiment

Embodiments according to the present invention will be described below with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram showing an overall configuration of an automatic operation control system including an operation mode switching control device according to a first embodiment of the present invention. This automatic driving control system is mounted on a vehicle 1 such as a passenger car.

The vehicle 1 includes, as basic equipment, a power unit 2 including a power source and a transmission, and a steering device 3 equipped with a steering wheel 4. Moreover, the vehicle 1 has a manual operation mode and an automatic operation mode as operation modes. An engine and / or a motor is used as the power source.

The manual driving mode is a mode in which the vehicle 1 is driven mainly by a driver's manual driving operation, for example. The manual operation mode includes, for example, an operation mode for driving the vehicle based only on the driver's driving operation, and an operation mode for performing driving operation support control for supporting the driving operation of the driver while mainly driving the driver's driving operation. Is included.

The driving operation support control assists the steering torque so that the driver's steering becomes an appropriate steering amount based on the curvature of the curve when the vehicle 1 is traveling on the curve, for example. The driving operation support control includes control for assisting a driver's accelerator operation (for example, operation of an accelerator pedal) or brake operation (for example, operation of a brake pedal), manual steering (manual operation of steering), and manual speed adjustment (speed). Adjustment manual operation). In manual steering, the vehicle 1 is steered mainly by the driver's operation of the steering wheel 4. In the manual speed adjustment, the speed of the vehicle is adjusted mainly by the driver's accelerator operation or brake operation.

Note that the driving operation support control does not include control for forcibly intervening in the driving operation of the driver and automatically driving the vehicle. In other words, in the manual driving mode, the driving operation of the driver is reflected in the driving of the vehicle within a preset allowable range, but forcibly intervenes in the driving of the vehicle under certain conditions (for example, deviation from the lane of the vehicle). Control to do is not included.

On the other hand, the automatic operation mode is a mode that realizes an operation state in which the vehicle automatically travels along the road on which the vehicle travels, for example. The automatic driving mode includes, for example, a driving state in which the vehicle automatically travels toward a preset destination without driving by the driver. The automatic driving mode does not necessarily need to automatically perform all the control of the vehicle, and includes a driving state in which the driving operation of the driver is reflected in the traveling of the vehicle within a preset allowable range. That is, the automatic driving mode includes control for forcibly intervening in driving of the vehicle under certain conditions, while reflecting the driving operation of the driver in driving of the vehicle within a preset allowable range.

The automatic operation control device 5 performs operation control in the automatic operation mode. The automatic driving control device 5 acquires sensing data from the steering sensor 11, the accelerator pedal sensor 12, the brake pedal sensor 13, the GPS receiver 14, the gyro sensor 15, and the vehicle speed sensor 16, respectively. The automatic driving control device 5 monitors these sensing data, route information generated by a navigation system (not shown), traffic information acquired by road-to-vehicle communication, and the positions and movements of surrounding people and vehicles. The travel of the vehicle 1 is automatically controlled based on information obtained by the monitoring system.

Automatic control includes, for example, automatic steering (automatic steering operation) and automatic speed adjustment (automatic driving of speed). Automatic steering is an operating state in which the steering device 3 is automatically controlled. Automatic steering includes Lane Keeping Assist (LKA). For example, the LKA automatically controls the steering device 3 so that the vehicle 1 does not deviate from the traveling lane even when the driver does not perform the steering operation. Even when LKA is being executed, the driver's steering operation may be reflected in the steering of the vehicle in a range where the vehicle 1 does not deviate from the travel lane (allowable range). Note that automatic steering is not limited to LKA.

Automatic speed adjustment is an operating state in which the speed of the vehicle 1 is automatically controlled. Automatic speed adjustment includes Adaptive Cruise Control (ACC). For example, when there is no preceding vehicle ahead of the vehicle 1, ACC performs constant speed control that causes the vehicle 1 to travel at a constant speed at a preset speed, and when the preceding vehicle exists ahead of the vehicle 1. Is a follow-up control that adjusts the vehicle speed of the vehicle 1 in accordance with the inter-vehicle distance from the preceding vehicle. The automatic operation control device 5 decelerates the vehicle 1 according to the driver's brake operation (for example, operation of the brake pedal) even when ACC is being executed. In addition, the automatic operation control device 5 can perform the driver's accelerator operation (for example, accelerator) up to a preset maximum allowable speed (for example, the maximum speed legally determined on the traveling road) even during execution of ACC. The vehicle can be accelerated according to the pedal operation. Note that the automatic speed adjustment is not limited to ACC, but also includes Cruise Control (CC) that performs only constant speed control.

By the way, the automatic operation control system of this embodiment is an operation mode switching control device 6 and a driver camera 7 as a first monitoring sensor as devices for switching between the manual operation mode and the automatic operation mode. And a torque sensor 8 as a second monitoring sensor and an alarm generator 9.

The driver camera 7 is installed, for example, at a position in front of the driver as on the dashboard, and images the driver and outputs a video signal thereof to the driving mode switching control device 6. The torque sensor 8 detects torque generated when the driver operates the steering wheel 4, and outputs a detection signal to the operation mode switching control device 6. The alarm generator 9 includes a speaker and a display. The alarm generator 9 outputs the voice signal of the message output from the operation mode switching control device 6 from the speaker and displays the display signal of the message on the display.

The operation mode switching control device 6 controls the switching of the operation modes as a whole, and is configured as follows.
FIG. 2 is a block diagram showing the functional configuration.

That is, the operation mode switching control device 6 includes a control unit 61, an input / output interface unit 62, and a storage unit 63.

The input / output interface unit 62 receives the video signal and the torque detection signal output from the driver camera 7 and the torque sensor 8, respectively, and converts them into digital data. Similarly, the input / output interface unit 62 receives detection signals as sensing data output from the steering sensor 11, the accelerator pedal sensor 12, and the brake pedal sensor 13, and converts them into digital data. The input / output interface unit 62 converts the message output from the control unit 61 into an audio signal and a display signal, and outputs them to the alarm generator 9. Further, the input / output interface unit 62 outputs the switching signal output from the control unit 61 to the automatic operation control device 5.

The storage unit 63 uses, as a storage medium, a non-volatile memory that can be written and read at any time, such as, for example, Solid State Drive (SSD) or Hard Disk Drive (HDD). The storage unit 63 includes a driver monitoring video storage unit 631 and a determination result storage unit 632 as storage areas used for carrying out this embodiment.

The control unit 61 has a central processing unit (CPU) and a program memory that constitute a computer. The control unit 61 includes a driver monitoring video acquisition unit 611, a determination unit 612, an operation detection unit 613, and a switching signal output unit 614 as control functions necessary for carrying out this embodiment. Each of these control functions is realized by causing the CPU to execute a program stored in the program memory.

The driver monitoring video acquisition unit 611 receives digital data (driver monitoring video data) of the driver video signal output from the driver camera 7 from the input / output interface unit 62, and stores the acquired driver monitoring video data in the storage unit 63. Are stored in the driver monitoring video storage unit 631.

The determination unit 612 reads the driver monitoring video data from the driver monitoring video storage unit 631 at a preset time interval. Then, each time the driver monitoring video data is read, the determination unit 612 performs a process of determining whether or not the driver can manually perform a driving operation based on the driver monitoring video data. For example, the determination unit 612 determines whether or not the driver is in a sleep state by checking whether or not the driver has closed his / her eyes. Then, the determination unit 612 stores information indicating the determination result in the determination result storage unit 632 in association with a time stamp indicating the determination timing.

Here, the determination unit 612 may execute the following processes.

(A1) Concentration detection processing for periodically detecting the driver's concentration on driving based on sensing data acquired from the driver camera 7 (via the input / output interface unit 62 and the driver monitoring video storage unit 631). . In this case, it is preferable to hold the detected concentration level in the determination result storage unit 632 in association with a time stamp indicating the detection timing from the viewpoint of enabling determination based on a plurality of concentration levels. Further, the determination unit 612 may acquire sensing data from the driver camera 7 without using the input / output interface unit 62 and the driver monitoring video storage unit 631. The same applies to the following embodiments.

(A2) 集中 Concentration that determines at every time interval whether or not the driver is in a state where the driver can perform the driving operation in the manual operation mode based on the latest concentration detected by the concentration detection process of (A1) above. Degree determination process. Note that the time interval of the concentration level determination process and the detection cycle of the concentration level detection process may be the same or different. In addition, the time interval of the concentration level determination process is preferably longer than the detection cycle of the concentration level detection process from the viewpoint of enabling determination based on a plurality of concentration levels.

The concentration determination process of (A2) is performed based on a plurality of past concentrations including the latest concentration detected by the concentration detection process of (A2a) (A1). Also good.

(A2b) In the concentration level determination process (A2a), the determination may be executed based on a tendency that the concentration levels fluctuate from the past to the present.

(A2b) In the concentration determination process (A2b), the driver's state may be predicted based on the tendency (A2c), and the determination may be executed based on the prediction result. It is preferable that the prediction here is directed to the state of the driver at the next determination timing from the viewpoint of improving the prediction accuracy. Note that the concentration determination process of (A2a), (A2b), or (A2c) above is a driver-specific process based on multiple concentrations for each driver because there are individual differences in the human condition. May be executed as In addition, each driver can be identified by face authentication from sensing data stored in the driver monitoring video storage unit 631.

Here, the determination unit 612 may further execute an adjustment process for adjusting the time interval based on the tendency of (A2b) or (A2c).

The determination unit 612 may include a concentration level detection unit that executes the concentration level detection process, a concentration level determination unit that executes the concentration level determination process, and an adjustment unit that executes the adjustment process. Of course, it may be distributed in each part.

The operation detection unit 613 detects an emergency override operation or a non-emergency override operation by the driver based on a detection signal output from the torque sensor 8 as an in-vehicle sensor capable of detecting a driving operation by the driver. In addition, as a vehicle-mounted sensor, not only this but the steering sensor 11, the accelerator pedal sensor 12, and the brake pedal sensor 13 can be used suitably.

When the operation detection unit 613 detects an override operation, the switching signal output unit 614 determines the driver based on the determination result corresponding immediately before the override operation is detected among the determination results held by the determination unit 612. Is in a state where the operation operation in the manual operation mode can be performed, a switching signal for switching the automatic operation mode to the manual operation mode is output to the automatic operation control device 5.

Next, the operation of the operation mode switching control device configured as described above will be described.
FIG. 3 is a flowchart showing the overall control procedure and control contents.

(1) Acquisition of Driver Monitoring Video When driving is started, the driver camera 7 is activated, and continuously captures a predetermined range including the driver's face and outputs the video signal. In this state, under the control of the driver monitoring video acquisition unit 611, the operation mode switching control device 6 first outputs the digital data (driver monitoring video data) of the video signal output from the driver camera 7 in step S1 to the input / output interface unit. The captured driver monitoring video data is stored in the driver monitoring video storage unit 631 of the storage unit 63 in step S2.

Note that the imaging of the driver may be intermittently performed at a period shorter than a determination period of a determination unit 612 described later. In addition, the driver camera 7 or the input / output interface unit 62 may encode the video signal according to a predetermined encoding method. In this way, it is possible to save the storage capacity of the driver monitoring video storage unit 631 by reducing the information amount of the monitoring video data.

(2) Determination of Driver's State When acquisition of the driver monitoring video data is started, the driving mode switching control device 6 is controlled by the determination unit 612 every time a certain period of time is determined in step S3. In step S10, the driver monitoring video data is read from the driver monitoring video storage unit 631. Then, from each of the read driver monitoring video data, it is determined as shown in the following steps S11 to S12 whether or not the driver can perform a driving operation manually.
FIG. 4 is a flowchart showing the control procedure and control contents.

That is, the determination unit 612 periodically detects the driver's concentration on driving based on the sensing data stored in the driver monitoring video storage unit 631 in step S11. For example, based on the driver monitoring video data, the eye open state of the driver's eyes, the frequency of blinking, or eye movements are detected to recognize the driver's arousal level. This arousal level is an example of the degree of concentration and is represented by a numerical value within a range of 0 to 100%. The degree of concentration is not limited to a numerical value within the range of 0 to 100%. For example, “1” is set when the driver's line-of-sight direction is within a predetermined range, and “0” is set when the driver is not within the predetermined range. For example, a value (flag) of “1” or “0” may be used.

Further, in step S12, the determination unit 612 determines, based on the latest concentration level detected in step S11, whether or not the driver is in a state where the driving operation in the manual operation mode can be performed at every time interval. . For example, it is determined whether or not the driver can manually perform a driving operation by comparing the recognized arousal level with a threshold value. Thus, the process of step S10 is completed.

Subsequently, returning to FIG. 3, when the determination result of whether or not the manual driving operation is possible is obtained, the determination unit 612 associates the information indicating the determination result with information indicating the determination timing, for example, time stamp information in step S4. The determination result is stored in the determination result storage unit 632.

Moreover, the determination part 612 adjusts the time interval which performs determination of step S10 by step S5 based on the concentration degree detected by step S11. For example, the determination unit 612 adjusts the time interval based on a tendency that a plurality of concentrations vary from the past to the present. Note that step S5 is not limited to after step S4, and may be executed at any time or may be omitted.

(3) Detection of Override Operation The operation mode switching control device 6 subsequently executes step S6 under the control of the operation detection unit 613. At this time, for example, the torque sensor 8 as an in-vehicle sensor outputs a detection signal according to the driving operation by the driver. The driving mode switching control device 6 detects an override operation by the driver based on a detection signal corresponding to the driving operation. As a result of step S6, when an override operation is not detected, the process returns to step S1, and when an override operation is detected, the process proceeds to step S7.

(4) Output control of switching signal When the override operation is detected, the operation mode switching control device 6 determines the determination result by the determination unit 612 immediately before the override operation is detected under the control of the switching signal output unit 614. Reading from the result storage unit 632.

In step S7, the switching signal output unit 614 determines whether or not the read determination result indicates that a manual driving operation is possible. If not, the process returns to step S1. When the result of determination in step S7 indicates that manual operation is possible, the switching signal output unit 614 outputs a switching signal for switching the automatic operation mode to the manual operation mode to the automatic operation control device 5 (step S8). As a result, the automatic driving control device 5 ends the automatic driving mode, and thereafter, driving control according to the manual operation of the driver is performed.

As described above in detail, in the first embodiment of the present invention, it is determined at any time interval whether or not the driver is in a state where the driving operation can be performed in the manual driving mode. When an override operation is detected by the operation detection unit 613, automatic driving is performed when the driver is in a state where the driver can perform a driving operation in the manual driving mode based on a corresponding determination result immediately before the override operation is detected. A switching signal for switching the mode to the manual operation mode is output. As described above, since the immediately preceding determination result is used, time for detecting and determining the state of the driver after the override operation becomes unnecessary. For this reason, the time until the switching signal from the override operation to the manual operation mode is output can be shortened.

Therefore, the time from the override operation to the manual operation mode can be shortened.

Also, the driver's concentration on driving is periodically detected, and based on the latest detected concentration, it is determined at each time interval whether or not the driver is in a state where the driving operation can be performed in the manual operation mode. Is done. For this reason, prior to switching, it is possible to check the determination result indicating whether or not the driver is in a state where the driver can perform the driving operation depending on whether or not the latest concentration of the driver is high.

Also, the determination time interval by the determination unit 612 is adjusted based on the tendency of the driver's concentration to vary. For this reason, for example, when it tends to fluctuate toward a state in which the driving operation in the manual operation mode can be performed, the determination time interval is adjusted to be longer. On the other hand, for example, when there is a tendency to fluctuate toward a state where the driving operation in the manual operation mode cannot be performed, the determination time interval is adjusted to be short. Therefore, the determination time interval can be appropriately adjusted according to the tendency of the driver's concentration to vary.

[Other Embodiments]
The present invention is not limited to the above embodiment. For example, the flowchart shown in FIG. 4 may be modified as shown in FIG. That is, the determination unit 612 determines whether or not the above-described manual driving operation is possible based on a plurality of past concentrations including the latest concentration detected in step S11 instead of step S12. (Step S12a). In this case, prior to the switching, it is possible to confirm a determination result indicating whether or not the driver is in a state where the driver can perform the driving operation according to a plurality of degrees of concentration in the past past of the driver.

For example, the flowchart shown in FIG. 5 may be modified as shown in FIG. That is, instead of step S12a, the determination unit 612 determines whether or not the above-described manual driving operation is possible based on a tendency that a plurality of concentrations vary from the past to the present (step S12b). . In this case, prior to switching, the driver's state indicates whether or not the driver is in a state where the driver can perform the driving operation based on the tendency that the driver's concentration is increasing, substantially stable, or decreasing from the past to the present. The determination result can be confirmed. For example, when the driver's concentration degree tends to increase or is almost stable, a determination result indicating that the driving operation can be performed can be confirmed. When the driver's concentration tends to decrease, a determination result indicating that the driver cannot perform the driving operation can be confirmed.

For example, the flowchart shown in FIG. 6 may be modified as shown in FIG. That is, instead of step S12b, the determination unit 612 predicts the driver's state based on a tendency that a plurality of concentrations vary (step S12c-1), and based on the prediction result, the manual driving operation described above is performed. A determination is made as to whether or not this is possible (step S12c-2). In this case, prior to switching, it is possible to confirm a determination result indicating whether or not the predicted state is in a state where a driving operation can be performed.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to the aforementioned drawings.

The second embodiment is another specific example of the determination unit 612 described above, and the determination unit 612 uses the estimated driver state as shown in FIG. Yes. In the figure, “driver state” is described as “driver state”.

For example, as described above, the determination unit 612 reads the driver monitoring video data from the driver monitoring video storage unit 631 at a preset time interval, and the driver manually operates the driver based on the driver monitoring video data. A process of determining whether or not it is ready to be performed is performed.

However, the determination unit 612 may execute the following processes instead of the processes (A1), (A2), (A2a), (A2b), and (A2c). Each process described below may be executed as a driver-specific process based on past data for each driver because there are individual differences in the human state. Each driver can be identified by face authentication as described above.

(B1) Estimation process for estimating the driver's state based on the sensing data acquired from the driver camera 7 (via the input / output interface unit 62 and the driver monitoring video storage unit 631). In this estimation process, for example, the state of emotion such as joy, anger, sorrow, comfort, and calm of the driver is estimated based on the driver monitoring video data. Such a determination unit 612 can be implemented using, for example, Artificial Intelligence (AI) that can estimate human emotions. In addition, as a driver | operator's state, you may use the state based not only on a driver | operator's emotional state but the degree of concentration with respect to a driving | operation.

(B2) 状態 A state determination process that determines, based on the state estimated by the estimation process in (B1) above, whether or not the driver is in a state where the driving operation can be performed in the manual operation mode. In this state determination process, for example, it is determined whether or not the driver is in a state where the driver can manually perform a driving operation by comparing the estimated emotion with a preset condition. As the set condition, for example, when the estimated emotional state is joy, sorrow, comfort, or calm, it may be determined that manual driving is possible. Alternatively, as the set condition, for example, it may be determined that manual driving is not possible when the estimated emotional state is anger.

(B2b) The state determination process (B2) may be performed based on a tendency that the estimated state changes from the past to the present.

In the state determination process (B2b), the driver state may be predicted based on the trend (B2c) and the determination may be executed based on the prediction result. It is preferable that the prediction here is directed to the state of the driver at the next determination timing from the viewpoint of improving the prediction accuracy.

Here, the determination unit 612 may further execute an adjustment unit that adjusts the time interval based on the tendency of (B2b) or (B2c).

The determination unit 612 may include an estimation unit that executes the estimation process, a state determination unit that executes the state determination process, and an adjustment unit that executes the adjustment process. May be implemented.

Other configurations are the same as those in the first embodiment.

Next, the operation of the operation mode switching control device configured as described above will be described with reference to FIGS. 3 and 8.
Now, it is assumed that steps S1 and S2 are executed as described above.

Subsequently, the driving mode switching control device 6 reads the driver monitoring video data from the driver monitoring video storage unit 631 in step S10 every time a predetermined time is determined in step S3 under the control of the determination unit 612. . Then, the determination unit 612 determines, from each of the read driver monitoring video data, whether or not the driver can manually perform the driving operation as shown in steps S15 to S16 below.
FIG. 8 is a flowchart showing the control procedure and control contents.

That is, the determination unit 612 estimates the driver's state based on the sensing data stored in the driver monitoring video storage unit 631 in step S15. For example, the state of emotion such as joy, anger, sorrow, comfort, or calm of the driver is estimated.

Further, in step S16, the determination unit 612 determines, based on the state estimated in step S15, whether or not the driver is in a state where the driving operation in the manual operation mode can be performed at every time interval. For example, when the estimated emotional state is joy, sadness, comfort, or calm, it is determined that manual driving is possible. If the estimated emotional state is anger, it is determined that manual driving is not possible. Thus, the process of step S10 is completed.

Moreover, the determination part 612 adjusts the time interval which performs determination of step S10 by step S5 based on the driver state estimated by step S15. For example, the determination unit 612 adjusts the time interval based on a tendency that a plurality of driver states fluctuate from the past to the present. Note that step S5 is not limited to after step S4, and may be executed at any time or may be omitted.

Thereafter, the processing in steps S6 to S8 is executed as described above.

As described above in detail, in the second embodiment of the present invention, steps S1 to S8 are executed as in the first embodiment. For this reason, as described above, it is not necessary to detect and determine the state of the driver after the override operation. Therefore, it is possible to shorten the time until the switching signal from the override operation to the manual operation mode is output. . Accordingly, it is possible to reduce the time from the override operation to the manual operation mode.

In step S10, the state of the driver is estimated, and it is determined for each time interval whether the estimated state of the driver is in a state where the driving operation in the manual operation mode can be performed. For this reason, prior to switching, it is possible to confirm a determination result indicating whether or not the estimated state is in a state where a driving operation can be performed.

[Other Embodiments]
The present invention is not limited to the above embodiment. For example, the flowchart shown in FIG. 8 may be modified as shown in FIG. That is, the determination unit 612 determines whether or not the above-described manual driving operation is possible based on a tendency that the estimated state varies from the past to the present instead of step S16 (step S16b). . In this case, prior to switching, a determination indicating whether or not the driver is in a state where the driver can perform a driving operation based on a tendency that the driver's state changes from any state to a state suitable for driving from the past to the present. The result can be confirmed. For example, when the driver's state tends to approach a state suitable for driving or a state that is substantially stable in a state suitable for driving, such as the tendency of an angry emotional state to approach a calm emotional state Can be confirmed. When the driver's state tends to approach a state unsuitable for driving or tends to be substantially stable in a state unsuitable for driving, it is possible to confirm the determination result that the driver is not in a state capable of performing a driving operation.

For example, the flowchart shown in FIG. 9 may be modified as shown in FIG. That is, the determination unit 612 predicts the driver's state based on the tendency of the estimated state to change instead of step S16b (step S16c-1), and based on the prediction result, the above-described manual driving operation is performed. It is determined whether or not it is possible (step S16c-2). In this case, prior to switching, it is possible to confirm a determination result indicating whether or not the predicted state is in a state where a driving operation can be performed.

Still other embodiments include, for example, the following.

In the first or second embodiment, the case where the video signal including the driver's face obtained by the driver camera 7 is used as the means for determining the driver's state has been described as an example. However, it is not limited to that. For example, based on a biological signal obtained by a biological sensor, for example, a pulse wave signal or a heartbeat signal of a driver detected by a pulse wave sensor or a heartbeat sensor, or a signal representing vertical movement of the diaphragm detected by a pressure sensor, You may make it determine a driver | operator's state. Alternatively, as a means for determining the state of the driver, the steering torque of the driver's steering operation detected by the torque sensor 8 is used, and the driver performs a manual driving operation when a torque exceeding a predetermined value is detected. It may be determined that it is in a state where it can be performed. Not limited to this, an operation input means that allows the driver to input that manual driving operation is possible, such as a push button provided on the steering wheel 4 or a soft button provided on the touch panel. May be used to determine the state of the driver. In addition, it is possible to use an accelerator pedal operation or the like.

In addition, the vehicle type, the function of the automatic operation control device, the control function and control procedure of the operation mode switching control device, and the control contents can be variously modified and implemented without departing from the gist of the present invention.

That is, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the scope of the invention in the implementation stage. In addition, the embodiments may be appropriately combined as much as possible, and in that case, the combined effect can be obtained. Furthermore, various inventions can be extracted by an appropriate combination of a plurality of constituent elements disclosed in the one embodiment.

A part or all of the above embodiment can be described as in the following supplementary notes, but is not limited thereto.
(Appendix 1)
An operation mode switching control device for switching a vehicle operation mode from an automatic operation mode to a manual operation mode,
A memory for storing a determination result obtained by determining whether or not the driver's state is in a state where the driving operation in the manual driving mode can be performed;
And at least one hardware processor connected to the memory,
The at least one hardware processor comprises:
From the monitoring sensor for monitoring the state of the driver, obtaining sensing data representing the state of the driver,
Based on the acquired sensing data, it is determined at an arbitrary time interval whether or not the driver's state is in a state where the driving operation in the manual driving mode can be performed, and the determination result is held in the memory,
Based on a detection signal output from an in-vehicle sensor capable of detecting a driving operation by the driver, an override operation by the driver is detected,
When the override operation is detected, based on the determination result corresponding to immediately before the override operation is detected among the determination results held in the storage unit, the driver's state is the driving in the manual operation mode. An operation mode switching control device configured to output a switching signal for switching the automatic operation mode to the manual operation mode when the operation is enabled.

(Appendix 2)
An operation mode switching control method executed by a device for switching a vehicle operation mode from an automatic operation mode to a manual operation mode,
Using at least one hardware processor to obtain sensing data representing the driver's state from a monitoring sensor for monitoring the driver's state; and
Using at least one hardware processor, based on the acquired sensing data, determine whether the driver is in a state where the driver can perform a driving operation in the manual driving mode at an arbitrary time interval, and A determination process for storing the determination result in at least one memory;
An operation detection process for detecting an override operation by the driver based on a detection signal output from an in-vehicle sensor capable of detecting a driving operation by the driver using at least one hardware processor;
When an override operation is detected by the operation detection process using at least one hardware processor, based on a determination result corresponding to immediately before the override operation is detected among the determination results held by the determination process. A switching signal output process for outputting a switching signal for switching the automatic driving mode to the manual driving mode when the driver is in a state where the driving operation can be performed in the manual driving mode. Control method.

Claims

An operation mode switching control device for switching a vehicle operation mode from an automatic operation mode to a manual operation mode,
An acquisition unit that acquires sensing data representing the state of the driver from a monitoring sensor for monitoring the state of the driver;
Based on the acquired sensing data, it is determined at any time interval whether the driver's state is in a state where the driving operation in the manual driving mode can be performed, and a determination unit that holds the determination result;
Based on a detection signal output from an in-vehicle sensor capable of detecting a driving operation by the driver, an operation detection unit that detects an override operation by the driver;
When an override operation is detected by the operation detection unit, the driver's state is changed to the manual based on a determination result corresponding to immediately before the override operation is detected among the determination results held in the determination unit. An operation mode switching control device comprising: a switching signal output unit that outputs a switching signal for switching the automatic operation mode to the manual operation mode when the operation mode can be operated.
The determination unit
Based on the acquired sensing data, a concentration level detection unit that periodically detects the concentration level of the driver with respect to driving,
A degree-of-concentration determination unit that determines, based on the latest degree of concentration detected by the degree-of-concentration detection unit, whether or not the driver is in a state where the driver can perform a driving operation in the manual operation mode at each time interval; An operation mode switching control device according to claim 1.
The operation mode switching control device according to claim 2, wherein the concentration level determination unit performs the determination based on a plurality of past concentration levels including the latest concentration level detected by the concentration level detection unit.
The operation mode switching control device according to claim 3, wherein the concentration degree determination unit performs the determination based on a tendency that the plurality of concentration degrees fluctuate from the past to the present.
The determination unit
Based on the acquired sensing data, an estimation unit that estimates the state of the driver;
The state determination part which determines whether the said driver | operator's state exists in the state which can perform the driving operation by the said manual driving mode based on the state estimated by the said estimation part for every said time interval. The operation mode switching control device described.
The operation mode switching control device according to claim 5, wherein the state determination unit performs the determination based on a tendency that the estimated state varies from the past to the present.
The operation mode switching control device according to claim 4 or 6, wherein the determination unit further includes an adjustment unit that adjusts the time interval based on the tendency.
An operation mode switching control method executed by a device for switching a vehicle operation mode from an automatic operation mode to a manual operation mode,
An acquisition process in which the device acquires sensing data representing the driver's state from a monitoring sensor for monitoring the driver's state;
Determination that the device determines whether or not the driver is in a state where the driver can perform the driving operation in the manual driving mode based on the acquired sensing data at an arbitrary time interval, and holds the determination result Process,
An operation detection process in which the device detects an override operation by the driver based on a detection signal output from an in-vehicle sensor capable of detecting a driving operation by the driver;
When an override operation is detected by the operation detection process, the device determines the driver's condition based on a determination result corresponding to immediately before the override operation is detected among the determination results held by the determination process. An operation mode switching control method comprising: a switching signal output process for outputting a switching signal for switching the automatic operation mode to the manual operation mode when the state is a state in which the operation operation in the manual operation mode can be performed.
A program that causes a computer to function as each of the units included in the operation mode switching control device according to any one of claims 1 to 7.