JP6958886B2 - Operator state estimator - Google Patents

Description

The present invention relates to a driver state estimation device that estimates a state such as a decrease in attention of a driver who operates a maneuvering object such as an automobile.

Human errors such as casual driving are the main causes of traffic accidents. Research and development of automatic driving systems aiming at reducing such accidents are being actively carried out. The Cabinet Office is aiming to market a semi-autonomous driving system by the first half of the 2020s. In the semi-autonomous driving system, the maneuvering is switched from the fully automatic driving system to the driver (operator) when requested by the system. When switching, it is desirable to confirm that the driver is in a state of sufficient maneuvering before switching. In addition, the technique of checking the driver's state even when the driver is driving (during manual driving) is useful in terms of preventing inattentive driving and inattentive driving.

The inventors have constructed a vehicle dynamics analysis method and a control system design method that explicitly use an operator model (driver model) for manipulating a vehicle (control object) (Non-Patent Documents 1 to 5). In addition, we have developed a real-time identification method of an operator model for estimating the operator (operator) state of operating a vehicle and a method of estimating the operator state from the identification model (Non-Patent Documents 6 to 8). By using the operator model, it is possible to analyze and design by directly considering the behavior of the operator.

Further, in Patent Document 1, the convergence angle defined by the line of sight of both eyes of the driver is sequentially calculated based on the image showing the face of the driver driving the vehicle, and based on the calculated standard deviation of the convergence angle, the convergence angle is sequentially calculated. An arousal degree determination device for determining whether or not a driver who drives a vehicle is in a state of reduced arousalness is disclosed.
In Patent Document 2, RRI data, which is the interval between adjacent R waves in the ECG waveform during awakening of the driver, is acquired, heart rate variability is analyzed based on the acquired RRI data, and a standardized HRV index during awakening is obtained. Drowsiness detection methods and devices for constructing a drowsiness detection model for each driver using multivariate statistical process management based on the data are disclosed.
Patent Document 3 describes a weak identification means for binarly identifying a plurality of feature amounts indicating input biological information or vehicle information, and a driver's identification result based on the identification results of the plurality of feature amounts by the weak identification means. A driver state estimation device including a state estimating means for estimating attention reduction and a learning means for learning the relationship between a plurality of features and the driver's attention reduction using AdaBoost is disclosed.
According to Patent Document 4, when a temporary change of a specific part of a face continues for a predetermined time or longer, or when the number of temporary changes within a predetermined cycle is equal to or longer than a predetermined number of times, the driver is in a vague state. A driver state detection method and an apparatus for determining the above are disclosed.

Japanese Unexamined Patent Publication No. 2012-113450 Japanese Unexamined Patent Publication No. 2015-226696 Japanese Unexamined Patent Publication No. 2009-301367 Japanese Unexamined Patent Publication No. 2016-71577

Fujinaga, J., Tokutake, H.and Miura, Y .: Pilot-in-the-Loop Analysis of Aileron Operation and Flight Simulator Experiments, Transactions of the Japan Society for Aeronautical and Space Sciences, vol.50, p.193-200 (2007) Miura, Y., Tokutake, H.and Fukui, K .: Handling qualityies evaluation method based on actual driver characteristics, Vehicle System Dynamics, vol.45, p.807-817 (2007) Tokutake, H., Fujinaga, J. and Miura, Y .: Lateral-Directional controller design using a pilot model and flight simulator experiments. The Aeronautical Journal, vol.112, p.213-218 (2008). Tokutake, H., Miura, Y. and Okubo, H .: Workload analysis method via optimal drive model, SAE Technical Papers (2004), 2004-01-3536, doi: 10.4271 / 2004-01-3536 Tokutake, H., Sato, M: Controller design using standard operator model, Journal of Guidance, Control and Dynamics, vol. 28, p.872-877 (2005) Tokutake, H., Sugimoto, Y., and Shirakata, T .: Real-time Identification Method of Driver Model with Steering Manipulation, Vehicle System Dynamics, vol.51, p.109-121 (2012). Tokutake, H .: Drowsiness Estimation from Identified Driver Model, Systems Science & Control Engineering, vol.3, p.381-390 (2015). Shoichiro Teranishi, Hiroshi Tokutake: Proceedings of the 53rd Airplane Symposium, 3G05, (2015).

Since the method for estimating the operator state proposed in Non-Patent Documents 6 to 8 uses only the vehicle motion and the maneuvering history, it can be implemented and operated at a lower price than the driver state estimation method using biological signals that is generally performed. .. However, since the driver does not steer during automatic driving, the method using the maneuvering history can be applied only to the situation where the driver is maneuvering.
Further, Patent Documents 1 to 4 determine the driver's condition by using only biological signals such as convergence angle, heart rate variability, eye movement amount, or change in mouth circumference, and the driver gazes around the vehicle. It is not possible to know how much attention is being paid to the object to be monitored (object to be monitored).

Therefore, the present invention can estimate the state of the operator regardless of whether the vehicle is manually driven or automatically driven, and can grasp how much the driver is paying attention to the monitored object around the object to be operated. It is an object of the present invention to provide an estimation device.

Operator state estimating apparatus of the present invention according to claim 1, a pilot state estimating device for estimating the attention low under status of operator steering the steering object, and the object to be monitored object position measuring means, The gaze point measuring means, the operator model identification means, and the operator state estimation means are provided, and the monitored object position measuring means continuously measures the position of the monitored object monitored by the operator, and the above-mentioned The gazing point measuring means continuously measures the gazing point of the operator who monitors the monitored object, and the operator model identifying means continuously adjusts the input / output relationship of the operator to the monitored object. as input Do displacement, said pilot model to identify showing the continued displacement of the fixation point of the operator as an output, the operator state estimation means, a constant number when the pilot model, the pilot constant number when the previous pilot model, or as compared to the constant number when the normative pilot model, the time constant of the pilot model is, the time constant of a previous pilot model of the operator, or the If it is larger than the time constant of the normative operator model, it is presumed that the current operator is in the state of reduced attention.
According to the second aspect of the present invention, in the operator state estimation device according to the first or second aspect, the operator model identification means means that the viewpoint of the operator is within a predetermined range from the monitored object. characterized by using the continuous displacement of the continuous displacement and the gaze point of the operator of the monitored object when it is in the identification of the pilot model.
According to the third aspect of the present invention, in the driver state estimation device according to the first or second aspect, the maneuvering object is a vehicle traveling on a road, and the monitored object is outside the vehicle. It is characterized by being a stationary object or a moving object existing in.
The present invention according to claim 4 is the driver state estimation device according to claim 3, wherein the vehicle has a manual driving mode in which the driver operates and all or a plurality of operations of acceleration, steering and braking. It is a semi-automatic driving system vehicle having an automatic driving mode that is automatically performed, and the driver state estimating means estimates the driver's attention reduction state before switching between the manual driving mode and the automatic driving mode. It is characterized by doing.

The present invention models the driver's viewpoint movement during traveling by utilizing the surrounding environment information (displacement of the monitored object) and the driver's (operator's) gaze movement, and the obtained driver model (driver's model). The driver state (driver state) is estimated from the feature amount of. Therefore, according to the present invention, the state of the driver can be estimated regardless of whether the vehicle is manually driven or automatically driven, and how much attention the driver is paying attention to which monitored object around the own vehicle (control object). It is possible to provide a driver state estimation device that can be grasped.

Schematic configuration of a human error prevention system using the operator state estimation device according to an embodiment of the present invention. Diagram showing the driver's gaze movement model Diagram showing a closed-loop system consisting of a driver driving a vehicle, a vehicle, and the environment. Schematic diagram of experimental equipment Conversion diagram of the vehicle position in front to the xy coordinate system The figure which shows the average value and standard deviation of the pupil diameter of each subject during automatic driving The figure which shows (a) residual, (b) gain, (c) time constant of each subject obtained from the driver model identified from the data during automatic driving. The figure which shows (a) residual, (b) gain, (c) time constant of each subject obtained from the driver model identified from the data during automatic driving. The figure which shows (a) residual, (b) gain, (c) time constant of each subject obtained from the driver model identified from the data during automatic driving. The figure which shows (a) residual, (b) gain, (c) time constant of each subject obtained from the driver model identified from the data during automatic driving. The figure which shows (a) residual, (b) gain, (c) average value and standard deviation of time constant at the time of automatic running of each subject. The figure which shows the relationship between the reaction time of a driver and the average value of the time constant obtained in 20 seconds before switching of operation with and without a secondary task.

The operator state estimation device according to the first embodiment of the present invention includes a monitored object position measuring means, a gazing point measuring means, a driver model identification means, and a driver state estimating means, and includes a monitored object. The position measuring means continuously measures the position of the monitored object monitored by the operator, and the gazing point measuring means continuously measures the gazing point of the operator who monitors the monitored object to identify the operator model. It means the input-output relationship of the steering longitudinal's inputs the continuous displacement of the monitored object, to identify the pilot model showing the continued displacement of the focus point of the operator as the output, operator condition estimating means is a constant number when the pilot model, a constant number when the previous pilot model of steering longitudinal person or compared with a constant number when the normative pilot model, the time constant of the pilot model, pilot If it is greater than the time constant of the previous operator model or the time constant of the normative operator model, it is presumed that the current operator is in a state of reduced attention. According to the present embodiment, in order to estimate the driver's state from the driver's model using the gaze movement of the driver with respect to the monitored object, the driver's state is estimated regardless of whether the driver is in manual driving or automatic driving. It is possible to grasp which monitored object around the own vehicle and how much the driver is paying attention to.
A second embodiment of the present invention is the operator state estimation device according to the first embodiment, wherein the operator model identification means is a displacement when the operator's gaze point is within a predetermined range from the monitored object. The continuous displacement of the watch and the continuous displacement of the operator's gaze point are used to identify the operator model. According to the present embodiment, the data when the gazing point is out of the predetermined range from the monitored object is not used for the identification of the operator model, so that the modeling can be performed more accurately.
A third embodiment of the present invention is the operator state estimation device according to the first or second embodiment, in which the maneuvering object is a vehicle traveling on a road and the monitored object is outside the vehicle. It is an existing stationary or moving object. According to this embodiment, it can be applied to a vehicle (automobile). In addition, by setting the monitored object as a stationary object such as another vehicle, a traffic light, a sign, and a lane around the own vehicle, or a moving object, it is possible to grasp which monitored object around the own vehicle and how much attention the driver is paying attention to.
A fourth embodiment of the present invention is the driver state estimation device according to the third embodiment, in which the vehicle has a manual driving mode in which the driver operates and all or a plurality of operations of acceleration, steering and braking. Is a semi-automatic driving system vehicle having an automatic driving mode in which is automatically performed, and the driver state estimation means estimates the driver's attention reduction state before switching between the manual driving mode and the automatic driving mode. be. According to the present embodiment, since the state of the operator is estimated before the mode is switched, it is possible to reduce accidents when switching from automatic driving to manual driving.

Hereinafter, the operator state estimation device according to an embodiment of the present invention will be described.
FIG. 1 is a schematic configuration diagram of a human error prevention system using the operator state estimation device according to the present embodiment.
The driver state estimation device is mounted on a vehicle (control object) equipped with a semi-automatic driving system. Here, the "quasi-automatic driving system" has a manual driving mode in which the operator performs all acceleration, steering and braking, and an automatic driving mode in which at least one of acceleration, steering and braking is automatically performed. However, it refers to a system that switches between manual driving mode and automatic driving mode under predetermined conditions.
The operator state estimating device includes a monitored object position measuring means 10, a gazing point measuring means 20, a driver model identifying means (driver model identifying means) 30, and a driver state estimating means (driver state estimating means) 40. Be prepared.
The monitored object position measuring means 10 continuously measures the position of the monitored object monitored by the driver (operator) and transmits the position to the driver model identification means 30. The monitored object is a stationary object or a moving object outside the vehicle, such as another vehicle, a traffic light, a pedestrian, a sign, or a lane, to which the driver should direct his / her line of sight when maneuvering his / her own vehicle.
The gazing point measuring means 20 continuously measures the gazing point by tracking the line of sight of the driver who monitors the monitored object, and transmits the gazing point to the driver model identification means 30.
The driver model identification means 30 inputs the input / output relationship of the driver by inputting the displacement of the monitored object obtained based on the position of the monitored object transmitted from the monitored object position measuring means 10, and from the gazing point measuring means 20. A driver model showing the displacement of the driver's gaze point obtained based on the transmitted driver's gaze point is identified as an output, and the identified driver model is transmitted to the driver state estimation means 40.
The driver state estimating means 40 compares the features of the driver model identified by the driver model identifying means 30 with the features of the driver's previous driver model or the normative driver model. , Estimate the current state of the driver. The feature amount used for comparison is not particularly limited, but is preferably gain, time constant, or residual. In this case, for example, the average value of the time constants of the driver model identified by the driver model identification means 30 for a predetermined time is compared with the average value of the time constants of the normative driver model stored in the storage means 60 for a predetermined time. By doing so, it is estimated whether or not the driver's attention is reduced. The driver model identified by the driver model means 30 is also transmitted to the storage means 60 and saved, and the feature amount of the latest driver model is compared with the feature amount of the saved previous driver model. It may be estimated whether or not there is a decrease in attention. In addition, the gain, time constant, and residual may be compared with other driver models, and the presence or absence of a decrease in attention may be comprehensively estimated.
Further, the human error prevention system according to the present embodiment includes the warning / maneuvering intervention means 50. Warning / The maneuvering intervention means 50 issues a warning when the driver state estimating means 40 estimates that the driver's attention is low. Further, when in the automatic driving mode, the automatic driving mode is continued without switching to the manual driving. This makes it possible to prevent accidents caused by human error such as inadvertent driving.
In this way, since the driver state is estimated from the driver model using the gaze movement of the driver with respect to the monitored object, the driver state can be estimated regardless of whether the driver is in manual driving or automatic driving, and the driver himself / herself. It is possible to grasp which monitored object around the vehicle and how much attention is paid to it.

Further, the driver model identification means 30 uses the displacement of the monitored object and the displacement of the driver's gazing point when the gaze point of the driver is within a predetermined range from the monitored object to identify the driver model.
The driver's gaze point data measured by the gaze point measuring means 20 may include the movement of the gaze point when the monitored object is not being monitored. Therefore, the data when the gazing point is outside the predetermined range from the monitored object is not used for the identification of the driver model, so that the modeling can be performed more accurately.

本実施例では解析を簡単にするため１次遅れ系をドライバモデルとして採用する。

（１）、（２）式で表現されるドライバモデルからドライバ状態を推定する。そのため残差の大きさＪres（式（３））、ゲインＫ、時定数Τを評価量として採用する。

ｉはデータセット番号、ｔ_ｉはｉ番目の切り分けたデータセットの開始時間、Ｌ_ｉはｉ番目の切り分けたデータセットの時間幅である。残差からは線形ドライバモデルで表現できない注視点 移動の非線形要素や周辺環境の変化と相関のない注視点移動量、注視点分布の範囲の大きさを評価できる。 Next, the gaze movement model of the driver will be described.
FIG. 2 is a diagram showing a gaze point movement model of the driver. The driver model identification means 30 identifies a driver model that indicates the input / output relationship of the driver by inputting the surrounding environment (displacement of the monitored object) and outputting the displacement of the gaze point of the driver.
The driver detects changes in the surrounding environment by moving the gazing point during automatic driving. The behavior of the driver is expressed by the following model.

In this embodiment, a first-order lag system is adopted as a driver model in order to simplify the analysis.

The driver state is estimated from the driver model expressed by the equations (1) and (2). Therefore, the magnitude of the residual Jres (Equation (3)), the gain K, and the time constant Τ are adopted as the evaluation quantities.

i is the data set number, t _i is the start time of the i-th isolate the data set, L _i is the time width of the i-th isolate the data set. From the residuals, it is possible to evaluate the non-linear elements of gazing point movement that cannot be expressed by the linear driver model, the gazing point movement amount that does not correlate with changes in the surrounding environment, and the size of the gazing point distribution range.

Next, the relationship between attention loss and the driver model will be described.
FIG. 3 shows a closed loop system consisting of a driver driving a vehicle, a vehicle, and an environment (object to be monitored). When maneuvering, the driver recognizes changes in the surrounding environment and decides how to maneuver. Then, during manual driving, maneuvering is performed, and during automatic driving, the surrounding environment is monitored. When the driver's attention is reduced, recognition delays and mistakes occur, and the driver makes mistakes in judgment and operation.
In the existing research, it has been experimentally confirmed that the driver's response to the stimulus presentation is delayed when the driver's attention is reduced in the simulated driving situation (Nobuyuki Uchida et al .: Evaluation of the driver's attention state during mobile phone conversation, IATSS Review, vol.30, No.3, p.57-65 (2005).). Therefore, the time constant of the driver model, which expresses the time delay of the gaze movement with respect to the change in the surrounding environment, increases due to the decrease in attention. In addition, the maneuvering performance immediately after switching to manual operation can be predicted from the model features.

Next, an experiment using the operator state estimation device of the present invention will be described.
In this experiment, the driver is monitored for the surrounding environment and at the same time is given a mental load due to a secondary task to simulate a state of reduced attention. The fluctuation of the pupil diameter is used as a reference to confirm whether the driver is mentally loaded.
The pupil is controlled by the parasympathetic nerve, which controls the sphincter muscle, and by the sympathetic nerve, which controls the dilator muscle (Kenji Ito, Sonoko Kuwano, Akitetsu Komatsubara: Ergonomics Handbook, Tokyo, Asakura Shoten, 2003, p.363.). Therefore, the pupil diameter is an objective index showing a person's psychological state (Yamanaka, K. and Kawakami, M .: Convenient Evaluation of Mental Stress with Pupil Diameter, International Journal of Occupational Safety and Ergonomics, vol.15, No. 4, p.447-450 (2009).). In this experiment, from the measurement data of the pupil diameter, the point where the pupil diameter changed by 0.1 mm in 1/30 second (Sandra, PM: US Patent, US6090051A, (2000).), And the pupil diameter ranged from 2 mm to 8 mm. The excess points (Industrial Technology Research Institute, Human Welfare and Medical Engineering Laboratory: Human Measurement Handbook, Tokyo, Asakura Shoten, 2003, p.113-115.) Are removed and used as an index of mental load.
In this experiment, the pupil diameter was measured using the eye tracker FOVIO manufactured by Seeing Machines, and the pupil diameter was used as an index of mental load.

FIG. 4 shows an outline of the experimental device. The driver X sees the image projected on the screen 2 from the projector 1, monitors the surrounding environment during automatic driving, and operates by using the steering 3 and the pedal 4 during manual driving. The pedal 4 includes an accelerator pedal 4A and a brake pedal 4B. There is no force feedback for steering 2. A non-contact eye tracker (FOVIO manufactured by Seeing Machines) is used as the gazing point measuring means 20 for measuring the gazing point and the pupil diameter. The driving simulator was constructed using UC-win / Road of FORUM8 Co., Ltd. Using the UC-win / Road SDK and Delphi XE2, it is possible to program experimental conditions such as arbitrary maneuvering target dynamics and disturbances. The computer 5 operates and controls the driving simulator.
The monitored object position measuring means 10, the driver model identifying means 30, and the driver state estimating means 40 are not shown.

In this experiment, the driver gazes only at the meandering vehicle in front. To further simplify the analysis, the driving environment was set to a straight road with a constant height. Further, the distance between the own vehicle and the vehicle in front is kept constant, and the position of the vehicle in front does not change in the depth direction and the vertical direction when viewed from the subject (driver X). Under these experimental conditions, in this experiment, only the gaze movement parallel to the road surface of the subject is targeted.
At the start of the experiment, the subject was in the driver's seat of a vehicle that was automatically traveling so as to keep the distance between the meandering vehicle in front and the vehicle constant. Approximately 120 seconds after the start of the experiment, another vehicle interrupts between the vehicle in front and the own vehicle. At the same time as the interrupting vehicle appears on the screen 2, the automatic driving is switched to the manual driving, and then the subject controls for 60 seconds and the experiment ends.

In this experiment, in order to model the gaze movement of the driver with respect to the movement of the vehicle in front, the subject was instructed to constantly monitor the vehicle in front as a main task. In addition, in order to simulate the driver's attention loss state, a mental load was given by performing a secondary task during automatic driving. The secondary task was given an N-back task of dictating the Nth number from the end of the numbers that flow continuously. Furthermore, the subject was instructed to step on the brake pedal 4B as soon as he / she recognized the interrupting vehicle.

The own vehicle automatically travels by feedback control so that the distance between the vehicle and the meandering vehicle in front becomes constant. Even if the steering 3 or pedal 4 is operated during automatic driving, it is not reflected in the own vehicle. The vehicle in front meanders randomly so that the subject cannot predict the next movement. Since the interrupting vehicle interrupts at a speed 20 km / h higher than that of the own vehicle, the interrupting vehicle does not collide unless the subject improperly steers.

We asked four subjects who fully explained the content and purpose of this experiment in advance and obtained an agreement to participate in the maneuvering experiment. After sufficient maneuvering practice, a maneuvering experiment was conducted in which the main task and the sub-task were given. The answer to the secondary task was verbally stated and heard by the experimental assistant.
Further, the measured gaze point position and the position of the vehicle in front are converted into the xy coordinate system as shown in FIG. 5 and used for model identification. This is a coordinate system that moves forward at a constant speed so that the vehicle in front is always present in the xy plane.

In this experiment, the subject was instructed to step on the brake pedal 4B immediately after recognizing the interrupting vehicle. Therefore, the time from when the interrupting vehicle is displayed on the screen 2 to when the brake pedal 4B is depressed (hereinafter referred to as "driver's reaction time" or "reaction time") is used as the maneuvering result. The shorter the reaction time of this driver, the better the maneuvering performance.

出力ｘ_δ，ｙ_δは平均値を除去した注視点の位置、入力ｘ_e，ｙ_ｅは平均値を除去した前方車両の位置、ｗ_ｘ，ｗ_ｙは残差である。前方 車両は主に左右に移動するためＨ_ｘ，ｗ_ｘを解析対象とし、ドラ イバモデルＨ_ｘのゲインおよび時定数、ｘ方向の残差ｗ_ｘの時間変動を確認する。
また、得られたデータには前方車両を監視していないときの注視点の移動も含まれている。そこで注視点と前方車両が５ｍ以上離れているデータは前方車両の監視を行っていないと見なしモデル同定に利用しない。５ｍ以上離れていないデータは１０秒間隔で切り分け、１０秒未満であっても２秒以上のデータは同定に用いた。 Next, data processing will be described.
The driver model is shown below.

Output x _[delta], y _[delta] is the gazing point removal of the average value position, the input x _e, the y _e position of the forward vehicle removing the mean value, the w _x, w _y is the residual. Since the vehicle in front mainly moves left and right, H _x and w _x are analyzed, and the gain and time constant of the _{driver model H x} and the time variation _{of the residual w x in the x direction are confirmed.}
The obtained data also includes the movement of the gazing point when the vehicle in front is not being monitored. Therefore, the data in which the gazing point and the vehicle in front are separated by 5 m or more are not used for model identification because it is considered that the vehicle in front is not monitored. Data not separated by 5 m or more were separated at 10-second intervals, and data of 2 seconds or more was used for identification even if it was less than 10 seconds.

FIG. 6 shows the average value and standard deviation of the pupil diameter of each subject during automatic driving.
(A) residual of each subject obtained from the driver model H _x identified from the data in the automatic traveling in FIG 7 to 10, (b) gain, indicating the time constant (c). The horizontal axis is the elapsed time [seconds], and as described above, automatic driving is performed until about 120 seconds have passed from the start of the experiment, and then manual driving is performed. FIG. 7 is for subject A, FIG. 8 is for subject B, FIG. 9 is for subject C, and FIG. 10 is for subject D.
FIG. 11 shows (a) residuals, (b) gains, (c) average values of time constants during automatic driving, and standard deviations of each subject. However, when the identification model was unstable, it was excluded from the analysis because it did not behave in a correlation with the input.
In FIGS. 6 to 11, the solid line indicates the case where there is no secondary task, and the broken line indicates the case where there is a secondary task.
Table 1 shows the reaction time [seconds] for each subject with and without the secondary task, and Table 2 shows the correct answer rate [%] for the secondary task.

From FIG. 6, it can be seen that the pupil diameters of subjects A, B, and D increased due to the secondary task during automatic driving, and that the secondary task gave a mental load. In addition, from Table 1, it can be seen that the reaction time when all three subjects performed the secondary task increased, and the attention decreased due to the secondary task. Although no increase in pupil diameter was confirmed in subject C, the reaction time increased by 0.36 seconds, and it can be estimated that subject C was also in a state of decreased attention due to a secondary task.
It should be noted that the model features of subject B when there is a secondary task have not been obtained after about 80 seconds from the start of the experiment (see FIG. 8). This is because the gazing point is often 5 m or more away from the vehicle in front, and the model was not identified.
In 3 of the 4 subjects, A, C, and D, the time constant was larger than when there was no secondary task (see FIG. 11 (c)). The influence of the secondary task on the time constant of subject B is not seen. In the experiment in which the secondary task of subject B was given, the gazing point was often separated from the vehicle in front by 5 m or more with the passage of time, and an identification model could not be obtained. Therefore, it is presumed that the result was that there was not much change because the average value was calculated only with the identification model that was not so affected by the secondary task immediately after the start of the experiment.
FIG. 12 shows the relationship between the reaction time of the driver and the average value of the time constants obtained in 20 seconds before switching the operation with and without the secondary task. “●” indicates that subject A has a secondary task, “○” indicates that subject A does not have a secondary task, “▲” indicates that subject C has a secondary task, and “△” indicates subject C's secondary task. When there is no next task, "■" is the case where the subject D has a secondary task, and "□" is the case where the subject D has no secondary task. Subject B did not plot because the model could not be obtained 20 seconds before the operation switching when there was a secondary task. From FIG. 12, it can be seen that in subjects A, C, and D, there is a positive correlation between the time constant during automatic driving and the reaction time of the driver. From this, it can be seen that when the time constant of the driver model, which represents the movement of the gazing point during automatic driving, is large, it can be predicted that the driver's attention is low. Since it is clear that the movement of the gazing point is strongly influenced by the secondary task also in subject B, a prediction algorithm can be constructed.

In the above description, the case where the vehicle is mounted on a vehicle equipped with a semi-automatic driving system has been described as an example, but the driver state estimation device according to the present invention can also be applied to a vehicle only for manual driving without a semi-automatic driving system. Is. Furthermore, it can be applied to other vehicles such as aircraft, motorcycles, ships, and railroads, or unmanned aerial vehicles (so-called "drones") that can be remotely controlled.
Further, the above equations (1) to (4) are examples for facilitating the understanding of the present invention, and can be variously modified without departing from the spirit of the present invention.

The driver state estimation device according to the present invention models the viewpoint movement of the person who controls the maneuvering object, and effectively estimates the state such as the driver's attention reduction based on the obtained feature amount of the driver model. By applying it to automobiles, etc., it contributes to the prevention of accidents due to human error of the operator.

1 Projector 2 Screen 3 Steering 4 Pedal 5 Computer 10 Observed object position measuring means 20 Gaze point measuring means 30 Driver model identification means (operator model identification means)
40 Driver state estimation means (driver state estimation means)
50 Warning / Maneuvering intervention means 60 Memory means X driver (operator)

Claims (4)

It is a driver state estimation device that estimates the attention reduction state of the driver who operates the maneuvering object.
Measured object position measuring means and
Gaze point measuring means and
Operator model identification means,
Equipped with a driver state estimation means
The monitored object position measuring means continuously measures the position of the monitored object monitored by the operator.
The gaze point measuring means continuously measures the gaze point of the operator who monitors the monitored object.
The driver model identification means obtains a driver model in which the input / output relationship of the driver is input by the continuous displacement of the monitored object and the continuous displacement of the gaze point of the driver is displayed as an output. Identify and
The operator state estimation means, a constant number when the pilot model, compared with the constant number when a constant number or normative pilot model, when the previous pilot model of the pilot, the pilot If the time constant of the model is greater than the time constant of the operator's previous operator model, or the time constant of the normative operator model, it is presumed that the current operator is in the diminished attention state. An operator state estimation device characterized by doing so. The operator model identification means is a continuous displacement of the monitored object when the gaze point of the operator is within a predetermined range from the monitored object and a continuous displacement of the gaze point of the operator. The operator state estimation device according to claim 1, wherein is used for identifying the operator model. The maneuvering object is a vehicle traveling on the road.
The operator state estimation device according to claim 1 or 2, wherein the monitored object is a stationary object or a moving object existing outside the vehicle. The vehicle is a semi-automatic driving system vehicle having a manual driving mode in which the driver drives and an automatic driving mode in which at least one of acceleration, steering and braking is automatically performed.
The driver state estimation device according to claim 3, wherein the driver state estimating means estimates the attention-reduced state of the driver before switching between the manual operation mode and the automatic driving mode. ..

Images