JP2022071595A - Proper driving state determination device and proper driving state determination program - Google Patents

Abstract

To accurately determine a proper driving state of a driver.SOLUTION: Data representing a viewing motion which is a motion of visual line of a driver of a vehicle are acquired by a viewing action acquisition unit 76. For each of a plurality of indexes relating to a driving action of the driver, an index value of the index is estimated from the acquired data representing the viewing action while using a driving action prediction model by a proper state level calculation unit 86. For each of the plurality of indexes, a proper driving state of the driver is then determined on the basis of the estimated index value by the proper state level calculation unit 86.SELECTED DRAWING: Figure 7

Description

The present invention relates to an operation appropriate state determination device and an operation appropriate state determination program, and more particularly to an operation appropriate state determination device and an operation appropriate state determination program for determining an operation appropriate state of a driver.

Conventionally, Patent Document 1 includes a database that stores time-series data of the line of sight in association with driving motion, and the movement of the line of sight of the driver detected by the line-of-sight detection unit and the time-series data of the line of sight stored in the database. Disclosed is a caution behavior detection technique for determining whether or not a driver has performed a line-of-sight movement associated with a driving motion by collating with.

In Patent Document 2, the behavioral characteristics of the driver are obtained based on the visual behavior information of the driver and the vehicle operation information, and the driving skill of the driver is evaluated according to the comparison result with the model data, and the driver is driven to the external environment. A driver emotion estimation technique for estimating emotions such as a person having fun or anxiety is disclosed.

In Patent Document 3, the current driving environment of the driver is described by accumulating data on the vehicle behavior and the operation of the driver, performing clustering, and creating a model for estimating the driving tendency of the driver for each cluster in advance. Disclosed is a technique for determining which cluster best fits and estimating the driver's current operating status using an estimation model in that cluster.

Patent Document 4 discloses a technique for switching to a forced automatic operation mode when the driver is looking away for a certain period of time.

Japanese Unexamined Patent Publication No. 2017-100562 Japanese Patent Application Laid-Open No. 2015-84253 Japanese Unexamined Patent Publication No. 2013-178827 Japanese Unexamined Patent Publication No. 2019-40630

In Patent Document 1, even if the driver's attention behavior can be detected, the driver's driving behavior cannot be evaluated because the suitability of the driving behavior to be performed thereafter and the relationship with the driving behavior are not shown.

The above-mentioned Patent Document 2 aims to estimate the current driving skill of a driver based on visual behavior information and vehicle operation information, but is actually a technique assuming parking behavior in a parking lot. There is no specific mention of other situations. In addition, since it is intended for normal driving, the effects of distraction and drowsiness other than driving skill are not taken into consideration. Therefore, it cannot be directly applied to the driving behavior for avoiding danger such as acceleration / deceleration and lane change in the presence of front / rear vehicles and side vehicles performed by drivers in various states on the road. Furthermore, situations such as switching from the automatic operation state to the manual operation are also unexpected.

In the above Patent Document 3, the driver's driving tendency is estimated only from the data related to the vehicle behavior and the driver's operation. Although the driving tendency when the driver is normal can be understood from these data, the degree of concentration and awareness of the driver's driving are known. Since the input data that can determine the level etc. is not included, it is not possible to determine how properly the driver can drive at this point as it is.

Further, in Patent Document 3, a model is created for each cluster determined by clustering, but such clustering may not be effective. Considering the relationship between the driving lane position of the own vehicle and the line-of-sight distribution as an example, when driving in the leftmost lane of a multi-lane road, it is almost unnecessary to look at the left mirror, so peripheral information necessary for driving. As the information in the left mirror is not useful. In particular, left mirror viewing for a certain period of time or longer is equivalent to looking aside. On the other hand, when traveling in the middle lane of multiple lanes, it is necessary to check the adjacent lanes on the left and right. In these cases, if the line-of-sight distribution is simply clustered, if the left mirror view is included for a certain period of time or more when driving in the leftmost lane, the cluster will be the same as when paying attention to the left and right lanes when driving in the middle lane. Because it is classified, it is not possible to separate the inappropriate driving state (when driving in the leftmost lane) and the appropriate state (when driving in the middle lane).

In Patent Document 4 above, it is determined that it is unsafe if the driver looks away for a certain period of time, but it is not possible to quantitatively evaluate how much danger occurs when looking aside, so it is simply All the looking away states for a certain period of time or longer have been determined to be inappropriate for manual operation, and only extremely conservative determinations have been made.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an operation appropriate state determination device and an operation appropriate state determination program capable of accurately determining an operation appropriate state of a driver.

In order to achieve the above object, the driving suitability determination device according to the present invention has a visual behavior acquisition unit that acquires data representing visual behavior that is the movement of the driver's line of sight of a vehicle, and a plurality of indexes related to the driving behavior of the driver. For each of the above, a driving behavior estimation unit that estimates an index value of the index from the acquired data representing the visual behavior using a model in which the data representing the visual behavior and the index are associated with the index, and the plurality of the above. Each of the indicators is configured to include a driving appropriate state determining unit for determining the driving appropriate state of the driver based on the index value estimated by the driving behavior estimation unit.

Further, in the driving suitability determination program according to the present invention, the computer is used for each of a visual behavior acquisition unit that acquires data representing visual behavior, which is the movement of the driver's line of sight, and a plurality of indicators related to the driver's driving behavior. The driving behavior estimation unit that estimates the index value of the index from the acquired data representing the visual behavior using the data representing the visual behavior and the model associated with the index, and each of the plurality of indexes. Is a program for functioning as a driving appropriate state determination unit for determining a driving appropriate state of the driver based on the index value estimated by the driving behavior estimation unit.

According to the present invention, the visual behavior acquisition unit acquires data representing visual behavior, which is the movement of the line of sight of the driver of the vehicle. For each of the plurality of indicators related to the driver's driving behavior by the driving behavior estimation unit, the index is obtained from the acquired data representing the visual behavior using a model in which the data representing the visual behavior and the index are associated with each other. Estimate the index value of. Then, the driving appropriate state determination unit determines the driving appropriate state of the driver for each of the plurality of indexes based on the index value estimated by the driving behavior estimation unit.

As described above, for each of the plurality of indexes related to the driver's driving behavior, the index of the index is obtained from the acquired data representing the visual behavior by using the model in which the data representing the visual behavior and the index are associated with each other. By estimating the value and determining the driving appropriate state of the driver for each of the plurality of indicators based on the estimated index value, the driving appropriate state of the driver can be accurately determined.

As described above, according to the operation appropriate state determination device and the operation appropriate state determination program of the present invention, it is possible to obtain the effect that the operation appropriate state of the driver can be accurately determined.

It is a schematic diagram which shows the structure of the operation appropriate state determination apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the learning part of the operation appropriate state determination apparatus which concerns on embodiment of this invention. (A) It is a figure which shows the example of the visual field area about a line-of-sight vector, and (B) is the figure which shows the example of the visual field area about face orientation. It is a figure for demonstrating the learning method by a model learning part. It is a figure which shows the example of the vehicle locus. It is a figure for demonstrating the calculation method of the vehicle locus deviation amount. It is a block diagram which shows the structure of the operation appropriate state determination part of the operation appropriate state determination device which concerns on embodiment of this invention. It is a figure for demonstrating the operation appropriate state determination method by the operation appropriate state determination unit. It is a figure for demonstrating the calculation method of the operation appropriate state level by an operation appropriate state determination part. It is a flowchart which shows the content of the model learning processing routine in the operation appropriate state determination apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the operation appropriate state determination processing routine in the operation appropriate state determination apparatus which concerns on embodiment of this invention. It is a flowchart which shows the content of the processing routine which calculates the operation appropriate state level in the operation appropriate state determination apparatus which concerns on embodiment of this invention. It is a graph which shows the prediction example of a reaction time. It is a graph which shows the example of the line-of-sight and face orientation distribution at time A. It is a graph which shows the example of the gaze time rate at time A. It is a graph which shows the example of the line-of-sight and face orientation distribution at time B. It is a graph which shows the example of the gaze time rate at time B.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a case where the present invention is applied to a driving suitability determination device mounted on a vehicle and determining a driving suitability state of a driver will be described as an example.

<Configuration of operation suitability determination device>
As shown in FIG. 1, the operation appropriate state determination device 10 according to the embodiment of the present invention includes a warning detection unit 12, a physiological amount sensor 14, a line-of-sight measuring instrument 16, an operation detector 18, and a position measuring instrument. The 20th, the computer 30, and the output unit 32 are provided.

The computer 30 can be composed of a computer including a CPU, a RAM, a ROM for storing a program for executing a model learning processing routine and an operation appropriate state determination processing routine described later, and various data. Functionally, as shown in FIG. 1, the computer 30 includes a learning unit 40, a model storage unit 62, and an operation appropriate state determination unit 70.

The warning detection unit 12 detects a warning issued to the driver. For example, in manual driving, an alarm issued for a danger such as a sudden traffic jam in front or an accident is detected. In addition, during automatic operation, automatic operation becomes difficult due to the same danger, and it becomes necessary to switch from the automatic operation state to manual operation, and a manual switching request (TOR: Take) is issued from the automatic operation system. Over Request) is detected.

The physiological amount sensor 14 measures the driver's heart rate, heart rate variability (RRI), LF / HF (ratio of low frequency power spectrum / high frequency power spectrum of heart rate variability), and skin conductance (index of sweating state).

The line-of-sight measuring device 16 measures the driver's head position / face direction and line-of-sight direction.

The operation detector 18 detects the driver's steering operation, accelerator operation, and brake operation.

The position measuring instrument 20 measures the position of the own vehicle and the positions of other vehicles in the vicinity.

The learning unit 40 collects a combination of a plurality of indicators related to driving behavior acquired at the time of warning, data representing visual behavior acquired immediately before the warning, and a physiological amount, and based on the collected results, each of the plurality of indicators. Learn the driving behavior prediction model for.

As shown in FIG. 2, the learning unit 40 includes a physiological amount acquisition unit 42, a physiological amount database 44, a visual behavior acquisition unit 46, a visual behavior database 48, an input data organizing unit 50, a driving behavior acquisition unit 52, and a driving behavior database 54. , A lane determination unit 56, a model switching information database 58, and a model learning unit 60. The physiological amount acquisition unit 42, the physiological amount database 44, the visual behavior acquisition unit 46, the visual behavior database 48, the input data organizing unit 50, the driving behavior acquisition unit 52, and the driving behavior database 54 are examples of the learning data collection unit. be.

The physiological amount acquisition unit 42 acquires the physiological amount of the driver immediately before the warning is detected from the physiological amount sensor 14. Specifically, the physiological amount acquisition unit 42 determines the driver's physiological amount (driver's heart rate, heart rate variability (RRI), LF / HF (] immediately before the warning is detected by the warning detection unit 12 from the physiological amount sensor 14. Time-series data of heart rate variability low frequency power spectrum / high frequency power spectrum ratio) and skin conductance (index of sweating state)) are acquired and stored in the physiological amount database 44. The physiological amount database 44 outputs the average value of each physiological amount in the most recent fixed time.

The visual behavior acquisition unit 46 acquires data representing the visual behavior, which is the movement of the driver's line of sight immediately before the warning is detected, from the line-of-sight measuring instrument 16. Specifically, the visual behavior acquisition unit 46 obtains time-series data of the driver's head position / face direction and the line-of-sight direction immediately before the warning is detected by the warning detection unit 12 from the visual behavior measuring device 16. Accumulate in database 48. The visual behavior database 48 is based on each visual field region (FIG. 3) in which the driver's visual field is divided into a plurality of regions, and time-series data of the driver's head position / face direction and line-of-sight direction within the latest fixed time. The gaze time rate, which indicates the ratio of the gaze time for each visual field area, is calculated and output.

FIG. 3A shows an example in which a plurality of visual field regions related to the line-of-sight direction are a front region, a rearview mirror region, a left mirror region, a right mirror region, and an instrument panel / HMI screen region. Further, FIG. 3B shows an example in which the plurality of visual field regions relating to the face orientation are the front region and the instrument panel / HMI screen region.

The case where the gaze behavior database 48 calculates and outputs the gaze time rate has been described as an example, but the present invention is not limited to this, and a series of gaze areas, which is the visual field area in which the driver's gaze is gaze, is output. You may try to do it. For example, the sequence of the visual field region that the driver's line of sight gazes at within the most recent fixed time is obtained and output.

The input data organizing unit 50 combines the output of the physiological amount database 44 and the output of the visual behavior database 48 into one vector and outputs it to the model learning unit 60 (see FIG. 4).

The driving behavior acquisition unit 52 acquires the driver's steering operation, accelerator operation, brake operation, vehicle travel locus, and positions of other vehicles in the vicinity acquired at the time of warning from the operation detector 18. Specifically, the driving behavior acquisition unit 52 performs steering operation, accelerator operation, brake operation, vehicle travel locus, and other vehicle positions around the time when the warning is detected by the warning detection unit 12 from the operation detector 18. The time-series data of the above is stored in the driving behavior database 54.

The driving behavior database 54 calculates and outputs a plurality of indexes related to driving behavior based on the accumulated data. Specifically, the driving behavior database 54 calculates and outputs the driver reaction time, the maximum steering speed, the maximum steering angle, and the vehicle trajectory deviation amount as a plurality of indexes related to the driving behavior.

Here, the driver reaction time is the time from the time when the alarm or TOR is issued until the driver starts the driving operation for avoiding danger. Whether or not there is a driving operation to avoid danger during manual driving depends on the change from the driving behavior until just before the alarm (for example, suddenly stepping on the brake or steering operation above a certain steering angle in a short time). I can judge.

The amount of vehicle locus deviation is determined by using a DTW (Dynamic Time Warping) method (see Non-Patent Documents 1 and 2) for the vehicle locus (FIG. 5) within a certain period of time immediately after the warning or TOR is issued. It is a value obtained by evaluating the amount of deviation or shortage remaining in the lateral direction of the road after absorbing the difference in track due to the difference in vehicle speed from the reference track as much as possible by the expansion / contraction processing (FIG. 6).

FIG. 5 shows two examples of a vehicle locus which is a time series of vehicle positions indicated by a vehicle traveling direction position and a road lateral position. In FIG. 6, the vehicle locus and the reference locus are expanded and contracted in the road traveling direction, and the average value (deviation or shortage amount) of the deviation or shortage amount (gray portion in FIG. 6) remaining in the lateral direction of the road is averaged by the number of data points. Two examples of evaluating the converted value) are shown.

Since the above data is dangerous in the actual environment, it is difficult to collect it, but in the simulated environment on the driving simulator, the driver's state and driving behavior can be easily collected without danger. ..

[Non-Patent Document 1] D. J. Berndt et al., “Using Dynamic Time Warping to Find Patterns in Time Series”, AAAI Technical Report WS-94-03 (1994), pp. 359-370, AAAI.

[Non-Patent Document 2] Toyoda et al., "Efficient Pattern Detection in Data Streams", IPSJ Research Report Vol. 2012-DBS-154 No. 9 (2012), Information Processing Society of Japan.

As described above, the combination of the index value of each of the plurality of indexes acquired at the time of warning, the data representing the visual behavior acquired immediately before the warning, and the physiological amount is collected.

The lane determination unit 56 determines the traveling lane position of the own vehicle based on the position of the own vehicle when the warning is detected, and stores the determination result in the model switching information database 58. Specifically, the lane determination unit 56 has a plurality of leftmost lanes, which are the leftmost lanes among the plurality of lanes, based on the position of the own vehicle acquired by the position measuring instrument 20 when the warning is detected. Driving in one of the rightmost lane, which is the rightmost lane, the leftmost lane and the middle lane, which is not the rightmost lane, and the single lane, which is one lane on each side. Determined as lane position.

The model learning unit 60 predicts the driving behavior of the index based on the combination of the output of the input data organizing unit 50 and the index value of the index output by the driving behavior database 54 for each of the plurality of indexes. Learn the model. Here, the driving behavior prediction model associates the vector output from the input data organizing unit 50 with any index for each traveling lane position.

Specifically, as shown in FIG. 4, the driving behavior prediction model includes the driving lane information of the own vehicle determined by the lane determination unit 56 and any index (reaction time, maximum steering speed) calculated by the driving behavior database 54. , Maximum steering angle, vehicle trajectory deviation amount) is selected as a teacher signal for model output, and is prepared for each combination of input / output signals.

Each driving behavior prediction model is an index value (reaction time, maximum steering speed) of the driver driving behavior immediately after the alarm or TOR is issued, which is selected as input vector data and a teacher signal for the driving lane position of the corresponding own vehicle. , Maximum steering angle, or vehicle trajectory deviation), and model parameters are determined using statistical or machine learning techniques. Each driving behavior prediction model outputs the predicted value of each driving behavior index when an input vector is given. An example of the simplest model is a multiple regression model in which the gaze ratio of each gaze area is an explanatory variable and the index value of driving behavior is an objective variable.

The model learning unit 60 switches the driving behavior prediction model to be used based on the driving lane position (leftmost lane, intermediate lane, rightmost lane, motorcycle) of the own vehicle, and predicts each driving behavior corresponding to the driving lane position. A vector of the input data organizing unit 50 is input to the model, and learning is performed so as to associate the vector with the index value of each index.

The model parameters of the driving behavior prediction model learned for each combination of the traveling lane position and the index are stored in the model storage unit 62.

At the time of warning, the driving appropriate state determination unit 70 determines the driving appropriate state of the driver based on the data representing the visual behavior acquired immediately before the warning and the physiological quantity.

As shown in FIG. 7, the driving appropriate state determination unit 70 includes a physiological amount acquisition unit 72, a physiological amount database 74, a visual behavior acquisition unit 76, a visual behavior database 78, an input data organizing unit 80, a lane determination unit 82, and model switching. It includes an information database 84, an appropriate state level calculation unit 86, and a final appropriate state level determination unit 88. The visual behavior acquisition unit 76 and the visual behavior database 78 are examples of the visual behavior acquisition unit. The appropriate state level calculation unit 86 and the final appropriate state level determination unit 88 are examples of a driving behavior estimation unit and a driving appropriate state determination unit.

Similar to the physiological amount acquisition unit 42, the physiological amount acquisition unit 72 determines the driver's physiological amount (driver's heart rate, heart rate variability (RRI), immediately before the warning is detected by the warning detection unit 12 from the physiological amount sensor 14. Time-series data of LF / HF (ratio of low frequency power spectrum / high frequency power spectrum of heart rate variability) and skin conductance (index of sweating state) are acquired and accumulated in the physiological amount database 74. Similar to the physiological amount database 44, the physiological amount database 74 outputs the average value of each physiological amount in the latest fixed time.

Similar to the visual behavior acquisition unit 46, the visual behavior acquisition unit 76 provides time-series data of the driver's head position / face direction and line-of-sight direction immediately before the warning is detected by the warning detection unit 12 from the line-of-sight measuring instrument 16. Is stored in the visual behavior database 78. Similar to the visual behavior database 48, the visual behavior database 78 is based on time-series data of the driver's head position / face direction and line-of-sight direction for each visual field region in which the driver's visual field is divided into a plurality of regions. The gaze time rate, which indicates the ratio of the gaze time for each visual field area within the most recent fixed time, is calculated and output.

Similar to the input data organizing unit 50, the input data organizing unit 80 combines the output of the physiological amount database 74 and the output of the visual behavior database 78 into one vector and outputs the output to the appropriate state level calculation unit 86.

As described above, the data representing the visual behavior and the physiological amount acquired immediately before the warning are input to the appropriate state level calculation unit 86.

Similar to the lane determination unit 56, the lane determination unit 82 is the leftmost lane among the plurality of lanes based on the own vehicle position acquired by the position measuring instrument 20 when the warning is detected. One of the lane, the rightmost lane of multiple lanes, the leftmost lane of multiple lanes and the middle lane that is not the rightmost lane, and the single lane that is one lane on each side. It is determined as the traveling lane position of the vehicle, and the determination result is stored in the model switching information database 84.

The appropriate state level calculation unit 86 obtains the acquired visual behavior for each of the plurality of indexes related to the driver's driving behavior by using the driving behavior prediction model of the index corresponding to the driving lane position determined by the lane determination unit 82. The index value of the index is estimated from the represented data and the physiological amount.

Specifically, as shown in FIG. 8, the driving behavior prediction model of each index is selected based on the driving lane position of the own vehicle, and the input data organizing unit is used by using the driving behavior prediction model of each selected index. Using the vector generated by 80 as an input, the predicted value (any of the reaction time, the maximum steering speed, the maximum steering angle, and the vehicle trajectory deviation amount) of the index corresponding to the driving behavior prediction model is output.

Further, the appropriate state level calculation unit 86 calculates the appropriate state level of the driver for each of the plurality of indicators based on the index value of the index estimated by using the driving behavior prediction model of the index.

Specifically, the appropriate state level calculation unit 86 continuously inputs the appropriate state level of the driver at that time for each driving behavior prediction model by inputting the predicted value of the index which is the output of the driving behavior prediction model of each index. Calculated as a value or a discrete value. For example, as shown in FIG. 9, the appropriate state level of the driver is set as a continuous value, and the larger the model output value is, the lower the appropriate state level is calculated, or the larger the model output value is, the higher the appropriate state level is calculated. .. At this time, the proper state level is calculated by the saturation process so that the proper state level is in the range of 0 to 1. The upper limit and the lower limit of the saturation processing may be determined from the range of the values of the teacher data at the time of learning the driving behavior prediction model.

As shown in FIG. 8, the final appropriate state level determination unit 88 selects the minimum value (selects the lowest appropriate state level), the average value, or the median value of the calculated appropriate state level. Determine the final proper condition level. If the proper state level is a discrete value, the final proper state level may be determined by selecting a majority vote or a mode. When the lowest value is adopted, it is the safest estimated value, and when the average value, median value, majority vote, and mode are adopted, it is the average estimated value.

In this way, by comparing and evaluating the appropriate state level output using each driving behavior prediction model corresponding to multiple driving behavior indicators and determining the final appropriate state level, individual differences and physical condition of the driver can be determined. It is possible to absorb the difference in driving behavior that depends on it and always estimate an appropriate appropriate state level.

<Operation of operation suitability judgment device>
Next, the operation of the operation appropriate state determination device 10 according to the present embodiment will be described.

First, the model learning processing routine shown in FIG. 10 is executed on the computer 30 while the vehicle equipped with the driving suitability determination device 10 is running.

In step S100, the warning detection unit 12 determines whether or not a warning issued to the driver has been detected. If a warning is detected by the warning detection unit 12, the process proceeds to step S102.

In step S102, the physiological amount acquisition unit 42 acquires the physiological amount of the driver immediately before the warning is detected from the physiological amount sensor 14. The physiological amount database 44 outputs the average value of each physiological amount in the most recent fixed time.

In the next step S103, the visual behavior acquisition unit 46 acquires data representing the visual behavior, which is the movement of the driver's line of sight immediately before the warning is detected, from the line-of-sight measuring instrument 16. The visual behavior database 48 calculates and outputs a gaze time rate indicating the ratio of the time for gaze at each visual field area within the latest fixed time based on the time-series data of the driver's head position / face direction and gaze direction. ..

In step S104, the input data organizing unit 50 combines the output of the physiological amount database 44 and the output of the visual behavior database 48 into one vector and outputs the output to the model learning unit 60.

In step S105, the driving behavior acquisition unit 52 acquires the driver's steering operation, accelerator operation, brake operation, vehicle travel locus, and peripheral vehicle positions acquired at the time of warning from the operation detector 18. The driving behavior database 54 calculates and outputs a plurality of indexes related to driving behavior based on the accumulated data.

In step S106, the lane determination unit 56 determines the traveling lane position of the own vehicle based on the position of the own vehicle when the warning is detected, and stores the determination result in the model switching information database 58.

Then, in step S108, it is determined whether or not the data obtained in the above steps S102 to S106 has been accumulated.

In the next step S112, the model learning unit 60 will use the combination of the output of the input data organizing unit 50 and the index value of the index output by the driving behavior database 54 for each of the plurality of indexes. It learns the driving behavior prediction model for the index and ends the model learning processing routine.

Next, the computer 30 repeatedly executes the driving suitability determination processing routine shown in FIG. 11 while the vehicle equipped with the driving suitability determination device 10 is running.

First, in step S120, the warning detection unit 12 determines whether or not a warning issued to the driver has been detected. If a warning is detected by the warning detection unit 12, the process proceeds to step S122.

In step S122, the physiological amount acquisition unit 72 acquires the physiological amount of the driver immediately before the warning is detected from the physiological amount sensor 14. The physiological amount database 74 outputs the average value of each physiological amount in the most recent fixed time.

In the next step S124, the visual behavior acquisition unit 76 acquires data representing the visual behavior, which is the movement of the driver's line of sight immediately before the warning is detected, from the line-of-sight measuring instrument 16. The visual behavior database 78 calculates and outputs a gaze time rate indicating the ratio of the time for gaze at each visual field area within the latest fixed time based on the time-series data of the driver's head position / face direction and gaze direction. ..

In step S126, the input data organizing unit 80 combines the output of the physiological amount database 74 and the output of the visual behavior database 78 into one vector and outputs the output to the appropriate state level calculation unit 86.

In step S128, the lane determination unit 82 determines the traveling lane position of the own vehicle based on the position of the own vehicle when the warning is detected, and stores the determination result in the model switching information database 84.

In step S130, the appropriate state level calculation unit 86 acquires each of the plurality of indexes related to the driver's driving behavior by using the driving behavior prediction model of the index corresponding to the driving lane position determined by the lane determination unit 82. The index value of the index is estimated from the data representing the visual behavior and the physiological amount.

In step S132, the appropriate state level calculation unit 86 calculates the appropriate state level of the driver for each of the plurality of indicators based on the index value of the index estimated by using the driving behavior prediction model of the index. ..

In step S134, as shown in FIG. 8, the final appropriate state level determination unit 88 selects the minimum value, the average value, the median value, the majority vote, or the mode value of the calculated appropriate state level values. , The final proper state level is determined, and the operation proper state determination processing routine is terminated.

The step S132 is realized by the processing routine shown in FIG. The processing routine is executed for each index. Here, the case where the appropriate state level is a discrete value will be described.

In FIG. 12, y is the output of the driving behavior prediction model, y ₁ , ..., Y _N-1 is the determination threshold value, and y _N-1 <y _N-2 << ... <y ₂ <y ₁ And.

The appropriate state level is calculated by comparing the output y of the driving behavior prediction model of the index with each of the determination thresholds y ₁ , ..., Y _N-1 . For example, if y is y ₁ or more, the appropriate state level is set to the lowest 1. If y is less than y _N-1 , the appropriate state level is set to the highest N.

<Example>
As an example of the model output, FIG. 13 shows the predicted value of the reaction time of the driver traveling in the intermediate lane, and FIGS. 14 and 15 show the line-of-sight / face orientation distribution and the gaze time rate at time A in FIG. 13, respectively. 16 and 17, respectively, show the line-of-sight / face orientation distribution and the gaze time rate at time B in FIG. At time A, attention is paid to the front, left and right mirrors, and rear (room mirror), and it can be seen that the driver has a firm grasp of the situation around the vehicle. On the other hand, at time B, the driver pays attention to the display screen of the instrument panel / HMI, and the grasp of the surrounding situation is neglected. Corresponding to such a driver's visual behavior, the reaction time predicted values in FIG. 13 have a minimum value and a maximum value, respectively, and it can be seen that an output reflecting the driver state is obtained.

As described above, according to the driving suitability determination device according to the present embodiment, for each of the plurality of indicators related to the driver's driving behavior, a model in which data and physiological quantities representing visual behavior are associated with the index. Estimate the index value of the index from the acquired data representing the visual behavior and the physiological quantity, and determine the driving appropriate state of the driver based on the estimated index value for each of the plurality of indexes. Therefore, the proper operation state of the driver can be accurately determined.

In addition, instead of determining the appropriate state from the driver's state based only on empirical knowledge, it is based on the data accumulated by associating each state of the driver with the specific and quantitative driving behavior at that time. Therefore, the past driving behavior data when the driver is in a similar state is modeled for each of a plurality of indexes, and the driver's driving appropriate state is determined by comprehensive evaluation of those models. In this way, the driver operation appropriate state can be determined more objectively and quantitatively than in the conventional case. In addition, it is possible to deal with drivers having various individual differences and daily changes in physical condition of individual drivers.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

For example, an appropriate physical quantity other than the above, such as the maximum braking force, may be added to the driving behavior index value calculated by the driver driving behavior database, and the corresponding driving behavior prediction model may also be added and used.

Further, when determining the final appropriate state level, an appropriate evaluation formula other than the above, such as a weighted average value of each appropriate state level, may be introduced and used.

Further, the case where one device includes a learning unit and an operation appropriate state determination unit has been described as an example, but the present invention is not limited to this, and the learning unit and the operation appropriate state determination unit are included in separate devices. It may be configured to be.

10 Operational suitability determination device 12 Warning detection unit 14 Physiological amount sensor 16 Line-of-sight measuring device 18 Operation detector 20 Position measuring device 30 Computer 32 Output unit 40 Learning unit 42, 72 Physiological amount acquisition unit 44, 74 Physiological amount database 46, 76 Visual behavior acquisition unit 48, 78 Visual behavior database 50, 80 Input data organization unit 52 Driving behavior acquisition unit 54 Driving behavior database 56, 82 Lane judgment unit 58, 84 Model switching information database 60 Model learning unit 62 Model storage unit 70 Driving appropriateness State determination unit 86 Appropriate state level calculation unit 88 Final appropriate state level determination unit

Claims (11)

A visual behavior acquisition unit that acquires data representing visual behavior, which is the movement of the vehicle driver's line of sight,
For each of the plurality of indicators related to the driver's driving behavior, the index value of the index is estimated from the acquired data representing the visual behavior using a model in which the data representing the visual behavior and the index are associated with each other. Driving behavior estimation department and
For each of the plurality of indicators, a driving appropriate state determination unit that determines the driving appropriate state of the driver based on the index value estimated by the driving behavior estimation unit, and a driving appropriate state determination unit.
Operational suitability determination device including. The operation appropriate state determination device according to claim 1, wherein the operation appropriate state determination unit further determines the final operation appropriate state based on the operation appropriate state determined for each of the plurality of indicators. The driving suitability determination device according to claim 1 or 2, wherein the plurality of indicators include a reaction time, a maximum steering speed, a maximum steering angle, or a vehicle locus deviation amount. Further including a physiological amount acquisition unit for acquiring the physiological amount of the driver,
The model is a model in which data representing the visual behavior, the index, and the physiological amount are associated with each of the plurality of indexes.
Claim 1 that the driving behavior estimation unit estimates the index value of the index from the acquired data representing the visual behavior and the acquired physiological amount for each of the plurality of indexes by using the model. The operating suitability determination device according to any one of claims 3. A learning data collection unit that collects a combination of an index value of each of the plurality of indexes acquired at the time of a warning and data representing the visual behavior acquired immediately before the warning.
A model learning unit that learns the model for each of the plurality of indicators based on the combination collected by the learning data collection unit.
The operation appropriate state determination device according to any one of claims 1 to 4, further comprising. Claims 1 to claim that the data representing the visual behavior is a gaze frequency distribution showing the gaze frequency of the driver's line of sight to each visual field region, or a series of gaze areas which are the visual field areas where the driver's line of sight is gaze. The operation appropriate state determination device according to any one of 5. Further including a traveling lane position determination unit for determining the traveling lane position of the vehicle.
The model associates the data representing the visual behavior with the index for each of the plurality of indexes for each of the traveling lane positions.
The driving behavior estimation unit uses the model corresponding to the driving lane position determined by the driving lane position determination unit for each of the plurality of indicators, and uses the acquired data representing the visual behavior to obtain the above data. The operation appropriate state determination device according to any one of claims 1 to 6, which estimates the index value of the index. The driving lane position is the leftmost lane, which is the leftmost lane of the plurality of lanes, the rightmost lane, which is the rightmost lane of the plurality of lanes, the leftmost lane of the plurality of lanes, and the rightmost lane. The driving suitability determination device according to claim 7, which is either an intermediate lane that is not a lane or a single lane that is one lane on each side. The operation appropriate state determination device according to any one of claims 1 to 8, wherein the visual behavior acquisition unit acquires data representing the visual behavior acquired immediately before the warning. 9. The warning includes an alarm issued for danger in manual driving of the vehicle, or a switching request issued when switching from automatic driving to manual driving of the vehicle becomes necessary. The described driving suitability determination device. Computer,
A visual behavior acquisition unit that acquires data representing visual behavior, which is the movement of the vehicle driver's line of sight.
For each of the plurality of indicators related to the driver's driving behavior, the index value of the index is estimated from the acquired data representing the visual behavior using a model in which the data representing the visual behavior and the index are associated with each other. Driving for each of the driving behavior estimation unit and each of the plurality of indicators to function as a driving appropriate state determination unit for determining the driving appropriate state of the driver based on the index value estimated by the driving behavior estimation unit. Proper condition judgment program.

Images