JP2019016213A - On-vehicle device, processing device and program - Google Patents

Abstract

To make it possible to accurately determine a state such as a drowsiness level of a driver based on various traveling data that can be acquired from an on-vehicle device without photographing the driver.SOLUTION: An on-vehicle device 10 detects a plurality of types of operation information D 1 to D 9, and combines a plurality of types of statistical values Dx of each of them to use as a feature amount. An evaluation value calculation unit 17 of the on-vehicle device 10 calculates an evaluation value Dy such as a drowsiness level or the like by substituting a combination of optimized feature amounts into a multiple regression equation learned and determined in advance. A learning processing apparatus executes a learning process for optimizing an evaluation operation of the on-vehicle device 10. This learning processing apparatus 50 sets experimental data of a simulated driving and its correct value as a target of learning processing, extracts each feature amount, and executes multiple regression analysis. A principal component analysis is applied to feature quantities and extract only principal components. A feature quantity with a small correlation coefficient is deleted to reduce a number of calculation items.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an in-vehicle device and a processing device, and more particularly to a technique for detecting a state of a driver who drives a vehicle.

  In various vehicles, in order to maintain a safe driving state, it is essential that the driver is in a normal driving state. For example, there is a possibility that a driver who is in a sleep-deficient state is driving a nap when driving a vehicle, and there is a high risk of causing a traffic accident. It is also possible that the driver suddenly loses consciousness due to the onset of some illness and falls into an impossible driving situation. Moreover, even if the driver itself does not have a physical problem, for example, when the side-by-side driving is continuously performed, there is a high risk of causing a traffic accident.

  Therefore, the snooze driving prevention device disclosed in Patent Document 1 is determined to fall asleep when it detects a deceleration exceeding a predetermined speed within a predetermined time from a predetermined monitoring speed based on the detected vehicle speed. A determination unit and a warning unit that issues a warning to the driver when a nap is determined are provided.

  In addition, the vehicle driving support device disclosed in Patent Document 2 dynamically changes the data range of the travel log used for calculating the driver (driver) characteristics according to the characteristics of the accumulated travel log. The technique for calculating the driver characteristics with higher accuracy is shown.

  Moreover, the state determination apparatus shown by patent document 3 has shown the technique for improving the determination precision at the time of determining the state of the driver | operator of a moving body. Specifically, as a result of repeatedly deriving a parameter representing the behavior of the vehicle according to the detection result of the sensor group and collating the parameter with the determination model, the parameter is included in one of the threshold regions of the determination model. For example, it is determined that the driver is driving casually.

  In addition, as a general technique for detecting a human health state, a wearable device may be used. Such a wearable device is worn on a human body like a wristwatch, for example, and measures biological information such as body temperature, heartbeat, blood pressure, and the like. It is possible to determine drowsiness using such biological information. In addition, an attempt has been made to monitor the state of the driver from the contents of the face image by photographing the face and the like of the driver driving the vehicle using an in-vehicle camera.

JP 2004-34938 A JP2015-143936A Japanese Patent Laid-Open No. 2006-2192

  However, when the driver's biometric information is used to estimate the driver's state, the driver must drive the vehicle with the wearable device always worn, which places a heavy burden on the driver. It is difficult to use continuously. In addition, when taking a picture of the face of a driver who is driving a vehicle using an in-vehicle camera, the driver will be aware that the driver is constantly monitored, especially driving a business vehicle. In the case of a crew member, the mental burden during driving is large. In other words, it is necessary to avoid such a situation because the crew members will always be monitored by others in the company while driving.

  On the other hand, when a state such as a snooze driving is detected based on the speed of the vehicle as in Patent Document 1, no special burden is imposed on the driver. However, since the detection target is limited only to a specific state such as snoozing driving and there is a possibility of erroneous detection, it cannot be said that the reliability is sufficiently high.

  The present invention has been made in view of the above situation, and its purpose is based on various operation data that can be acquired from the in-vehicle device, such as the state of the driver, for example, the state of driving while out, the shaking. It is to provide an in-vehicle device, a processing device, and a program that can be used to accurately determine the state of driving while driving.

In order to achieve the above-described object, the vehicle-mounted device according to the present invention is characterized by the following (1) to (5).
(1) An input unit for inputting operation information of L (L is an integer of 2 or more) types of vehicles having different types of information;
An output unit for notifying a driver's state according to an evaluation value having a statistical value acquired from each of the L types of operation information as a feature value;
An in-vehicle device comprising:
(2) As the statistical value, there are M (M is an integer of 2 or more) types of statistical values acquired from each of the operation information,
The evaluation value is a combination ( _{N CP} ) when a predetermined number P is extracted from the M types of statistical values (total N = L × M statistical values) acquired for each of the L types of operation information. In part or all of the above, the characteristic amount includes a value calculated from the statistical value used for each combination.
The in-vehicle device according to (1) above, wherein
(3) The in-vehicle device according to (2), wherein the feature amount is a value calculated from a product of the statistical values used for each combination.
(4) In addition to the N statistical values, the statistical value further includes an evaluation value at least in the past.
The evaluation value is a value calculated from the statistical value used for each combination for each combination ( _{N + 1} C _P ) when a predetermined number P is extracted from the ( _{N + 1} ) statistical values. As a feature quantity,
The in-vehicle device according to (3) above, wherein
(5) The evaluation value is a numerical value obtained by substituting a plurality of the feature quantities into a multiple regression equation.
The in-vehicle device according to any one of (1) to (4) above,

According to the in-vehicle device having the configuration of (1) above, evaluation values calculated based on statistical values of each of a plurality of types of operation information are used. It is possible to distinguish between subtle differences in the driver's condition. Further, it is not necessary to photograph the driver's face and the driver does not need to wear the wearable device, so the burden on the driver is small.
According to the in-vehicle device having the configuration (2), the evaluation value calculated based on the feature amount calculated from each of at least a part of the combination of the plurality of types of operation information and the plurality of types of statistical values is used. Therefore, various types of feature amounts can be evaluated, and subtle differences in the driver's state can be identified.
According to the in-vehicle device having the configuration (3), since the feature amount is calculated by the product of the statistical values, various types of feature amounts can be evaluated.
According to the in-vehicle device having the configuration (4), it is possible to feed back an evaluation value at least at a point in the past to the latest calculated value. Therefore, even if it takes time until the correct feature value is obtained after the detection of the feature value of the driving state, it is possible to evaluate the correct feature value based on the past evaluation values fed back. Become.
According to the in-vehicle device having the configuration (5), one objective variable representing the driver's state can be calculated as the evaluation value using each of the plurality of feature quantities as an explanatory variable based on a multiple regression equation.

In order to achieve the above-described object, the processing apparatus according to the present invention is characterized by the following (6) and (7).
(6) an input unit for inputting various information;
A data processing unit that performs data processing based on information input from the input unit;
With
The input unit is input with operation information of L (L is an integer of 2 or more) types of vehicles having different types of information, and a driver's state evaluated when the operation information is acquired,
The data processing unit
Part or all of combinations ( _{N CP} ) when a predetermined number P is extracted from M types of statistical values (total N = L × M statistical values) acquired for each of the L types of operation information. In the above, a value calculated from the statistical value used for each combination is calculated as a feature amount,
Further, a processing apparatus is characterized in that a multiple regression analysis is performed using the driver's state as an objective variable, and a coefficient for multiplying the feature quantity is calculated for each feature quantity.
(7) The data processing unit further performs a principal component analysis on the feature quantity, and sets a feature quantity having a relatively high correlation with the objective variable as a target of the multiple regression analysis.
The processing apparatus according to (6) above, wherein

According to the processing device having the above configuration (6), learning is performed so as to coincide with the state of the driver, which is an objective variable, and a coefficient for each necessary feature amount is calculated. By giving the appropriate coefficient calculated here to the in-vehicle device described in (1) above, the in-vehicle device can appropriately evaluate the state of the driver.
According to the processing apparatus having the configuration of (7), the number of feature quantities necessary for the in-vehicle device described in (1) to correctly evaluate the driver's state can be greatly reduced, so that the calculation load is reduced. Even if it is an in-vehicle device equipped with a computer having a small processing capacity, evaluation results can be obtained in a short time.

In order to achieve the above-described object, a program according to the present invention is characterized by the following (8).
(8) A program for causing a computer to realize the functions of the input unit and the data processing unit according to (6) or (7).

  According to the program configured as described in (8) above, learning is performed so as to match the driver's state, which is an objective variable, and a coefficient for each necessary feature amount is calculated. By giving the appropriate coefficient calculated here to the in-vehicle device described in (1) above, the in-vehicle device can appropriately evaluate the state of the driver.

  The in-vehicle device, the processing device, and the program of the present invention can be used for accurately determining the state of the driver based on various operation data that can be acquired from the in-vehicle device. The processing apparatus and program of the present invention have a function of calculating a coefficient for each feature amount used for calculation by the in-vehicle device by learning. The vehicle-mounted device of the present invention can improve the accuracy of evaluating the driver's condition by employing each coefficient calculated by the processing device of the present invention.

  The present invention has been briefly described above. Further, the details of the present invention will be further clarified by reading through a mode for carrying out the invention described below (hereinafter referred to as “embodiment”) with reference to the accompanying drawings. .

FIG. 1 is a block diagram illustrating a configuration example of an in-vehicle device according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration example of the learning processing system according to the embodiment of the present invention. FIG. 3 is a front view showing an example of an image taken by the in-vehicle camera. FIG. 4 is a side view showing the positional relationship between the driving simulator and the accompanying device. FIG. 5 is a flowchart illustrating a specific example of the learning process in the learning processing apparatus. FIG. 6 is a time chart showing specific examples of the correct value regarding the time transition of the sleepiness level, the estimated value of the multiple regression equation, and a value obtained by quantizing the estimated value of the multiple regression equation. FIG. 7 is a flowchart illustrating a modification of the learning process in the learning processing apparatus.

  Specific embodiments relating to the vehicle-mounted device and the processing apparatus of the present invention will be described below with reference to the drawings.

<Description of In-vehicle Device 10 of Embodiment>
<Overview>
A configuration example of the in-vehicle device 10 in the embodiment of the present invention is shown in FIG. The in-vehicle device 10 shown in FIG. 1 is used in various vehicles such as business vehicles such as taxis, trucks, and buses, and is used for automatically detecting the driver's state. Here, as a representative state of the detection target, a driver's dozing state is assumed. However, this in-vehicle device 10 can be used for detecting various states as well as being asleep.

  The in-vehicle device 10 shown in FIG. 1 is a drive recorder, and a function for detecting a driver's dozing operation and notifying a danger is added. However, the in-vehicle device of the present invention is not limited to a drive recorder, and the functions of the present invention can be installed in various types of in-vehicle devices.

<Description of configuration>
A vehicle speed pulse signal SG1 output from a vehicle speed sensor (not shown) mounted on the host vehicle is applied to the input of the in-vehicle device 10 shown in FIG. The vehicle speed pulse signal SG1 includes, for example, a pulse that is generated every time the output shaft of the vehicle transmission rotates by a predetermined amount. Therefore, the traveling speed (km / h) and moving distance of the host vehicle can be calculated based on the vehicle speed pulse signal SG1.

  The in-vehicle camera 30 is connected to the input of the in-vehicle device 10 shown in FIG. The in-vehicle camera 30 is fixed, for example, in the vicinity of a window in front of the host vehicle, and is directed in a direction in which a situation such as a road surface or a landscape in front of the host vehicle can be photographed. A plurality of in-vehicle cameras 30 can be used as necessary. A video signal SG <b> 4 representing the content captured by the in-vehicle camera 30 is input to the in-vehicle device 10.

  In an actual vehicle, it is possible to input various signals other than the vehicle speed pulse signal SG1 and the video signal SG4 from the outside to the vehicle-mounted device 10, but only for the purpose of detecting the driver's sleep, There is no need to input other signals.

  As shown in FIG. 1, an in-vehicle device 10 includes a vehicle speed detection function 11, an acceleration sensor 12, an image recognition engine 13, a drive recorder function 14, an operation data detection function 15, a statistical value calculation unit 16, and an evaluation value calculation. A unit 17, a feedback processing unit 18, an evaluation value determination unit 19, an audio output unit 20, and a wide area wireless communication adapter 21 are provided. An antenna 22 is connected to the wide area wireless communication adapter 21.

  The entity of the main components inside the in-vehicle device 10 shown in FIG. 1 is composed of software composed of hardware of an electric circuit mainly composed of a microcomputer and programs and data executed by the microcomputer. Become. Further, if necessary, an in-vehicle device 10 may include a processor dedicated to image processing, a processor for numerical calculation, and the like.

  The vehicle speed detection function 11 constantly calculates the latest traveling speed (km / h) of the host vehicle based on the vehicle speed pulse signal SG1 input from the outside. The calculated traveling speed signal SG5 is output from the vehicle speed detection function 11 and input to the operation data detection function 15.

  The acceleration sensor 12 is fixed to the vehicle-mounted device 10 and has a function of detecting the magnitude of acceleration in the traveling direction (front-rear direction) and the magnitude of acceleration in the lateral direction (left-right direction) of the host vehicle. . A signal SG2 indicating the magnitude of the acceleration in the traveling direction and a signal SG3 indicating the magnitude of the lateral acceleration are each output from the acceleration sensor 12 and applied to the operation data detection function 15.

  The image recognition engine 13 performs recognition processing in real time on the image of each frame of the input video signal SG4 and detects various feature amounts. Specifically, it is possible to automatically recognize the position of each white line (lane boundary separation position) marked on the road surface in the image, the position of the host vehicle, and the like.

  The drive recorder function 14 realizes the same function as a general drive recorder. That is, information on the input video signal SG4, information on the running state of the host vehicle, such as travel speed and acceleration, is acquired and recorded. In order to record a large amount of information, the drive recorder function 14 is provided with a storage device such as a memory card constituted by a nonvolatile memory, for example.

  The operation data detection function 15 includes nine types of operation information D1 to D9 required for detecting the driver's condition, a traveling speed signal SG5, a traveling direction acceleration signal SG2, and a lateral direction acceleration signal. Detection is based on SG3 and signal SG6 of the image recognition result. In addition, it is also possible to increase / decrease the number of types of operation information D1-D9 as needed. The operation information D1 to D9 will be described later in detail.

  The statistical value calculation unit 16 performs statistical processing on each of the nine types of operation information D1 to D9 output from the operation data detection function 15, and calculates each statistical value Dx. The types of statistical values Dx calculated by the statistical value calculator 16 are standard deviation (std), maximum value (max), minimum value (min), average value (mean), median value (median), mode value ( There are seven types: mode) and range. Note that the number of types of the statistical value Dx may be increased or decreased as necessary.

  The evaluation value calculation unit 17 calculates the seven types of statistical values Dx of the nine types of operation information D1 to D9 and the feedback signal Df applied from the feedback processing unit 18 by substituting them into a predetermined calculation formula. Thus, the evaluation value Dy is obtained. In this example, the evaluation value Dy represents the degree of the driver's drowsiness in a numerical value. A specific example of the calculation formula of the evaluation value calculation unit 17 will be described later.

  The feedback processing unit 18 performs processing for feeding back old information to the current information and reflecting it in the evaluation of the current information. Specifically, the feedback processing unit 18 has a function of inputting the evaluation value Dy output from the evaluation value calculation unit 17 and temporarily holding it, and outputting it as a feedback signal Df after a predetermined time (for example, 120 seconds) has elapsed. doing.

  The evaluation value determination unit 19 determines whether or not to output an alarm or the like by comparing the evaluation value Dy output by the evaluation value calculation unit 17 with a predetermined threshold value. For example, when the evaluation value determination unit 19 determines that the evaluation value Dy is large and the driver is very sleepy, the notification signal SG8 necessary for avoiding the occurrence of the sleep driving is output.

  The audio output unit 20 executes audio output useful for avoiding the occurrence of a drowsy driving according to the notification signal SG8. For example, a sound such as “Dangerous, please stop and take a break” is output to the driver.

  The wide area wireless communication adapter 21 provides a wireless communication function for data communication with a remote computer. For example, the operation information recorded by the drive recorder function 14 and the notification signal SG8 can be transmitted to a remote server, an office PC of a company managing the vehicle, or the like as necessary.

<Description of operation>
<Operation of the operation data detection function 15>
The operation data detection function 15 shown in FIG. 1 detects the following nine types of operation information D1 to D9.

D1: The traveling speed of the host vehicle.
D2: Advancing direction (front-rear direction: x-axis) acceleration of the host vehicle.
D3: A lateral direction (left-right direction: y-axis) acceleration of the host vehicle.
D4: FOE (Focus Of Expansion) fluctuation. More specifically, (moving average value for 10 seconds−current value) in the x coordinate is used.
D5: FOE difference R. Specifically, an absolute value in the x coordinate (10-second moving average value−FOE x coordinate value after calibration) is used.
D6: The vehicle position. Specifically, (absolute value (left white line intersection coordinates + right white line intersection coordinates) / 2-vehicle center projection position (coordinates) at a 6m point when the vehicle is traveling straight) is converted into a distance of 6m points. To do.
D7: Distance from the left white line. More specifically, the lateral distance between a point (own vehicle position) 6 m ahead from the current position of the host vehicle and the left white line is used.
D8: Distance from the right white line. More specifically, a lateral distance between a point (own vehicle position) 6 m ahead from the current position of the host vehicle and the right white line is used.
D9: Horizontal movement speed.

  The traveling speed D1 of the host vehicle corresponds to the traveling speed signal SG5. The vehicle traveling direction (front-rear direction: x-axis) acceleration D2 and the lateral direction (left-right direction: y-axis) acceleration D3 correspond to acceleration signals SG2 and SG3 output from the acceleration sensor 12. Each of the other operation information D4 to D9 can be calculated based on the recognition result (signal SG6) of the image recognition engine 13.

  The vanishing point (FOE) of motion will be described. For example, in an image of a camera taken in front of a running vehicle, the pixel moves at a location with a high luminance such as a white line on the road surface. In addition, there is no movement at the intersection point where the vectors representing such movement are extended linearly. This intersection is the motion vanishing point (FOE). For example, in a vehicle traveling straight on a straight road, the point at infinity ahead of the traveling direction of the traveling road surface is FOE. For example, when the host vehicle runs meandering or the traveling direction changes, the FOE also changes. Therefore, the change in FOE can be used to detect the possibility of drowsy driving.

  An example of an image taken by the in-vehicle camera 30 is shown in FIG. In the image shown in FIG. 3, positions of a vanishing point (FOE) of motion, a left white line 6 m ahead, and a right white line 6 m ahead are shown.

<Description of Formula Performed by Evaluation Value Calculation Unit 17>
A specific example of a calculation formula used by the evaluation value calculation unit 17 shown in FIG. 1 to calculate the evaluation value Dy is shown below.

y = k01 × D1max
+ K02 × D6max
+ K03 x D6min
+ K04 × D9max
+ K05 × Df
+ K06 × D1max × D6min
+ K07 × D1max × D7min
+ K08 × D1max × D7mean
+ K09 x D1max x Df
+ K10 × D1min × D8median
+ K11 × D4std × D5median
+ K12 × D4min × D5range
+ K13 × D5max × D7median
+ K14 × D5mode × D6min
+ K15 x D6max x D9max
+ K16 × D6max × Df
+ K17 × D6min × D7min
+ K18 × D7std × D7mean
+ K19 × D9max × Df
+ Kc0 (1)
However,
y: evaluation value (corresponding to Dy)
k01 to k19: Coefficients assigned to the corresponding statistical values of the operation information (D1 to D9) of each item kc0: Constant term D1max: Maximum value of traveling speed D1 of own vehicle D6max: Maximum value of own vehicle position D6 D6min: Own Minimum value of vehicle position D6 D9max: Maximum value of lateral movement speed D9 Df: Feedback value (calculated value of y before 120 seconds)
D7min: Minimum value of the distance D7 from the left white line D7mean: Average value of the distance D7 from the left white line D8median: Median of the distance D8 from the right white line D4std: Standard deviation of the FOE fluctuation D4 D5median: Median of the FOE difference R D4min : Minimum value of FOE fluctuation D4 D5range: Range of FOE difference R D5max: Maximum value of FOE difference R D7median: Median value of distance D7 from left white line D5mode: Mode of FOE difference R D7std: Distance D7 from left white line Standard deviation of

  The content of the expression (1) is merely an example, and in practice, various calculation expressions can be adopted to improve the calculation accuracy by performing a learning process described later.

In this embodiment, there are a maximum of 2080 types of combinations in each item of the calculation formula used to calculate the evaluation value y. That is, there are nine types of operation information (D1 to D9), the number of statistical types Dx is 7, and one type of feedback signal Df is used, so there are 64 types of calculation items.
9 × 7 + 1 = 64 (2)

The number of types of combinations for calculating the product of two items for each of these 64 items is 2080 as shown in the following equation.
64 × (64−1) / 2 + 64 = 2080 (3)

  However, a large calculation load is required to calculate all 2080 items. Also, not all 2080 types of items have a great causal relationship with the driver's doze. Therefore, only the items having high utility value are selected and adopted by the learning process from the 2080 types of combination items. A typical example of the result is the content of the above formula (1). Actually, it is assumed that a calculation formula having about 230 calculation items is employed.

  However, in any case, for example, each item of the expression (1) includes statistical values of operation information (D1 to D9) correlated with the driver's doze, so the driver's doze Can be used to calculate the evaluation value Dy. The values of the coefficients k01 to k19 included in this calculation formula are real numbers within the range of “−1 to +1”, and for example, numerical values having an accuracy of about four digits after the decimal point are adopted. These contents are optimized by the learning process.

<Description of Learning Processing System 40 of Embodiment>
<Overview>
A configuration example of the learning processing system 40 in the embodiment of the present invention is shown in FIG. The learning processing device 50 included in the learning processing system 40 shown in FIG. 2 corresponds to the processing device of the present invention. The learning processing system 40 implements learning processing necessary for the evaluation value calculation unit 17 of the in-vehicle device 10 illustrated in FIG. 1 to calculate the evaluation value Dy using an appropriate calculation formula. Depending on the result of this learning process, the content of the above expression (1) adopted by the evaluation value calculation unit 17 changes.

<Description of preparation for learning process>
FIG. 4 shows the positional relationship between the driving simulator 41 and the devices accompanying it. In the present embodiment, the driving simulator 41 shown in FIG. 4 is used to acquire experimental data that can be used in the learning process.

  The driving simulator 41 has a driver seat on which a driver can be seated in the same manner as an actual driver seat of a vehicle, and further includes a steering wheel, an accelerator pedal, a brake pedal, and a shift operation lever that can be operated by the driver. Equipment. In addition, a six-sided display 41a is arranged so as to surround the driver with the driver's front as the main body. On the screen of the display 41a, the contents simulating the scene of the front view that the driver can visually recognize in the actual driving state are displayed according to the driving operation with respect to the driving simulator 41.

  When the simulation driving experiment is performed, the subject 42 sits in the driving seat of the driving simulator 41 as shown in FIG. Then, as a driver of the driving simulator 41, the subject 42 performs simulated driving by operating devices such as a steering wheel, an accelerator pedal, a brake pedal, and a shift operation lever while visually confirming the contents of the display 41a. The contents of the screen of the display 41a change sequentially according to the simulated driving operation of the subject 42.

  In order to acquire experimental data necessary for the learning process, as shown in FIG. 4, a front video recording camera 43 arranged in front of the driver's seat captures a front video on the screen of the display 41a. A driver's face video recording camera (stereo camera) 44 arranged in front of the driver's seat photographs the face of the subject 42 sitting in the driver's seat and the vicinity thereof. In addition, an electrocardiograph 45 is attached to the subject 42 in order to acquire an electrocardiogram that is the biological information of the subject 42.

  The experimental data necessary for the learning process is recorded by the information recording device 46 shown in FIG. That is, the front image obtained by photographing with the front video recording camera 43, the face image of the subject 42 obtained by photographing with the driver face video recording camera 44, and the electrocardiogram information output from the electrocardiograph 45 Are recorded in the information recording device 46 as a set of time-series data. Information such as the traveling speed of the host vehicle, the longitudinal acceleration, and the lateral acceleration is input from the driving simulator 41 and recorded by the information recording device 46. The information recording device 46 is equipped with a storage device such as a hard disk in order to record a large amount of information.

  In order to perform the learning process by applying the multiple regression analysis, in addition to the simulated driving experiment data recorded by the information recording device 46, a correct answer value representing the driver's state corresponding to the objective variable of the multiple regression analysis is required. become. Therefore, as shown in FIG. 2, the correct value at each time point is specified by the correct value specifying process 47 based on the information recorded by the information recording device 46.

  As for the correct value specifying process 47, here, as a human task performed by a skilled person, a reproduction image of a face (facial expression, head movement, etc.) taken by the driver's face video recording camera 44, and an electrocardiograph 45 It is assumed that the result of sensory evaluation based on the output electrocardiogram is output as a correct value at each time point.

  Of course, it is also possible to automate the correct value specifying process 47 and eliminate the need for human work. For example, by automatically recognizing the facial expression or head movement of the subject 42 in the reproduced video by using image recognition technology, or by combining the recognized state and the electrocardiogram state, the correct answer value evaluation result is automatically output. Can do.

When the correct value is evaluated in the correct value specifying process 47, for example, it is assumed that an evaluation index shown in Table 1 is used and an evaluation according to this is performed. The evaluation indices in Table 1 are proposed by the New Energy and Industrial Technology Development Organization (NEDO).

  That is, when the evaluation index shown in Table 1 is used, the facial expression of the subject 42, the movement of the head, and the like can be associated with the five-level rating values. Therefore, while reproducing the information recorded by the information recording device 46, the correct value when the sleepiness level is evaluated in five levels as the state of the subject 42 at each time point is determined by the person or machine based on the contents of Table 1. It can be specified by the value specifying process 47.

<Description of learning process>
The learning processing device 50 shown in FIG. 2 includes, as input data 48 used in the learning processing, the simulated driving record information recorded by the information recording device 46 and the correct value at each time point obtained by the correct value specifying processing 47. The learning process is implemented by combining

  The entity of the learning processing device 50 is a general computer such as a personal computer, and is loaded with application software necessary for executing learning processing including special calculation processing such as multiple regression analysis.

  The learning processing device 50 basically optimizes the number of items, the type of each item, and the coefficient of each item in the multiple regression equation with the driver state (in this example, sleepiness level) as the objective variable y. To learn how to make it happen. The explanatory variables for which the multiple regression equation can be used in this case correspond to the statistical values Dx in the operation information D1-D9 input to the evaluation value calculation unit 17 in the in-vehicle device 10. Therefore, also in the learning processing apparatus 50, the number of explanatory variables in the multiple regression equation is 2080, which is the number of types of combinations shown in the above equation (3) in the initial state.

  The learning processing device 50 performs the learning process to reduce the number of items of the explanatory variable of the multiple regression equation, optimize the type of the explanatory variable, and calculate each appropriate objective variable y. A coefficient is calculated for each item. Various coefficients 52 after learning processing calculated by the learning processing device 50 as a result of the learning processing, the number of explanatory variables of the multiple regression equation, the type of each item, and the like are registered in the evaluation value calculation unit 17 of the in-vehicle device 10. .

<Specific procedure of learning process>
A specific example of the procedure of the learning process in the learning processing apparatus 50 is shown in FIG. The learning process shown in FIG. 5 will be described below.

  In step S <b> 11, the learning processing device 50 sequentially extracts usable data from the input data 48. That is, data corresponding to each of the nine types of operation information D1-D9 detected by the operation data detection function 15 of the in-vehicle device 10 is extracted here.

  In step S12, the learning processing device 50 calculates each feature amount based on each data of the operation information D1-D9 extracted in S11. That is, similar to the operation of the statistical value calculation unit 16 of the in-vehicle device 10, seven types of statistical values Dx are calculated for each of the nine types of operation information D1-D9, and these are used as feature amounts. Actually, the same number of feature quantities as the number of combination types 2080 shown in the above equation (3) can be obtained.

  In step S13, the learning processing device 50 performs principal component analysis (PCA) for the purpose of improving the followability of the estimated value to the correct value. In this example, the cumulative contribution rate is set as a constant, and the principal component analysis is performed. The principal component analysis will be described below.

  For example, when a certain vehicle is viewed from directly above, the vertical width and the horizontal width of the vehicle are known, but the height is not known. On the other hand, when the vehicle is viewed from the front, the width and height of the vehicle are known, but the vertical width is unknown. Further, when the vehicle is viewed from the side, the vertical width and height of the vehicle are known, but the horizontal width is unknown. Therefore, when this vehicle is viewed from a 45 ° oblique direction, the detection accuracy is lowered, but all of the vehicle's vertical width, horizontal width, and height can be comprehensively detected.

  The principle of principal component analysis is to grasp all feature quantities from one viewpoint, that is, the principal components, instead of grasping the feature quantities from the viewpoints directly above, in front, and directly beside. .

  On the other hand, the 2080 types of feature values input to the evaluation value calculation unit 17 and the learning processing device 50 of the in-vehicle device 10 include 7 types of statistical values Dx and 9 types of operation information D1-D9. All the combinations of the two item products of the 64 feature amounts constituted by the feedback signal Df are included. That is, all the main components are already included in the 2080 types of feature amounts, as in the case of viewing the vehicle from an oblique direction. Therefore, extracting only the principal components having high importance from the 2080 types of feature amounts means that the principal component analysis is performed.

  For example, by calculating a determinant represented by the following equation (4), q types (p >> q) of main components f1 to fq are obtained based on p types of feature amounts C1 to Cp and various weights W. It can be taken out. That is, the number of dimensions of the explanatory variable can be compressed from p to q.

  In step S14, the learning processing device 50 acquires a matrix of correlation coefficients between the objective variable y and each feature quantity in the multiple regression equation (R = corrcoef). That is, in step S13, a matrix representing each correlation coefficient between q feature quantities reduced from 2080 to q (for example, 230) and the objective variable y is acquired.

  Then, the learning processing device 50 extracts only the correlation coefficient set (matrix) for each feature quantity having a correlation coefficient equal to or greater than a predetermined threshold (0.1 in this example) in step S15. Feature values less than are deleted in step S16. That is, feature quantities having a correlation coefficient less than the threshold value are excluded from items to be calculated by the multiple regression equation.

  In step S17, the learning processing device 50 selects an explanatory variable for the feature quantity having a correlation coefficient equal to or greater than a predetermined threshold in the set (matrix) of correlation coefficients for each feature quantity extracted in step S15. A general stepwise method is applied as a method to select a feature. That is, which combination is best applied while looking at the R-square value (decision coefficient) adjusted for the degree of freedom of the result of the multiple regression analysis, the F value and the p value of the coefficient (significance probability of the coefficient of each explanatory variable) Search through repeated processing.

  In step S18, the learning processing device 50 performs a multiple regression analysis using the feature selected in step S17. That is, the combination of each feature quantity and the coefficient of each feature quantity are specified as parameters so that the objective variable y of the multiple regression equation can be sufficiently explained by the combination of each feature quantity that is an explanatory variable. At this time, since the correct value is also input as the input data 48 used in the learning process, the combinations of the types of the feature amounts and the respective values are optimized so that the relationship between the correct value of the objective variable y and each feature amount is optimized. Combination coefficients can be determined.

  The parameters obtained in step S18 are the various coefficients 52 after the learning process shown in FIG. 2 and are registered in the evaluation value calculation unit 17 of the in-vehicle device 10. As a result, the evaluation value calculation unit 17 executes the calculation of a multiple regression equation such as the above equation (1). Therefore, for example, when a parameter obtained as a result of learning by the learning processing system 40 of FIG. 2 is registered in the in-vehicle device 10 for the sleepiness level of the driver corresponding to the contents of Table 1, the evaluation value calculation unit of the in-vehicle device 10 17 and the evaluation value determination unit 19 perform a determination operation useful for preventing a drowsy driving.

<Specific examples of sleepiness levels>
Correct value 61 regarding the time transition of the drowsiness level (in the figure, the correct value 61 is clearly shown, a gradation is given to a region between the correct value 61 and the horizontal axis), an estimated value 62 of the multiple regression equation (Described in a line graph in the figure), and a specific example of a value 63 (denoted by a “+” shape plot in the figure) obtained by quantizing the estimated value of the multiple regression equation. As shown in FIG. In FIG. 6, the horizontal axis represents time, and the vertical axis represents sleepiness level. That is, when the learning processing device 50 executes the learning process, as shown in FIG. 6, the difference between the multiple regression equation estimated value (y) 62 and the correct answer value 61 becomes smaller as the learning process progresses. It can be seen that the parameters of the multiple regression equation are optimized.

<Modification of learning process procedure>
A modified example of the procedure of the learning process in the learning processing system 40 is shown in FIG. The learning process in FIG. 7 will be described below.

  In step S <b> 21, the learning processing device 50 sequentially extracts usable data from the input data 48. That is, data corresponding to each of the nine types of operation information D1-D9 detected by the operation data detection function 15 of the in-vehicle device 10 is extracted here.

  In step S22, the learning processing device 50 calculates each feature amount based on each data of the operation information D1-D9 extracted in S21. That is, similar to the operation of the statistical value calculation unit 16 of the in-vehicle device 10, seven types of statistical values Dx are calculated for each of the nine types of operation information D1-D9, and these are used as feature amounts. Actually, the same number of feature quantities as the number of combination types 2080 shown in the above equation (3) can be obtained.

  In step S23, the learning processing device 50 acquires a matrix of correlation coefficients between the objective variable y and each feature quantity in the multiple regression equation (R = corrcoef). That is, a matrix representing each correlation coefficient between 2080 feature quantities and the objective variable y is acquired.

  Then, the learning processing device 50 extracts only the correlation coefficient set (matrix) for each feature quantity having a correlation coefficient equal to or greater than a predetermined threshold (0.3 in this example) in step S24. Feature values less than are deleted in step S25. That is, feature quantities having a correlation coefficient less than the threshold value are excluded from items to be calculated by the multiple regression equation.

  In step S <b> 26, the learning processing device 50 performs principal component analysis (PCA) for the purpose of improving the followability of the estimated value to the correct answer value. In this example, the principal component analysis is performed with the cumulative contribution rate set as a constant. That is, the number of feature quantities extracted in step S24 is further reduced to extract only the main components having high utility value.

  In step S27, the learning processing apparatus 50 selects a feature by applying a general stepwise method as an explanatory variable selection method. That is, which combination is best applied while looking at the R-square value (decision coefficient) adjusted for the degree of freedom of the result of the multiple regression analysis, the F value and the p value of the coefficient (significance probability of the coefficient of each explanatory variable) Search through repeated processing.

  In step S28, the learning processing device 50 performs a multiple regression analysis using the feature selected in step S27. That is, the combination of each feature quantity and the coefficient of each feature quantity are specified as parameters so that the objective variable y of the multiple regression equation can be sufficiently explained by the combination of each feature quantity that is an explanatory variable. At this time, since the correct value is also input as the input data 48 used in the learning process, the combinations of the types of the feature amounts and the respective values are optimized so that the relationship between the correct value of the objective variable y and each feature amount is optimized. Combination coefficients can be determined.

  The parameters obtained in step S28 are the various coefficients 52 after the learning process shown in FIG. 2 and are registered in the evaluation value calculation unit 17 of the in-vehicle device 10. As a result, the evaluation value calculation unit 17 executes the calculation of a multiple regression equation such as the equation (1). Therefore, for example, when a parameter obtained as a result of learning by the learning processing system 40 of FIG. 2 is registered in the in-vehicle device 10 for the sleepiness level of the driver corresponding to the contents of Table 1, the evaluation value calculation unit of the in-vehicle device 10 17 and the evaluation value determination unit 19 perform a determination operation useful for preventing a drowsy driving.

<Advantages>
The vehicle-mounted device 10 shown in FIG. 1 uses a plurality of types of operation information D1 to D9, and further uses a combination of each of these types of statistics Dx, so that the driver's state such as sleepiness level can be determined. It can be detected with high accuracy. In addition, by adopting a product of a plurality of items as the feature quantity used for the calculation, it is possible to extract a principal component having a high utility value and improve the accuracy of the evaluation value. Moreover, the accuracy of the evaluation value can be improved by feeding back the evaluation value calculated in the past to the input of the multiple regression equation as the feedback signal Df. Further, the accuracy of the evaluation value can be increased by determining the content of the multiple regression equation based on the learned parameters.

  In the learning processing system 40 shown in FIG. 2, the learning processing device 50 performs the learning processing of the procedure shown in FIG. 5 or 7 based on the input data 48 used for the learning processing, so that the in-vehicle device 10 The required high-precision parameters can be calculated automatically. Moreover, since the learning processing apparatus 50 can reduce each feature-value computed by a multiple regression equation, ie, the number of items of an explanatory variable, it can reduce the calculation load in the vehicle equipment 10.

<Possibility of deformation>
In the above-described embodiment, the sleepiness level of the driver is assumed as the objective variable (y) of the multiple regression equation in the in-vehicle device 10 and the learning processing device 50. However, the objective variable (y) is changed appropriately. By performing learning, it can be applied to various other purposes. For example, there is a possibility that it can be used to detect a state in which the driver's attention is reduced, a state in which the driver's athletic ability is reduced, or a state in which driving with high risk is performed.

Here, the features of the above-described embodiments of the in-vehicle device, the processing device, and the program according to the present invention are briefly summarized and listed in the following [1] to [8], respectively.
[1] An input unit (evaluation value calculation unit 17) to which operation information (D1 to D9) of L types (L is an integer of 2 or more) of different types of information is input;
According to an evaluation value (Dy) having a statistical value (Dx) acquired from each of the L types of operation information as a feature value, an output unit (speech output unit 20) for notifying a driver's state;
A vehicle-mounted device (10) comprising:
[2] As the statistical value, there are M (M is an integer of 2 or more) types of statistical values (Dx) acquired from each of the operation information,
The evaluation value (Dy) is a combination when a predetermined number P is extracted from the M types of statistical values (total N = L × M statistical values) acquired for each of the L types of operation information ( _N In part or all of C _P ), a value calculated from the statistical value used for each combination is included as the feature amount (see Equation (1)).
The in-vehicle device according to [1] above, wherein
[3] The feature amount is a value calculated from a product of the statistical values used for each combination (see Formula (1)).
The in-vehicle device according to [2] above, wherein
[4] In addition to the N statistical values, the statistical value further includes an evaluation value (feedback signal Df) at least in the past.
The evaluation value is a value calculated from the statistical value used for each combination for each combination ( _{N + 1} C _P ) when a predetermined number P is extracted from the ( _{N + 1} ) statistical values. As a feature quantity (see equation (1)),
The in-vehicle device according to [3] above, wherein
[5] The evaluation value (Dy) is a numerical value obtained by substituting a plurality of the feature quantities into a multiple regression equation.
The in-vehicle device according to any one of the above [1] to [4].
[6] An input unit (input data 48 used for learning processing) to which various information is input;
A data processing unit (learning processing device 50) that performs data processing based on information input from the input unit;
With
The input unit is input with operation information of L (L is an integer of 2 or more) types of vehicles having different types of information, and a driver's state evaluated when the operation information is acquired,
The data processing unit
L type of the operation information has been M types of statistics acquired for each (a total of N = L × M pieces of statistics: Dx) combinations when taking out a predetermined number P out of the _{(N C P)} of the part or In each case, a value calculated from the statistical value used for each combination is calculated as a feature amount (S12, S22),
Further, a multiple regression analysis is performed using the driver's state as an objective variable, and a coefficient for multiplying the feature quantity is calculated for each feature quantity (S18, S28).
The processing apparatus characterized by the above-mentioned.
[7] The data processing unit further performs principal component analysis on the feature amount (S13, S26), and sets a feature amount having a relatively high correlation with the objective variable as a target of the multiple regression analysis.
The processing apparatus according to [6] above, wherein
[8] A program for causing a computer to realize the functions of the input unit and the data processing unit according to [6] or [7].

DESCRIPTION OF SYMBOLS 10 Onboard equipment 11 Vehicle speed detection function 12 Acceleration sensor 13 Image recognition engine 14 Drive recorder function 15 Operation data detection function 16 Statistical value calculation part 17 Evaluation value calculation part 18 Feedback processing part 19 Evaluation value determination part 20 Voice output part 21 Wide area wireless communication Adapter 22 Antenna 30 Vehicle-mounted camera 40 Learning processing system 41 Driving simulator 42 Subject 43 Front video recording camera 44 Driver face video recording camera 45 Electrocardiograph 46 Information recording device 47 Correct value identification processing 48 Input data used for learning processing 50 learning processing device 51 various coefficients before learning processing 52 various coefficients after learning processing 61 correct value regarding time transition of sleepiness level,
62 Estimated value of multiple regression equation 63 Quantized estimated value of multiple regression equation D1, D2, D3, D4, D5, D6, D7, D8, D9 Operation information Df Feedback signal Dx Statistical value Dy Evaluation value

Claims (8)

An input unit for inputting operation information of L types of vehicles having different types of information (L is an integer of 2 or more);
An output unit for notifying a driver's state according to an evaluation value having a statistical value acquired from each of the L types of operation information as a feature value;
An in-vehicle device comprising: As the statistical value, M (M is an integer of 2 or more) types of statistical values acquired from each of the operation information,
The evaluation value is a combination ( _{N CP} ) when a predetermined number P is extracted from the M types of statistical values (total N = L × M statistical values) acquired for each of the L types of operation information. In part or all of the above, the characteristic amount includes a value calculated from the statistical value used for each combination.
The in-vehicle device according to claim 1. The in-vehicle device according to claim 2, wherein the feature amount is a value calculated from a product of the statistical values used for each combination. The statistical value has an evaluation value at least at a point in the past in addition to the N statistical values,
The evaluation value is a value calculated from the statistical value used for each combination for each combination ( _{N + 1} C _P ) when a predetermined number P is extracted from the ( _{N + 1} ) statistical values. As a feature quantity,
The in-vehicle device according to claim 3. The evaluation value is a numerical value obtained by substituting a plurality of feature quantities into a multiple regression equation.
The in-vehicle device according to any one of claims 1 to 4, wherein the in-vehicle device is characterized. An input unit for inputting various information;
A data processing unit that performs data processing based on information input from the input unit;
With
The input unit is input with operation information of L (L is an integer of 2 or more) types of vehicles having different types of information, and a driver's state evaluated when the operation information is acquired,
The data processing unit
Part or all of combinations ( _{N CP} ) when a predetermined number P is extracted from M types of statistical values (total N = L × M statistical values) acquired for each of the L types of operation information. In the above, a value calculated from the statistical value used for each combination is calculated as a feature amount,
Further, a processing apparatus is characterized in that a multiple regression analysis is performed using the driver's state as an objective variable, and a coefficient for multiplying the feature quantity is calculated for each feature quantity. The data processing unit further performs principal component analysis on the feature quantity, and sets a feature quantity having a relatively high correlation with the objective variable as a target of the multiple regression analysis.
The processing apparatus according to claim 6.   A program for causing a computer to realize the functions of the input unit and the data processing unit according to claim 6 or 7.