JP5065224B2 - Psychological state estimation device and psychological state estimation method - Google Patents

Description

  The present invention relates to a psychological state estimation device and a psychological state estimation method for estimating a psychological state of a vehicle driver such as an irritated state.

In order to improve safety during driving, various devices for estimating a driver's psychological state such as an irritated state have been proposed. As this estimation device, for example, Patent Document 1 discloses that an irritated state is estimated based on biological information such as a heart rate and skin potential of a vehicle driver and vehicle operation (driving behavior) by the driver. Yes.
Japanese Patent Application No. 2007-56235 JP 2006-34803 A JP 2003-61939 A

  Factors that change the driver's frustration include various factors such as traffic jams, avoidance of on-street parking, problems with voice recognition, and inappropriate route guidance for navigation. Among these factors, there are factors that show an influence of an irritated state in driving behavior but hardly appear in biological information, and factors that appear in biological information but hardly appear in driving behavior. Therefore, when the psychological state is always estimated using all information such as biological information and driving behavior, the estimation accuracy may decrease.

  Then, this invention makes it a subject to provide the psychological state estimation apparatus and the psychological state estimation method which estimate the psychological state of the driver | operator of a vehicle with high precision.

  A psychological state estimation device according to the present invention is a psychological state estimation device that estimates a psychological state of a driver of a vehicle, and detects biological information of the driver and driving behavior of the driver. Driving behavior detection means, vehicle exterior environment detection means for detecting an environment outside the vehicle that affects the psychological state of the driver, system malfunction detection means for detecting a malfunction of the system in the vehicle that affects the psychological state of the driver, When the outside environment detection means detects the outside environment affecting the driver's psychological state, the driver's psychology is given priority over the driving behavior detected by the driving action detection means over the biological information detected by the biological information detection means. When the system malfunction detection means detects an in-vehicle system malfunction that affects the driver's psychological state, the biometric information detection means detects the driving behavior detected by the driving action detection means. Characterized in that it comprises the the estimated means biometric information with priority for estimating the psychological state of the driver.

  In this psychological state estimation device, in order to acquire information that expresses the psychological state of the driver of the vehicle, the biological information of the driver is detected by the biological information detection means, and the driving behavior of the driver is detected by the driving behavior detection means. Is detected. In addition, the psychological state estimation device detects the environment outside the vehicle by the vehicle exterior environment detection means and acquires the system malfunction in the vehicle by the system malfunction detection means in order to acquire factors that affect the psychological state of the driver of the vehicle. To detect. The information on the change in the driver's psychological state may differ depending on the factor that affects the psychological state, and the estimation accuracy is improved by estimating the psychological state using information suitable for the factor. In particular, in the case of a factor outside the vehicle and a factor inside the vehicle, a change in psychological state is likely to appear in driving behavior with respect to a factor outside the vehicle, and a change in psychological state is likely to appear in biometric information with respect to a factor inside the vehicle. Therefore, the psychological state estimation device estimates the driver's psychological state with priority given to driving behavior over biometric information when the estimation means detects an environment outside the vehicle that affects the driver's psychological state. When an in-vehicle system malfunction affecting the driver's psychological state is detected, the driver's psychological state is estimated with priority on biometric information over driving behavior. Thus, in the psychological state estimation device, the driver's psychological state can be estimated with high accuracy by preferentially using information suitable for the psychological state estimation in accordance with factors that affect the driver's psychological state.

  The biometric information is various information that shows a person's psychological state and can be measured by a living body, and includes, for example, heart rate, skin potential, and blood pressure. The driving action is various actions performed by the driver, and includes, for example, vehicle operations (accelerator operation, brake operation, steering operation, etc.). Examples of the environment outside the vehicle that affect the psychological state of the driver include traffic jams, avoiding obstacles (pedestrians, parking on the street, etc.), merging, tilting and interrupting other vehicles, and red signal stop. Examples of system malfunctions in the vehicle that affect the psychological state of the driver include voice recognition mistakes, input recognition mistakes, and inappropriate system responses (such as navigation route guidance). Examples of the psychological state include an irritated state, an impatient state, and a panic state.

  In the psychological state estimation apparatus according to the present invention, the estimation means drives more than the weight for the biological information detected by the biological information detection means when detecting the environment outside the vehicle that affects the psychological state of the driver by the external environment detection means. The driver's psychological state is estimated by weighting the driving behavior detected by the behavior detecting means, and the driving behavior detecting means is detected when the system malfunction detecting means detects in-vehicle system malfunction affecting the driver's psychological condition. The weight of the biometric information detected by the biometric information detection unit may be made heavier than the weight of the driving behavior detected in step 1, and the driver's psychological state may be estimated.

  In the estimation unit of this psychological state estimation device, when an environment outside the vehicle that affects the driver's psychological state is detected, each information is used to estimate the driver's psychological state with priority on driving behavior over biological information. In the weighting for, the weight for driving behavior is made heavier than the weight for biological information. In addition, in the estimation means, when an in-vehicle system malfunction affecting the driver's psychological state is detected, in order to estimate the driver's psychological state with priority on biometric information over driving behavior, The weight for biometric information is set higher than the weight for driving behavior. As described above, in the psychological state estimation device, the degree of influence of each information can be easily reflected when the psychological state is estimated by weighting each information according to the factor.

  In the psychological state estimation device according to the present invention, the estimation unit does not use the driving behavior detected by the driving behavior detection unit when the system malfunction detection unit detects an in-vehicle system malfunction affecting the driver's psychological state. The configuration may be such that the driver's psychological state is estimated using only the biological information detected by the biological information detection means.

  In the case of a system malfunction in the vehicle, a change in the driver's psychological state hardly appears in the driving behavior (it may not appear at all), but appears significantly in the biological information. Therefore, the estimation means of the psychological state estimation device estimates the driver's psychological state using only biological information without using driving behavior when an in-vehicle system malfunction affecting the driver's psychological state is detected. .

  The psychological state estimation method according to the present invention is a psychological state estimation method for estimating the psychological state of a driver of a vehicle, and detects a biometric information detecting step for detecting the biometric information of the driver and the driving behavior of the driver. A driving behavior detection step, a vehicle environment detection step that detects an environment outside the vehicle that affects the psychological state of the driver, a system malfunction detection step that detects a malfunction of the system inside the vehicle that affects the psychological state of the driver, If the outside environment that affects the driver's psychological state is detected in the outside environment detection step, the driver's psychology is given priority over the driving behavior detected in the driving behavior detection step over the biological information detected in the biological information acquisition step. Estimate the state, and if a system malfunction that affects the driver's psychological state is detected in the system malfunction detection step, it is detected in the driving behavior detection step. Characterized in that than driving behavior and a estimation step of estimating the psychological state of the driver with priority the detected biometric information in the biometric information detection step.

  In the psychological state estimation method according to the present invention, when an external environment that affects the psychological state of the driver is detected in the external environment detection step in the estimation step, driving is performed more than the weight for the biological information detected in the biological information detection step. The driver's psychological state is estimated by weighting the driving behavior detected in the behavior detecting step, and the in-vehicle system malfunction affecting the driver's psychological state is detected in the system malfunction detecting step. The weight of the biometric information detected in the biometric information detection step may be made heavier than the weight of the driving behavior detected in step 1, and the driver's psychological state may be estimated.

  In the psychological state estimation method of the present invention, in the estimation step, when an in-vehicle system malfunction affecting the driver's psychological condition is detected in the system malfunction detection step, the driving behavior detected in the driving behavior detection step is not used. The configuration may be such that the driver's psychological state is estimated using only the biological information detected in the biological information detection step.

  Each psychological state estimation method has the same effect by the same operation as each psychological state estimation device described above.

  The present invention can estimate the driver's psychological state with high accuracy by preferentially using information suitable for the psychological state estimation in accordance with factors that affect the driver's psychological state.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a psychological state estimation device and a psychological state estimation method according to the present invention will be described with reference to the drawings.

  In the present embodiment, the present invention is applied to a frustration estimation device that is mounted on a vehicle and estimates a driver's frustration level. In the frustration estimation apparatus according to the present invention, the frustration level of the driver is estimated using a Bayesian network based on the biological information, behavior information, and driving behavior that indicate the frustration state of the driver. In the frustration estimation device according to the present invention, the estimated frustration level is provided to various driver support devices, alarm devices, frustration mitigation devices, and the like.

  The outline of the present embodiment will be described with reference to FIGS. FIG. 1 is a conceptual diagram of the present embodiment. FIG. 2 is a diagram showing the relationship between the driver's frustration factor, the driver's living body and driving behavior, where (a) is a system malfunction factor, and (b) is an external environment factor. It is.

  Factors that can increase the driver's frustration can be broadly divided into factors inside and outside the vehicle. A factor outside the vehicle is an environment outside the vehicle that frustrates the driver. For example, there are traffic jams, avoidance of parking on the street, pedestrians, interruptions and other vehicles by other vehicles, and stop in red light. The factors in the car are the malfunction of the system that frustrates the driver. For example, recognition errors in speech recognition, recognition errors in various inputs, inappropriate response results from the system (for example, inappropriate navigation routes Guidance).

  In addition, as information indicating the driver's frustrated state, there are biological information and behavior information that appear in the driver itself and driving behavior that the driver expresses as behavior. The biological information includes, for example, heart rate, skin potential, and blood pressure. The behavior information includes, for example, facial expressions, gestures, and speech sounds. The driving behavior includes, for example, vehicle operations (accelerator operation, brake operation, etc.).

  Conventionally, it has been considered that the estimation accuracy is higher when the estimation is performed using all the information indicating the frustrated state of the driver as described above. However, variations in estimation accuracy were observed due to frustrating factors. Therefore, the information used for estimation is changed among various types of information indicating the driver's frustration state for various factors of frustration, and the frustration level is estimated and the actual frustration level is obtained by sensory evaluation, and the estimation result and The actual frustration level was compared and analyzed. As a result of this analysis, it was found that even if the factors were the same, the estimation results varied depending on the information used for estimation, and there were information that was easily affected and information that was not easily affected by the factors.

  For example, when there is an avoidance of a road parked vehicle or a voice recognition error as a cause of irritation, estimation is performed using only biological information (including behavior information), and biological information (including behavior information) and driving behavior are also calculated. Estimation. In the case of avoiding parked vehicles on the road, the estimation accuracy was higher when driving behavior was also used. On the other hand, in the case of a speech recognition error, the estimation accuracy is higher when only the biological information is used without using the driving action. In addition to this, the following conclusions were obtained as a result of analysis using various patterns.

  As shown in FIG. 2A, when there is a system malfunction in the vehicle that affects the frustrated state, the influence of the frustrated state appears more greatly in the biometric information than the driving behavior, and the biometric information is prioritized for the estimation of the frustrated level. Thus, the estimation accuracy is improved by using it (in particular, it is better not to use driving behavior at all). On the other hand, as shown in FIG. 2B, when there is an environment outside the vehicle that affects the irritated state, the influence of the irritated state appears more greatly in the driving action than the biological information, and the driving action is prioritized for estimating the irritated level. As a result, the estimation accuracy is improved.

  Therefore, various types of information for determining an environmental factor outside the vehicle are sensed and various types of information for determining an in-vehicle system malfunction are sensed. And based on these various sensing information, the factor classification which affects an irritated state is determined. In order to select biological information, behavior information, and driving behavior used for estimating the frustration level according to the determined factor type, weighting is performed on the biological information, behavior information, and driving behavior. In addition, it senses biometric information, behavior information, and driving behavior representing the driver's frustrated state. Then, using the Bayesian network, the sensed biological information, behavior information, and driving behavior are used as feature quantities, and the weight of the biological information, behavior information, and driving behavior is added to the conditional probability to estimate the frustration level. When the weight value is given as 0 or 1, the frustration level is estimated using only the feature quantity having the weight of 1.

  The frustration level estimated in the present embodiment is a value from 0 to 2, may be an integer value (0, 1, 2), or may not be an integer value. The frustration level 0 is a state where there is no awareness of frustration and is a normal calm state. The frustration level 1 is a state in which the awareness of the frustration is small, and it is a state that makes it stuffy. The frustrating level 2 is a state in which the awareness of frustration is large, endures anger, and is a state of immediate touch. When the level is up to about this frustrating level 2, misjudgments and sudden operations are likely to occur during driving. Incidentally, the frustrating level 3 is a state in which anger is expelled. You may make it estimate to this irritation level 3.

  With reference to FIGS. 3-6, the irritation | stimulation estimation apparatus 1 is demonstrated. FIG. 3 is a configuration diagram of the frustration estimation apparatus according to the present embodiment. FIG. 4 is an example of reference data for determining the degree of weighting, and is configured only with a weight of 1 (denoted as ◯) and a weight of 0 (denoted as x). FIG. 5 is a diagram showing the structure of a Bayesian network for estimating the frustration level. FIG. 6 is an example of preprocessing for sensing information, where (a) is preprocessing for driving behavior, and (b) is preprocessing for biological information (skin potential).

  The frustration estimation device 1 estimates the frustration level based on biometric information, behavior information, driving behavior and an out-of-vehicle environment factor that affects the frustrated state, and an in-vehicle system malfunction factor. In particular, the frustration estimation device 1 determines a factor that is generated in order to improve estimation accuracy, and estimates the frustration level by weighting biological information, behavior information, and driving behavior according to the factor. Therefore, the frustration estimation device 1 includes a driving behavior detection means 2, behavior information detection means 3, biological information detection means 4, environmental information detection means 5, system information detection means 6, factor type determination means 7, and reference data storage device 8. The selection selection determining means 9 and the frustrating level determining means 10 are provided. In particular, the factor type determination unit 7, the reference data storage device 8, the selection selection determination unit 9, and the frustration level determination unit 10 are configured in an ECU [Electronic Control Unit] of the frustration estimation device 1.

  In the present embodiment, the driving behavior detection unit 2 corresponds to the driving behavior detection unit described in the claims, the biological information detection unit 4 corresponds to the biological information detection unit described in the claims, The environmental information detecting means 5 and the factor type determining means 7 correspond to the vehicle exterior environment detecting means described in the claims, and the system information detecting means 6 and the factor type determining means 7 are the system malfunction detecting means described in the claims. The reference data storage device 8, the sorting selection determination means 9, and the frustration level determination means 10 correspond to the estimation means described in the claims.

  The driving behavior detection means 2 is a sensing means for detecting each piece of information indicating the driving behavior (vehicle operation) of the driver. Examples of driving behavior include accelerator pedal operation and brake pedal operation. Examples of the sensing means include an accelerator pedal sensor that detects a depression force (or depression amount) of an accelerator pedal and a brake pedal sensor that detects a depression force (or depression amount) of a brake pedal.

  The behavior information detection means 3 is a sensing means for detecting each piece of information for obtaining the driver's behavior. Examples of the behavior information include facial expressions, gestures, and speech sounds. Examples of the sensing means include a camera that captures the driver's face and the like, and a microphone that collects the driver's voice.

  The biological information detection means 4 is a sensing means for detecting each information indicating the driver's biological body. Examples of biological information include skin potential (biological information that electrically indicates mental sweating), heart rate, and blood pressure. Examples of the sensing means include a skin potential sensor, a heart rate sensor (which may be an electrocardiogram sensor), and a blood pressure sensor.

  The environmental information detection means 5 is a sensing means for detecting information used for determining environmental factors outside the vehicle. As environmental factors outside the vehicle, there are, for example, traffic jams, obstacles (street parking, pedestrians, etc.) avoidance, and signal stop. Examples of the sensing means include a vehicle speed sensor, a steering angle sensor, a laser radar and camera for detecting an obstacle, a VICS device for receiving VICS [Vehicle Information and Communication System] information, and a GPS [Global Positioning System] information. There is a GPS device for acquisition.

  The system information detection means 6 is a sensing means for detecting information used for determining a system malfunction in the vehicle. As a system malfunction factor in the vehicle, there are, for example, a voice recognition error, an input recognition error, and an unsatisfactory system response result. The sensing means is not a sensor for actually sensing, but means for acquiring information from a system in the vehicle. Therefore, as sensing means, for example, means for acquiring recognition results and internal parameters of the speech recognition system, means for acquiring input results and internal parameters of each system having an input interface with the driver, and responding to the driver There are means for acquiring the response results and internal parameters of each system.

  The factor type determination unit 7 is a unit that determines a factor type that affects the driver's frustration based on various information detected by the environment information detection unit 5 and various information detected by the system information detection unit 6. . Specifically, the factor type determination unit 7 performs preprocessing necessary for determining the factor type for each piece of information detected by the environment information detection unit 5 and each piece of information detected by the system information detection unit 6. Do. However, for information that does not require preprocessing, the sensing information is used as it is.

  For example, in the case of information detected by a laser radar (laser transmission / reception information scanned in the left-right direction in front of the vehicle), the presence / absence of an obstacle is determined based on the laser transmission / reception information. In this case, the relative distance, relative speed, lateral position, etc. of the obstacle are calculated. In the case of information detected by the camera (a captured image in front of the vehicle), the presence / absence of an obstacle is determined from the captured image, and if there is an obstacle, the type, large, etc. of the obstacle is obtained.

  Then, the factor type determination means 7 extracts information necessary for the determination from each of the pre-processed information, and uses the information to determine the traffic jam, the pedestrian avoidance determination, the road parking avoidance determination, and the signal stop. Determination, voice recognition error determination, input recognition error determination, system response result dissatisfaction determination, and the like are performed to detect the factor E. Here, there are cases where only one factor E is detected, cases where a plurality of factors E are detected, and cases where none are detected. Below, an example of the determination method is shown.

  In the case of traffic jam determination, determination is made based on VICS information, deceleration of the vehicle speed over a predetermined period (for example, 20 km / h or less continues for 5 minutes or more), distance between the vehicle and the preceding vehicle determined from information by laser radar. In the case of pedestrian avoidance determination and road parking avoidance determination, when an obstacle is detected based on information from a laser radar, a captured image of a camera, or the like, the determination is made based on a change in the steering angle and deceleration of the vehicle speed. In the case of signal stop determination, when 0 of the vehicle speed is detected, it is determined whether or not the vehicle is near an intersection from GPS information (current position information) and map information.

  In the case of voice recognition error determination, when certain utterance information (for example, “Oh!”, “Chit!”, “So!”) Is detected, and the certainty (corresponding to the matching accuracy) at the time of voice recognition is low It is determined when the speech is detected but the utterance content is not recognized. In the case of an input recognition error determination, the determination is made when re-input is performed in the same input phase when a predetermined input is performed (redo). In the case of the system response result dissatisfaction determination, it is determined when there is a redo operation for requesting a reresponse immediately after the information is presented.

  The reference data storage device 8 is a storage device that stores reference data used by the sorting selection determining means 9. The reference data is data in which biometric information, behavior information, and weighting degree of driving behavior are set for each factor E, and is set in advance based on the analysis result of the experiment as described above.

  FIG. 4 shows an example of reference data. In this example, the factors E include traffic jam, road parking avoidance, pedestrian avoidance, voice recognition error, input recognition error, and system response dissatisfaction. As the feature amount, there are four feature amounts, that is, a heart rate average, a heart rate fluctuation, an average skin potential, and a total sum of changes in skin potential, and each feature amount quantified with respect to behavior and driving behavior.

  In the example of FIG. 4, “◯” used for estimation / “x” not used is set as the weighting degree. In the case of environmental factors such as traffic jams, on-street parking avoidance, and pedestrian avoidance, “◯” is set for all information, and estimation is performed using all information. In the case of system malfunction factors such as voice recognition mistakes, input recognition mistakes, and system response dissatisfaction, “×” is set for the driving action, “○” is set for each information and action of the living body, and the driving action is not used. Estimation is performed using only information and behavior information.

  FIG. 4 only shows an example of weighting, and various other settings can be applied to weighting. For example, in the setting of “O” to be used / “X” not to be used, in the case of an environmental factor outside the vehicle, “O” is set only for driving behavior, “X” is set for each biological information and behavior, The estimation may be performed only by driving behavior without using the behavior information.

  Further, although the weighting degree is set for each piece of information for only the living body, the weighting degree may be set for each piece of information for behavior and driving behavior, or only one weighting degree is set for the living body. May be.

  The sorting selection determination unit 9 determines the weighting degree for the biological information, behavior information, and driving behavior according to the factor E determined by the factor type determination unit 7. Specifically, the sorting selection determination means 9 refers to the reference data stored in the reference data storage device 8 using the factor E as a key. Then, the sorting selection determining unit 9 sets the weighting degree corresponding to the factor E to the biological information, the behavior information, and the driving behavior based on the reference data. However, if both the external environment factor and the system malfunction factor are detected as the factor E, or if the factor E is not detected, the weighting degree is not set, or the biological information, behavior information, and driving behavior are Set the weighting level to be used equally.

As a method of reflecting the weighting degree in the conditional probability, there is a method of emphasizing the conditional probability in proportion to the weight α according to the equation (1) in addition to the method of selectively using the feature amount. In Equation (1), P _i represents the conditional probability before weighting, and P _i ′ represents the conditional probability after weighting.

  The frustration level determination means 10 uses the Bayesian network to detect the biological information detected by the biological information detection means 4 that is the characteristic amount, the behavior information detected by the behavior information detection means 3, and the driving behavior detected by the driving behavior detection means 2. On the other hand, the frustration level is estimated by taking into account the weights set by the sorting selection determination means 9.

  Specifically, the frustration level determination means 10 determines the network structure of the Bayesian network for estimating the frustration level. FIG. 5 shows the structure of the Bayesian network B. The feature quantity input to each of the nodes N1 to N6 of the network B includes a behavior, a heartbeat average of a living body, a heartbeat fluctuation, a skin potential average, a sum of skin potential changes, and driving behavior. The conditional probabilities between the nodes L1 to L6 of the network B reflect the weighting degree set by the sorting selection means 9 with respect to the basic value obtained from the learning data. In the network B shown in FIG. 5, rectangular nodes are discrete nodes, and circular nodes are continuous nodes. Note that a plurality of pieces of information may be used as feature amounts for behavior and driving behavior.

  When the learning data is collected, the frustration level determination means 10 determines a conditional probability (basic value) between the nodes of the network based on the learning data. The learning data is data collected while the vehicle is running. The actual irritability level obtained by sensory evaluation and the sensed feature values (the body heart rate average, heart rate fluctuation, skin potential average, and sum of skin potential changes). , Behavior, driving behavior). As for the learning data, data on many travel times and data collected at various factors E are desirable, and a conditional probability that improves the estimation accuracy can be obtained. About this conditional probability (basic value), only an initial value may be set, or it may be updated every time learning data is collected after the initial value is set.

  In the frustrating level determination means 10, when the weighting degree is set by the sorting selection determination means 9, the weighting degree is reflected in the conditional probability between the nodes. However, when the weighting degree is not set by the sorting selection determining means 9, the weighted degree is not reflected and the conditional probability (basic value) is used as it is. Specifically, when “O” used for estimation / “X” not used is set as the weighting degree as shown in FIG. 4, when “O” used for estimation is set, it corresponds. The conditional probability (basic value) between nodes is used as it is, and when “x” which is not used is set, the corresponding nodes are disconnected and the feature amount (node) is not used. By reflecting the weighting degree in this way, it is possible to obtain a network structure that uses only the nodes for which “◯” is set (that is, “×” is set from the network structure determined first). Nodes can be excluded.)

  In the frustrating level determination means 10, living body sensing information detected by the biological information detection means 4, which is a feature quantity, behavior sensing information detected by the behavior information detection means 3, and driving behavior sensing information detected by the driving action detection means 2. The necessary preprocessing is performed for each.

  For example, as shown in FIG. 6A, in the case of sensing information (accelerator pedal force, brake pedal force) of driving behavior, each sensing value is down-sampled to 10 Hz, bias is removed, median filter is applied, and then cepstrum and cepstrum Pre-processing for determining the amount of change of each is performed. As shown in FIG. 6 (b), in the case of living body sensing information (skin potential), the sum of the average value and the absolute value of the change amount is obtained after downsampling the sensing value to 10 Hz and performing Savitzky-Golay FIR Smoothing. Pre-processing to find In addition, in the case of behavior sensing information (captured images around the driver's face), a face area and a face part area (eyes, mouth, eyebrows, etc.) are extracted from the captured image, and the extracted area image and a normal image are extracted. Or the preprocessing which detects the change of the facial expression is compared with the image at the time of an irritated state.

  Then, the frustrating level determination means 10 inputs each feature amount subjected to the preprocessing to each node of the network, and calculates the frustrating level.

  FIGS. 7 and 8 show the estimation result of the frustration level by the Bayesian network and the actual frustration level when the driving action is used and not used in addition to the biological information and the behavior information. The frustration level is an integer value of 0-2. A solid line RL is an actual frustration level by sensory evaluation or the like, and a broken line EL is an estimated frustration level.

  The example shown in FIG. 7 is a case where an off-road environmental factor for avoiding on-street parking has occurred as a factor, (a) is an estimation result using driving behavior, and (b) is an estimation result using no driving behavior. . The error between the actual frustration level RL1 shown in FIG. 7A and the estimated frustration level EL1a (corresponding to the case where estimation is performed using the reference data shown in FIG. 4 in the frustration estimation apparatus 1) is shown in FIG. It is smaller than the error between the actual frustration level RL1 shown and the estimated frustration level EL1b. Therefore, when an environmental factor outside the vehicle occurs, the estimation accuracy is higher when the estimation is performed using the driving behavior.

  The example shown in FIG. 8 is a case where a system malfunction factor due to a voice recognition error occurs as a factor, (a) is an estimation result using driving behavior, and (b) is an estimation result not using driving behavior. The error between the actual frustration level RL2 shown in FIG. 8B and the estimated frustration level EL2b (corresponding to the case where estimation is performed using the reference data shown in FIG. 4 in the frustration estimation apparatus 1) is shown in FIG. It is smaller than the error between the actual frustration level RL2 shown and the estimated frustration level EL2a. Therefore, when a system malfunction factor occurs, the estimation accuracy is higher when the estimation is performed without using the driving action.

  With reference to FIGS. 3-6, the operation | movement in the irritation | stimulation estimation apparatus 1 is demonstrated. In particular, the processing of the factor type determination means 7 will be described along the flowchart of FIG. 9, the processing of the sorting selection determination means 9 will be described along the flowchart of FIG. 10, and the processing of the frustration level determination means 10 will be described. 11 will be described along the flowchart. FIG. 9 is a flowchart showing the flow of processing in each detection means and factor type determination means in FIG. FIG. 10 is a flowchart showing the flow of processing in the sorting selection determining means of FIG. FIG. 11 is a flowchart showing the flow of processing in the frustration level determination means of FIG.

  The driving behavior detection means 2 senses each information indicating the driving behavior. The behavior information detection means 3 senses each piece of information for obtaining behavior information. The living body information detecting means 4 senses each information indicating the driver's living body.

  In order to determine the cause of frustration, the environmental information detection means 5 senses each information for determining an environmental factor outside the vehicle, and the system information detection means 6 senses each information for determining a system malfunction factor ( S10a-S10n).

  The factor type determination unit 7 performs preprocessing on each sensed sensing information (S11a to S11n). When all the preprocessing is completed, the factor type determination unit 7 uses the sensing information that has been subjected to the preprocessing to determine whether or not the factor has occurred for each factor (for example, traffic jam determination, pedestrian Avoidance determination, on-street parking avoidance determination, signal stop determination, voice recognition error determination, input recognition error determination, system response result dissatisfaction determination) are performed (S12a to S12m). When all the determinations are completed, the factor type determination unit 7 detects the currently occurring factor E based on the determination results of all the determinations (S13).

  The sorting selection determining means 9 refers to the reference data in the reference data storage device 8 using the detected factor E as a key (S20). Then, the sorting selection determining unit 9 acquires a weighting degree for each piece of information (living body, behavior, driving behavior) for estimating the lie-rai level according to the factor E (S21).

  The frustration level determination means 10 determines the structure of the Bayesian network for determining the frustration level (S30).

  The frustrating level determination means 10 determines whether there is learning data for determining the conditional probability (S31). If it is determined in S31 that the learning data has not yet been collected sufficiently, the frustration level determination means 10 returns to the determination in S31 and waits until learning data is collected. When it is determined that there is sufficient learning data in S31, the frustration level determination means 10 determines conditional probabilities (basic values) between the nodes of the Bayesian network using the learning data (S32).

  The frustrating level determination means 10 determines whether or not the weighting degree has already been acquired by the sorting selection determination means 9 (S33). When it is determined in S33 that the weighting degree has not been acquired, the frustrating level determination means 10 returns to the determination in S33 and waits until the weighting degree is acquired by the selection selection determination means 9.

  If it is determined in S33 that the weighting degree has been acquired, the frustrating level determination means 10 reflects the weighting degree in the conditional probability between the nodes, and uses only the nodes for which “O” is set as the weighting degree. A network structure is set (S34).

  The frustrating level determination means 10 performs necessary preprocessing for the biological information detected by the biological information detection means 4, the behavior information detected by the behavior information detection means 3, and the driving behavior detected by the driving behavior detection means 2. Then, the frustration level determination means 10 inputs biometric information, behavior information, and driving behavior that have been preprocessed to each node of the Bayesian network, and calculates the frustration level (S35).

  Then, the frustration estimation device 1 outputs the estimated frustration level to various driver support devices, alarm devices, frustration mitigation devices, and the like.

  According to the frustration estimation device 1, the frustration level of the driver is preferentially used by using information suitable for estimation in accordance with factors (external vehicle environmental factors and in-vehicle system malfunction factors) that affect the frustration state of the driver. Can be estimated with high accuracy.

  The frustration estimation apparatus 1 can easily reflect the degree of influence of each information on the frustrated state on the Bayesian network by weighting each information according to the factor.

  As mentioned above, although embodiment which concerns on this invention was described, this invention is implemented in various forms, without being limited to the said embodiment.

  For example, in the present embodiment, the present invention is applied to the frustration estimation device that estimates the driver's frustration level, but the present invention is also applicable to a device that estimates other states such as impatience, boredom, awakening, and panic.

  In the present embodiment, the frustration level is estimated using behavior information, but the behavior information may not be used, or the frustration level may be estimated using other information.

  Moreover, although an example of the determination method of the factor type is shown in the present embodiment, the factor type may be determined by another method.

  Further, in the present embodiment, which information is given priority among a plurality of pieces of information used for estimation of the frustration level by the weighting degree may be represented by a method other than weighting.

  In this embodiment, the frustration level is estimated using a Bayesian network, but other methods such as threshold processing, evaluation function, decision tree, neural network, HMM (Hidden Markov Model), support vector machine, etc., are used. It may be used to estimate the frustration level. For example, when the threshold process is used, the weighting degree may be used for setting the threshold value, and when the evaluation function is used, the weighting degree may be used for setting the coefficient of the evaluation function.

  When other estimation methods are used as described above, a numerical weighting degree as shown in FIG. 12 can be used. In the example of FIG. 12, a numerical value (so that the sum of the numerical values of each information becomes 1) is set as the weighting degree. In the case of environmental factors such as traffic jams, road parking avoidance, and pedestrian avoidance, each biological information and behavior is set to “0.1”, and driving behavior is set to “0.5”. The estimation is performed. In the case of system malfunction factors such as voice recognition error, input recognition error, and system response dissatisfaction, “0.18” is set for each information and behavior of the living body, “0.1” is set for the driving action, and biological information Estimation is performed with emphasis on behavior information. In the case of a system malfunction factor in the numerical setting, “0” is set for the driving action, “0.2” is set for each information and behavior of the living body, and the estimation can be performed only by the living body information and the behavior information. It may be performed. Further, in the case of an environmental factor outside the vehicle, the weight for driving behavior may be set larger than the total value of the weight of each piece of information on the living body and the weight of behavior, and estimation may be performed with more emphasis on driving behavior. Moreover, although the same numerical value is set to the weight of each information of a biological body and the weight of behavior, a different numerical value may be set.

  For example, using the weighting degree (αh1 (average heart rate), αh2 (heart rate fluctuation), αs1 (average skin potential), αs2 (total skin potential change), αb (behavior), αd (driving behavior)) by this numerical value, It can be applied to threshold processing to estimate the frustration level. Specifically, the frustration levels (Ih1 (heart rate average), Ih2 (heart rate fluctuation), Is1 (skin potential average)), Is2 (skin potential change sum), Ib (behavior), Id ( Ask for driving behavior)) respectively. Then, the frustration level I (= Ih1 × αh1 + Ih2 × αh2 + Is1 × αs1 + Is2 × αs2 + Ib × αb + Id × αd) is calculated from the frustration level and the weighting degree for each sensing information.

It is a conceptual diagram of this Embodiment. It is a figure which shows the relationship between a driver | operator's irritation | stimulation factor, a driver | operator's biological body, and driving behavior, (a) is a case where there is a system malfunction factor, and (b) is a case where there exists an external environment factor. It is a block diagram of the frustration estimation apparatus which concerns on this Embodiment. It is an example of the reference data for determining a weighting degree. It is a figure which shows the structure of the Bayesian network for estimating an irritated level. It is an example of the pre-processing with respect to sensing information, (a) is a pre-processing of driving action, (b) is a pre-processing of biometric information (skin potential). It is an example of an estimation result of an irritated level when there is an environmental factor outside the vehicle, (a) is an estimation result when driving behavior is used, and (b) is an estimation result when driving behavior is not used. It is an example of an estimation result of an irritated level when there is a system malfunction factor, (a) is an estimation result when driving behavior is used, and (b) is an estimation result when driving behavior is not used. It is a flowchart which shows the flow of a process in each detection means and factor classification determination means of FIG. It is a flowchart which shows the flow of a process in the selection selection determination means of FIG. It is a flowchart which shows the flow of a process in the frustration level determination means of FIG. It is another example of the reference data for determining a weighting degree.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Frustration estimation apparatus, 2 ... Driving action detection means, 3 ... Behavior information detection means, 4 ... Biological information detection means, 5 ... Environmental information detection means, 6 ... System information detection means, 7 ... Factor classification determination means, 8 ... Reference data storage device, 9 ... sorting selection determining means, 10 ... frustrating level determining means

Claims (6)

A psychological state estimation device for estimating a psychological state of a driver of a vehicle,
Biological information detection means for detecting the biological information of the driver;
Driving behavior detecting means for detecting the driving behavior of the driver;
Vehicle exterior environment detection means for detecting the vehicle exterior environment affecting the driver's psychological state;
System malfunction detection means for detecting malfunction of the system in the vehicle affecting the psychological state of the driver;
Driving with priority given to the driving behavior detected by the driving behavior detection means over the biological information detected by the biological information detection means when the external environment detection means detects an environment outside the vehicle that affects the psychological state of the driver. When the system malfunction detection unit detects an in-vehicle system malfunction that affects the driver's psychological state, the biometric information detection unit performs the driving behavior detected by the driving behavior detection unit. A psychological state estimation device comprising: estimation means for presuming a driver's psychological state with priority on the detected biological information.   The estimation means detects the driving behavior detection means rather than the weight for the biological information detected by the biological information detection means when the outside environment detection means detects the external environment affecting the driver's psychological state. The driver's psychological state is estimated by weighting the driving behavior, and when the system malfunction detecting unit detects an in-vehicle system malfunction that affects the driver's psychological state, the driving detected by the driving behavior detecting unit The psychological state estimation apparatus according to claim 1, wherein the psychological state of the driver is estimated by weighting the biological information detected by the biological information detection unit more than the weight for the behavior.   When the system malfunction detection means detects an in-vehicle system malfunction affecting the driver's psychological state, the estimation means does not use the driving behavior detected by the driving behavior detection means, but the biological information detection means. The psychological state estimation device according to claim 1 or 2, wherein the psychological state of the driver is estimated using only the detected biological information. A psychological state estimation method for estimating a psychological state of a driver of a vehicle,
A biological information detection step for detecting the biological information of the driver;
A driving behavior detection step for detecting the driving behavior of the driver;
An external environment detection step for detecting an external environment that affects the psychological state of the driver;
A system malfunction detection step for detecting a malfunction of the system in the vehicle that affects the psychological state of the driver;
Driving is performed with priority given to the driving behavior detected in the driving behavior detection step over the biological information detected in the biological information acquisition step when the external environment affecting the driver's psychological state is detected in the driving environment detection step. When the in-vehicle system malfunction affecting the driver's psychological state is detected in the system malfunction detection step, the biological information detection step is more effective than the driving behavior detected in the driving action detection step. A psychological state estimation method comprising: an estimation step of estimating the psychological state of the driver with priority on the detected biological information.   In the estimation step, when an environment outside the vehicle that affects the psychological state of the driver is detected in the vehicle environment detection step, the weight is detected in the driving action detection step rather than the weight for the biological information detected in the biological information detection step. When the weight of driving behavior is increased to estimate the driver's psychological state, and in-vehicle system malfunction affecting the driver's psychological state is detected in the system malfunction detection step, the driving detected in the driving behavior detection step The psychological state estimation method according to claim 4, wherein the psychological state of the driver is estimated by weighting the biological information detected in the biological information detecting step more heavily than the weight for the behavior.   In the estimation step, when an in-vehicle system malfunction affecting the driver's psychological state is detected in the system malfunction detection step, the driving information detected in the driving action detection step is not used, and the biological information detection step is performed. The psychological state estimation method according to claim 4 or 5, wherein the psychological state of the driver is estimated using only the detected biological information.

Images