JP2009098970A - Safe driving support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To support safe driving by learning relation between driving operation of a driver and an outside environment at any time, and recognizing deviation from an ordinary internal state of the driver. <P>SOLUTION: A correspondence between risk in a driving environment, recognized by a vehicle outside environment recognition part 2, and a feature amount of driving operation data, quantized by an operation feature amount quantization part 3, are learned by a model study part 4, and the model parameter is acquired. A state prediction part 5 predicts the internal state of the driver from the relevance of the operation feature amount and the driving environmental risk level using the learned model parameter. The prediction result of this internal state is evaluated by a reliability evaluation part 6 and outputted to an alarm/support part 7. The alarm/support part 7 compares the internal state of the driver with the driving environmental risk level, evaluates whether the vehicle is essentially in the safety or the dangerous state and executes the alarm and the operation support, and the like. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a safe driving support system that estimates the state of a driver who drives a moving body and provides support for safe driving.

  In recent years, with the goal of enabling safe movement of moving objects such as automobiles, techniques have been developed to ensure the safety by monitoring the state of the driver while driving to detect a decrease in wakefulness and falling asleep. . The estimation of the driver state is mainly performed by measuring the living body state and that estimated from the driving operation data, and various proposals have been made regarding these techniques.

  As a technique related to estimation of a driver state by measuring a biological state, there are techniques disclosed in Patent Documents 1 to 5. The technology of Patent Document 1 estimates the driver's state by measuring eye movement using an electrode attached to the driver's face, a camera that captures the eyeball, and the like. The degree of arousal is estimated based on the opening of the heel.

  Moreover, the technique of patent document 3 and patent document 4 estimates a driver's state by measuring a driver's heartbeat signal, and the technique of patent document 5 detects a brain current signal. .

On the other hand, there are technologies disclosed in Patent Documents 6 and 7 as technologies related to estimation of the driver state based on driving operation data. The technique of Patent Document 6 discriminates driving operation data of a driver under a predetermined condition and reflects it in a change in vehicle control characteristics. The technique of Patent Document 7 is based on fuzzy inference from vehicle state data. Is intended to estimate the intention and psychological state.
JP 2003-230552 A JP 2004-89272 A Japanese Patent No. 3596198 Japanese Patent Laid-Open No. 7-10024 JP-T-2006-524157 Japanese Patent No. 3058966 Japanese Patent No. 3036183

  As mentioned above, in order to realize safe driving of automobiles, it is effective to perform warnings, change control characteristics, assist operations, etc. using technology that estimates the state of the driver. Has been made.

  However, in the technology for performing living body measurement for estimating the driver state as disclosed in References 1 to 5, generally, a measurement amount having a large noise or individual difference must be dealt with. There is a problem that it cannot cope with the environment and the diversity of people who use it.

  That is, Patent Document 1 gives an example in which electrodes are arranged on the face for measurement of eye movement, but it is troublesome that a driver must be equipped with a measurement device that involves physical restraint during actual driving. Not only lacks reality.

  In Patent Document 2, data is measured based on information from a camera that captures the eyes of the driver. However, from the cost of adding additional devices to the vehicle, changes in the driver's attitude, and the external environment such as West Japan There is a possibility that photographing cannot be performed normally due to the influence of light.

  Patent Document 3 describes only electrodes and other devices installed on the seat and the steering, and Patent Document 4 merely describes psychological state detection means, but both provide examples of measuring heartbeats. However, these technologies also measure the electrocardiogram, which is relatively easily disturbed by noise, require additional equipment on the vehicle, and determine the characteristics of the heartbeat that vary from individual to individual. It must be said that there is a problem in view of the fact that it is used in the actual environment of an automobile, such as the point that must be performed.

  Patent Document 5 describes non-invasive measurement by mounting a device such as MRI on the vehicle, but the device described in the document directly or indirectly restrains the driver's posture and other conditions. Similarly, it must be said that the industrial application at the present time is not realistic, including its cost.

  In addition, as a matter common to Patent Documents 1 to 5, we are trying to measure the degree of arousal and fatigue with signs that are considered common to general humans even though there is advance calibration. There are some that do not actively deal with environmental relevance, but rather focus on smoothing out personal differences through preprocessing. Although such a method can be expected to some extent, there is no mechanism to actively handle individual differences and the external environment when estimating the driver state with higher accuracy. There is a certain limit to its performance.

  On the other hand, in Patent Documents 6 and 7, which use operation or vehicle data for estimating the driver state, when the driver's operation satisfies a predetermined condition, the vehicle characteristics in a predetermined direction that seems to correspond to the intention. Is changing. In this method, the vehicle characteristics are changed with the characteristics of the operation that are considered common to general humans, so the state estimation according to each driver's individuality is not made in the true sense, and it is wrong for some people There is a risk of being estimated to the state.

  As described above, the conventional technology not only has problems to be considered with respect to noise and additional costs, but also has a lack of ability due to uniform indicators and conditions for estimating the driver state, and realizes safe driving of the car. It is not enough for this.

  The present invention has been made in view of the above circumstances, and learns the relationship between the driving operation of the driver and the external environment as needed, and recognizes the deviation from the normal internal state of the driver to support safe driving. The purpose is to provide a safe driving support system.

  In order to achieve the above object, a safe driving support system according to the present invention is a safe driving support system that estimates the state of a driver driving a mobile body and provides support for safe driving, and includes an external environment of the mobile body. Using the stochastic state transition model to learn the correspondence between the driving environment risk and the driving operation of the driver, and build a learning model Based on the model learning unit and the learning model, the driver's risk recognition state is estimated as the driver's internal state from the driving operation data including the vehicle behavior data as a reflection of the driving operation of the driving driver, and the probabilistic state A state estimation unit that outputs an expected value of the existence probability of all states in the transition model as an estimation result of the internal state of the driver; Comparing the disk and and the internal state, characterized in that it comprises a driving support unit for obtaining support information according to the safe operation of the moving object.

  According to the present invention, the relationship between the driving operation of the driver and the environment outside the vehicle can be learned at any time during driving, and a safe driving can be supported by recognizing a deviation from the usual internal state of the driver. .

  Embodiments of the present invention will be described below with reference to the drawings. 1 to 13 relate to an embodiment of the present invention, FIG. 1 is a basic configuration diagram of a safe driving support system, FIG. 2 is an explanatory diagram showing a learning phase and an estimation phase of the safe driving support system, and FIG. FIG. 4 is an explanatory diagram showing a stochastic state transition, FIG. 5 is a graph showing a state transition probability after learning, and FIG. 6 is a graph showing an output probability after learning. FIG. 7 is an explanatory diagram showing the state estimation algorithm, FIG. 8 is an explanatory diagram showing the probability distribution when the estimation reliability is high, and FIG. 9 is an explanatory diagram showing the probability distribution when the estimation reliability is low. 10 is an explanatory diagram showing the difference in reliability depending on the winner unit distance, FIG. 11 is an explanatory diagram of the unit distance ratio total, FIG. 12 is a graph showing an example of calculation of the reliability index, and FIG. 13 is an explanatory diagram showing the estimation test result. is there.

  The safe driving support system according to the present invention recognizes a deviation from the normal internal state of the driver by comparing the risk included in the external environment with the risk recognized by the driver when driving a moving body such as an automobile. Thus, it is possible to provide appropriate driving assistance without unnecessary intervention, and it is possible to improve the safety when driving a car while eliminating the “dissonance between system and driver”.

  Hereinafter, in the present embodiment, driving assistance for automobiles will be described. The safe driving support system according to the present embodiment uses operation data such as a steering wheel, an accelerator, and a brake including vehicle behavior data as a reflection of the driving operation of the driver as main data for estimating the state of the driver. As a driver state estimation index, the “relationship between the vehicle surrounding environment and the operation” in the normal driving of the driver is used as a probabilistic reference model without using uniform conditions. This reference model is updated sequentially by continuously driving the vehicle, and the state estimation accuracy is improved according to the individual.

  Specifically, as shown in FIG. 1, the safe driving support system 1 in the present embodiment includes a single computer system or a plurality of computer systems connected via a network or the like, such as a camera or a radar. An external environment recognition unit 2 for recognizing a risk (driving environment risk) included in the external environment around the vehicle from external information by the sensing device, an operation feature quantity quantization unit 3 for quantizing the feature quantity of the driving operation data of the driver, A model learning unit 4 that learns the correspondence between driving operation (operation feature amount) and recognition environment (risk level), and estimates the internal state of the driver from the relationship between the learned model and the current operation data / exterior environment The state estimation unit 5, the reliability evaluation unit 6 that evaluates the reliability of the internal state estimation of the driver, and the vehicle is reduced according to the estimated driver internal state. It obtains support information for operating the is configured to include an alarm and support part 7 for such warnings and operation support. In this embodiment, an in-vehicle camera is used as a sensing device for the external environment.

  As will be described later, the reliability evaluation unit 6 may be configured as part of the operation feature quantization unit 3 and the state estimation unit 5.

  As shown in FIG. 2, the system operation of the safe driving support system 1 includes a “learning phase” in which a model of the internal state of the driver is learned, and an internal state of the driver is estimated based on the obtained model. It is divided into “estimation phase”.

  In the learning phase, when the vehicle front is imaged with an in-vehicle camera, and the risk level of the driving environment is recognized by learning, the correspondence between the driving environment risk level and the operation feature amount data obtained by quantizing the feature amount of the driving operation data of the driver Learn relationships, get learned parameters and build learning models.

  In the estimation phase, the relationship between the operation feature amount and the travel environment risk level is acquired using the learned model parameters, and the current internal state of the driver is estimated. The estimated current internal state of the driver is compared with the current driving environment risk level to assess whether the vehicle is inherently safe or dangerous.

  The learning phase and the estimation phase do not actually proceed separately from each other, but both proceed simultaneously. As a result, it is possible to realize a function that can adapt to changes in the environment in which the driver is driving and changes in the characteristics of the driver itself, and to provide a system that increases the performance as the vehicle travels. Can do. In addition, the model parameters in the learning phase can all be acquired by online learning while driving. However, it is recognized that the learning speed can be improved by using offline learning in part with a simulator. An improvement in accuracy can be expected.

  Next, the function of each part of the safe driving support system 1 will be described. The exterior environment recognition unit 2 degenerates the state of the exterior environment into a single scalar value (or vector) by learning the relationship between the feature quantity of the image obtained from the in-vehicle camera and the risk level at that time. And transmitted to the model learning unit 4.

  For the recognition of the risk level from the image feature amount, for example, the technology of the online risk learning system proposed in Japanese Patent Application No. 2007-77625 by the present applicant can be adopted. As described in detail in Japanese Patent Application No. 2007-77625, this technique is based on an image feature amount (edge information, motion information, color information, etc.) of an N-dimensional vector depending on an event such as a sudden accelerator return operation or a brake depression. Is converted into a one-dimensional state, and the risk contained in the environment is learned and recognized from the correlation between this state and the teacher information created from the vehicle information (driver operation information).

  In the present embodiment, an example using the risk level extracted from the image feature amount will be described. However, the risk level is not limited to this, and for example, the risk level is extracted from the inter-vehicle distance or the like. Anyway.

  The operation feature quantity quantization unit 3 quantizes the data from the sensor or the operation data of the driver such as the accelerator, the brake, the steering, etc. acquired via the in-vehicle network (not shown) according to the feature of the measurement frequency, This is transmitted to the model learning unit 4. In other words, the driving operation data has a very large amount of information as it is, and it is difficult to handle the relationship with the risk. For this reason, statistical processing to learn features while preventing loss of information contained in the data by appropriately quantizing in consideration of the distribution (appearance tendency) of the observed data Is possible.

  Quantization of observation data can be performed by data division or data degeneration using thresholds. In this embodiment, driving operation data is quantized using a self-organizing map (SOM). . SOM is a type of neural network that models the visual cortex of the cerebral cortex of living organisms. Each unit arranged in M dimensions has a vector value (usually called the weight of the connection with the input). The winner unit is determined based on the vector distance.

  Then, the reference vector values of the winner unit and the surrounding units are updated so as to approach the input vector. By repeating this, competitive learning is performed so that the entire distribution of the input data can be expressed optimally, the dimensions of the input information are compressed based on this competitive learning, and clustering and visualization are performed according to the characteristics of the data Can do.

  In this embodiment, an example in which input data is degenerated by unsupervised competitive learning using S0M will be described. However, a vector quantization (LVQ; Learning Vector Quantization) model that is supervised competitive learning can also be used. It is.

  Examples of SOM parameters are shown below. The feature values adopted are the steering angular speed, accelerator opening, brake pressure, accelerator opening change, brake pressure change, and vehicle speed. However, absolute values are used for the rudder angular velocity, and the accelerator opening and the brake pressure are normalized to be dimensionless amounts from -100 (maximum brake pressure) to +100 (accelerator fully open). The change amount of the accelerator and the brake is also expressed as the change amount of the normalized value. As a result, the dimension of the input feature quantity is 4, which is compressed to one dimension by SOM and quantized to 256 units.

[SOM parameters]
Number of SOM dimensions 1
256 units
Input feature value ADSTR (steering angular velocity absolute value)
PEDAL (accelerator / brake operation normalized amount)
DPEDAL (PEDAL value change amount)
SPEED (vehicle speed)

  FIG. 3 shows data learned by SOM (however, only three of the four input dimensions), and data obtained by traveling in an urban area (corresponding to 5-hour driving) is learned by SOM. The measurement data is plotted like a cloud, and the one-dimensional SOM network is arranged in a string shape. It can be seen from FIG. 3 that learning is performed according to the density and distribution of measurement data, and that units are appropriately arranged. The measured data is quantized and reduced to one-dimensional data represented by a unit number by being approximated by the nearest SOM unit.

  The unit arrangement of the SOM is determined by learning measured driving data, and it can be expected that the same person will adapt to changes in driving characteristics due to time variations even for the same person.

  The model learning unit 4 acquires the relationship between the operation feature quantity and the risk level by learning, and enables probabilistic calculation for estimating the internal state of the driver. As shown in FIG. 4 (a), human behavior changes according to mental states such as relief, tension, anxiety, impatience, and anger, and transitions thereof, and is not necessarily deterministic but expressed as probabilistic behavior. can do. Similarly, as shown in FIG. 4 (b), the driver's driving behavior includes a stochastic operation output for scenes such as following a preceding vehicle, passing, parking, lane change, and merging. It appears.

  Therefore, the model learning unit 4 models the internal state of the driver using a Hidden Markov Model (HMM), which is a kind of probabilistic state transition model, as a reference model for expressing the probabilistic behavior of human behavior. Turn into. The HMM is a model that assumes that a target internal state (state) transitions by a probabilistic conditional branch and that a signal is output to the outside with a different probability depending on the transitioned state.

  In the model using the HMM, in the task of estimating the risk level that the driver is aware of, the risk level that is currently conscious corresponds to the state of the HMM, as shown in FIG. Observed driving operation data corresponds to a signal output to the outside. FIG. 4C illustrates an example in which the risk levels are set to 5 levels and numbers 1 to 5 are assigned to the respective states. The number 1 recognizes that the risk of the external environment is the lowest. The state, number 5, indicates a state where the driver recognizes that the risk of the external environment is the highest. In addition, the risk level that is actually handled is 10 levels of 0, 1,.

  In this way, the driving operation data is discretized, the appearance tendency of the data is obtained, and the internal state of the driver is approximated as a probabilistic model, so that it is deterministically derived from the external environment like the operation data in actual driving. It is possible to appropriately handle information that cannot be handled. However, in order to estimate from which state the operation data observed at the time of estimation is output, it is necessary to perform two probability calculations: a state transition probability and an operation output probability.

  For this reason, the model learning unit 4 first learns the state transition probability by a statistical method based on the scalar value transmitted from the vehicle exterior environment recognition unit 2, and then uses the data from the operation feature quantization unit 3. Based on this, the observation probability distribution (operation output probability) of the operation feature amount for each state is learned. For example, the steering angle, the accelerator opening, the brake pressure, and the speed, yaw rate, acceleration, and the like that reflect the operation are discretized as appropriate, and the number of observations for each discrete value is counted to statistically calculate the probability.

  Hereinafter, calculation processing of the state transition probability and the operation output probability using the HMM will be described.

[Specification example of HMM]
Drive frequency 3Hz
Number of states 10
Output signal SOM unit number (0 to 255)
Network structure All-state interconnection type

[Calculation of state transition probability]
In general, when driving a car, it is unlikely that there will be a driver who always causes an accident even though it is unintentional every time the car is driven. In other words, it can be regarded macroscopically that the driver is appropriately performing a driving operation corresponding to the risk level of the driving environment. Based on these assumptions, when data is collected and statistically processed over a relatively long time range, it is assumed that the transition of the internal state of the driver in the HMM depends on the transition of the risk level of the driving environment. Can do.

Therefore, the transition probability is calculated according to the following procedures (1-1) to (1-5), and the transition probability of the driver internal state is calculated by obtaining the transition probability of the risk level at the time of learning.
(1-1) Define the drive frequency of the state transition model (for example, 3 Hz)
(1-2) Define the number of internal states (for example, 10 states)
(1-3) Discrete the risk level by the number of states (1-4) Count the number of transitions between each state (1-5) Calculate the statistical transition probability between each state

  In a simple example, if the time step of the state transition is fixed to a constant value and the internal state of the driver transitions in synchronization with the transition of the environmental state, the calculation becomes simple. Alternatively, it is possible to define a state separately according to the length of time for stopping in a specific scalar value range.

  FIG. 5 shows a graph of the state transition probability of the HMM obtained by learning from the measurement data in the urban area. The larger the state number, the higher the risk level. From this graph, in the framework of the HMM drive frequency of 3 Hz, the self-transition (transition staying in the same numbered state), the transition probability for one state above and below is large, and the transition probability that the risk level suddenly decreases in the high risk state It can be seen that it is relatively large. This is generally equivalent to the fact that moderate and moderate risks rise and fall continuously, and high risk factors disappear rapidly with the passing of the vehicle. It can be judged that there is nothing.

[Calculation of output signal probability]
The SOM described above is used for learning the output probability of the operation data feature in the state transitioned to a certain state. The measured operation data is dimensionally compressed and quantized into SOM unit numbers, and the statistical output signal probability is calculated by counting the number of times each unit number is observed. Since the output probability of each unit obtained for each state is different, it can be modeled that the operation tendency changes depending on the risk level, and the output probability for each state as shown in FIG. 6 can be obtained.

  When the above state transition probability and output probability are acquired by learning, the state estimation unit 5 next estimates the internal state of the driver using both pieces of information. The internal state is estimated using a forward algorithm that forwards the trellis between the event and each state in consideration of online application.

  Specifically, the forward probability (state existence probability) α is calculated by performing sequential calculation according to the following steps (2-1) and (2-2). In the following equations, π: prior probability, a: state transition probability, b: output signal probability, o: zero vector, and subscript 1 of each variable indicates an initial value.

(2-1) The forward probability is initialized for each state i = 1,.
α ₁ (i) = π ₁ b ₁ (o ₁ )
(2-2) For each time t = 1,..., T−1 and each state j = 1,. Note that Σ in the recursive calculation is the sum for j = 1 to N.
α _{t + 1} (j) = [Σα _t (i) a _ij ] b _j (o _{t + 1} )
In the state estimation by this forward algorithm, as shown in FIG. 7, all transition probabilities are calculated at each step of time t = 0, 1, 2, 3, 4,. Therefore, the internal state can be estimated with high definition. In the first step of probability calculation, the prior probability obtained from the state transition probability of the HMM is used as the existence probability in each state.

From the state existence probability α for each state obtained by the sequential calculation, an expected value μ is calculated as shown below. The expected value μ is output as a driver internal state in which the state number is precisely expressed by a continuous numerical value instead of a discrete value, and becomes a risk level that the driver is aware of. Note that x is the value of the random variable (here, the state number), and Σ is the sum of i = 1 to N.
μ = Σx _i α _i

  In addition, the internal state of the driver estimated by the state estimation unit 5 is evaluated by the reliability evaluation unit 6, and uncertain estimation output is suppressed. As shown in the following (3-1) and (3-2), this reliability is evaluated by a method of evaluating in consideration of the probability distribution shape between states, and an error in quantizing the operation feature amount. The estimation reliability of the driver internal state is evaluated using both or any one of these evaluation methods.

  The reliability evaluation unit 6 may be a part of the state estimation unit 5 and the operation feature quantity quantization unit 3. The state estimation unit 5 performs reliability evaluation based on the probability distribution shape between states, and operates characteristics. The quantity quantizing unit 3 may evaluate the reliability based on the quantization error of the operation feature quantity.

(3-1) Reliability Evaluation from Probability Distribution Shape When the estimation is performed normally, as shown in FIG. 8, there is a high probability distribution peak near the expected value, and the other probabilities are low. Ideally. In this state, it can be said that the estimation is performed with certainty. On the other hand, as a typical example of when the certainty of estimation is lowered, there is an example as shown in FIG. That is, there is a case where no clear peak exists in the probability distribution as shown in FIG. 9A, or a plurality of peaks exist in the probability distribution as shown in FIG. 9B.

  Therefore, by expressing the difference in the shape of the probability distribution numerically, not only the estimation result of the driver internal state but also the reliability of the estimation result can be output. As an index for evaluating the reliability by quantifying the shape of the probability distribution, as shown in the following (a), (b), (c), the state having the entropy, the variance, the maximum existence probability and the adjacent state The total probability or the like can be used, and the reliability is evaluated using all or at least one of these indices.

(A) Entropy The entropy H of the probability distribution is calculated by the following formula, and the reliability is evaluated. Entropy is maximized when the probability distribution is uniform, and it can be evaluated that the smaller the entropy, the higher the reliability.
H = Σp _i logp _i
Where p _{i is the} probability of existence of state i
N: Number of states
Σ: Sum of i = 1 to N

(B) Variance The variance V of the probability distribution is calculated by the following formula, and the reliability is evaluated. The variance is minimized when the probability of the state having the maximum existence probability is 1. Therefore, it can be evaluated that the smaller the variance, the higher the reliability.
V = Σ (x _i −μ) ² p _i
Where p _{i is the} probability of existence of state i
N: Number of states
x _i : Risk level value of state i
Σ: Sum of i = 1 to N

(C) Total Probability of State with Maximum Existence Probability and Adjacent State Calculate the total probability Psum between the state with the maximum existence probability and the adjacent state by the following formula, and evaluate the reliability. The total probability Psum is 1 when the state having the maximum existence probability has the probability 1 or the adjacent state has all the remaining existence probabilities. Therefore, it can be evaluated that the larger the value, the sharper the probability distribution, and the higher the reliability.
Psum = Σp _i
However, Σ: i = K−1 to K + 1
K: State number with maximum existence probability

(3-2) Reliability Evaluation Based on Operation Feature Quantity Quantization Error As described above, the operation feature quantity quantization unit 3 learns the distribution of input operation data using the SOM, and changes the distribution shape into the distribution shape. By arranging the units along the line, the data is compressed and quantized, and it plays a role of compressing various combinations of driving operations to a level that is easy to handle in estimating the internal state.

  The quantization by the SOM is performed by representing the operation data with the SOM unit closest to the observed operation data input. At this time, the representative unit is the “winner unit”. Here, the distance in the feature space between the winner unit and the input operation data is defined as “winner unit distance”, and this winner unit distance is an index for evaluating the reliability of the estimation result in the state estimation unit 5. Use as one.

  The winner unit distance corresponds to a quantization error when the input data is quantized, and the error is small when the distance is small, and the error is large when the distance is large. When an output having a large error is input to the state estimation unit 5 in the subsequent stage, it can be said that the reliability of the estimation result is inevitably lower than that of an input having a small error. Therefore, this winner unit distance can be used as one of the reliability evaluation indexes.

  For example, as shown in 109, when the same unit is a winner but the winner unit distance is different, as shown in FIG. 10 (a), learning is performed when the distance d from the input data to the winner unit is small. The approximation / compression of the time is performed well, and it can be said that the quantization error is small. On the other hand, as shown in FIG. 10B, when the distance d from the input data to the winner unit is large, data different from the distribution of the operation data at the time of learning is input, and the quantization error is large. I mean. Therefore, it can be considered that the reliability of the driver internal state estimation in such a state is low.

In addition to using the “winner unit distance” as a reliability evaluation index, the total of the “ratio of the distance between the winner unit distance and other units” can also be used as the reliability evaluation index. Therefore, as shown in FIG. 11, the distance from the input data to the winner unit is dw, the distance to the other units is di, and the index S is defined by the sum of the unit distance ratio di / dw as follows: To do.
S = Σ (di / dw)
However, Σ: Sum of i = 1 to N−1
N: Total number of units

  As shown in FIG. 11A, when there is a unit that appropriately quantizes the input in the feature amount space, the index S takes a large value, and as shown in FIG. When an input is observed in a distant space, the index S has a small value. Therefore, it can be considered that the reliability is high when the value of the index S is large, and the reliability is low when the value of the index S is small.

  Compared with the method using the winner unit distance directly, the method using the index S considers the arrangement in the feature amount space of the entire SOM unit. Even when the distance is small, the distance to many units is almost the same, and the case where the quantization is unstable can be detected. That is, it is possible to calculate the reliability of the estimation result in the state estimation unit 5 in the subsequent stage from the viewpoint of the stability evaluation of quantization.

[Output of the driver's conscious risk level]
The expected value calculated from the probability of existence in each state as the internal state of the driver by the state estimation unit 5 is filtered by a primary low-pass filter (for example, a cutoff frequency of 0.3 Hz) or the like, and then is sent to the alarm / support unit 7 Is output. This output value is a driver internal state in which state numbers are expressed by continuous numerical values, corresponds to a risk level recognized by the driver, and can be regarded as a risk level handled by the driver.

  At this time, the reliability evaluation unit 6 also outputs the reliability for the estimation result of the driver internal state. When the reliability is remarkably low below a preset threshold, the output from the operation feature quantization unit 3 or the state estimation unit 5 is restricted, for example, to prevent inaccurate information from being transmitted to the subsequent stage. .

  FIG. 12 shows a calculation example of the reliability index for the state estimation. In FIG. 12, along with the risk level of the external environment and the driver recognition risk level estimated to be recognized by the driver, entropy, variance, total adjacent state probability, and SOM winner unit distance are shown as reliability indicators for the estimation. When the characteristic between each index, that is, entropy, variance, SOM winner unit distance is small and the reliability is relatively high, the adjacent state total probability is large, and conversely, the entropy, variance, SOM winner unit distance is When it is large and the reliability is relatively low, it can be seen that the adjacent state total probability is small.

  By comparing the internal state of the driver, that is, the risk level recognized by the driver, with the driving environment risk level value, it is possible to evaluate whether the vehicle is in an inherently safe or dangerous state. That is, if the alarm / support unit 7 is close to the state to which the scalar value currently obtained from the outside environment recognition unit 2 belongs as a result of comparison with the driving environment risk level value, the driver recognizes the environment normally and is normal. It is determined that an operation is being performed. On the other hand, when the state based on the expected value from the state estimation unit 5 is different from the state to which the scalar value currently obtained from the outside environment recognition unit 2 belongs, the internal state of the driver differs from the level required by the environment. It is determined that

  For example, if the estimated risk level of the state is lower than the risk level obtained from the vehicle environment recognition unit 2, the driver's risk recognition level is low, and there is a high possibility that safety will be impaired. Alert and support from Department 7. Conversely, when the estimated risk level of the state is higher than the risk level obtained from the outside environment recognition unit 2, it is estimated that the driver is excessively tensioned, and the safety may be similarly impaired. Alarm is given as high probability.

  FIG. 13 shows a part of data measured in the state estimation running test. From S1 street (one side lane) to the S2 street (one lane) through the narrow path (no center line) indicated by the symbols A, B, C, and further from the position indicated by symbol D, S3 street ( Changes in the environmental risk level and the internal state risk level when traveling up to one lane on one side are shown together with changes in the operation feature amount.

  In the test results shown in FIG. 13, the driver is driving with a normal arousal level and a sense of tension based on the relationship between the operation feature amount and the risk level, and corresponds to a high risk level situation such as an intersection or a narrow road. Can be confirmed from the estimated internal state, that is, the risk level perceived by the driver is high. In particular, the points A, B, C, and D shown in the graph are characteristic high-risk events, mainly intersections and narrow streets. When the other risk level is high, it is mainly a portion where a pedestrian, a bicycle, an oncoming vehicle, or the like is encountered, and it can be seen that the same estimation result is obtained also in this portion.

  As described above, the safe driving support system according to the present embodiment uses the data such as the accelerator opening, the brake pressure, and the steering angle that are mounted on the vehicle as inputs, and the correspondence relationship with the risk level of the driving environment is stochastic. The characteristics of the driving operation of the driver are learned by approximation with the state transition model, and the correspondence between the risk level of the driving environment and the internal state of the driver is evaluated. This system does not judge whether alarms, operation assistance, or other interventions are necessary based only on the risk level of the external environment, but rather uses the evaluation index as to whether the driver correctly understands the external environment. This eliminates the discomfort and annoyance caused by unnecessary interventions that occur when judging whether or not a safety system intervention is necessary based only on the risk of the driving environment, and provides appropriate support when necessary. It is possible to construct an advanced safety system adapted to the driver.

  Moreover, in this system, since the driving operation data is used as the main data for state estimation, it is possible to divert the sensors for observing the operation of the steering wheel and accelerator / brake already installed for various vehicle control, There is no need for additional cost as in the case of a measuring device. Since these sensors are relatively robust against noise, it is possible to stably estimate the state, and further, the normal driving operation of the driver is not hindered and the posture is not restrained. Unnecessary interference that adversely affects the estimation can be eliminated.

  In addition, in this system, since the correspondence relationship between the environment outside the vehicle and the driving operation is a normative model, there is no need to determine conditions and templates in advance, and by updating the model through learning, individual differences for each driver and each driver Thus, the state estimation accuracy can be improved by repeated traveling.

  In this case, the learning model that becomes the norm is used as a probabilistic state transition model, and the operation data with variations is statistically processed. Relevance can be learned. In addition, it is possible to estimate which risk level the estimated internal state corresponds to by matching the state transition of the probabilistic state transition model with the transition of the risk level recognized from the external environment at the time of learning.

  In other words, when the risk level corresponding to the estimated internal state is lower than the recognized external environment risk level, the driver is not in a safe driving state even if the external environment risk level is high.・ Operation support can be performed. On the other hand, when the risk level corresponding to the estimated internal state is the same or higher than the recognized external environment risk level, the driver is in a state that can handle the environment even if the external environment risk level is high. Since it can be estimated, useless warnings and unnecessary operation support can be suppressed.

  Furthermore, since the internal state of the driver is output as the expected value of the existence probability of all states in the probabilistic state transition model, it is possible to estimate the risk level that the driver is aware of online while driving, and from moment to moment It becomes possible to construct an advanced system that can cope with the changing driving environment in real time.

  In addition, since the internal state of the driver is output with the expected value of the existence probability of all states, it is easy to estimate the internal state of the driver and simultaneously evaluate the reliability of the estimation. The driving support can be adaptively changed according to the estimation result. This can reduce or eliminate warnings and assistance based on uncertain estimation results and suppress driver distrust, discomfort, and misunderstandings.For example, when the reliability of estimation is low, online learning By setting the frequency and importance level of the above to high, it is possible to learn an environment with low estimation performance more efficiently. Moreover, it is possible to reduce overconfidence and misunderstanding of the safety system by presenting the estimated reliability to the driver.

Basic configuration of safe driving support system Explanatory drawing showing the learning phase and estimation phase of the safe driving support system Explanatory diagram showing the network of operation features and self-organizing maps Explanatory diagram showing stochastic state transition Graph showing state transition probability after learning Graph showing output probability after learning Explanatory diagram showing algorithm for state estimation Explanatory drawing showing the probability distribution when the reliability of estimation is high Explanatory drawing showing the probability distribution when the reliability of estimation is low Explanatory diagram showing the difference in reliability depending on the winner unit distance Explanation of total unit distance ratio Graph showing calculation example of reliability index Explanatory diagram showing estimation test results

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Safe driving support system 2 External environment recognition part 3 Operation feature-quantization part 4 Model learning part 5 State estimation part 6 Reliability evaluation part 7 Warning / support part

Claims (8)

A safe driving support system that estimates the state of a driver driving a moving body and provides support for safe driving,
An environmental risk recognition unit that senses the driving environment risk included in the external environment by sensing the external environment of the mobile body,
A model learning unit that learns the correspondence between the driving environment risk and the driving operation of the driver using a probabilistic state transition model, and constructs a learning model;
Based on the learning model, the driver's risk recognition state is estimated as the driver's internal state from the driving operation data including the vehicle behavior data as a reflection of the driving operation of the driving driver, and all states in the stochastic state transition model A state estimation unit that outputs an expected value of the existence probability of the driver as an estimation result of the internal state of the driver;
A safe driving support system, comprising: a driving support unit that compares the driving environment risk with the internal state and acquires support information related to safe driving of the mobile body.   The safe driving support system according to claim 1, further comprising a reliability evaluation unit that evaluates the reliability of the estimation of the internal state.   3. The safe driving support system according to claim 2, wherein the driving support based on the support information is adaptively changed based on the reliability evaluation result.   The safety driving support system according to claim 2, wherein the reliability of the estimation of the internal state is evaluated based on an index representing a probability distribution shape of each state in the probabilistic state transition model.   The safe driving support system according to claim 2 or 4, wherein the reliability of the estimation of the internal state is evaluated based on a quantization error of the driving operation data.   5. The safe driving support system according to claim 4, wherein entropy of probability distribution of each state is used as the index.   5. The safe driving support system according to claim 4, wherein a variance of probability distribution of each state is used as the index.   5. The safe driving support system according to claim 4, wherein a total probability of a state having a maximum existence probability and an adjacent state is used as the index.