JP2013111348A - State determining device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy in determining an arousal level or a concentration level.SOLUTION: An alpha wave feature amount calculation section 13 calculates a feature amount of alpha waves from electroencephalogram signals. An eye fixation-related potential feature amount calculation section 14 calculates a feature amount of an eye fixation-related potential on the basis of an eyeball moving amount and electroencephalogram signals of a subject. An arousal level determination section 15 determines an arousal level of the subject on the basis of the feature amount of the alpha waves calculated by the alpha wave feature amount calculation section 13 and the feature amount of the eye fixation-related potential calculated by the eye fixation-related potential feature amount calculation section 14.

Description

  The present invention relates to a state determination device and a state determination method for determining a state of arousal or concentration of a subject based on a brain wave signal of the subject.

  Due to monotonous driving or fatigue, the driver tends to feel sleepy. That is, the driver's arousal level is likely to decrease. In addition, when attention is paid to conversation or thoughts, the driver's degree of concentration on driving tends to decrease. Therefore, by measuring the driver's brain wave signal and determining the driver's arousal state (for example, the degree of arousal level) or the concentration state (for example, the degree of concentration level) for driving, A method for assisting driving has been proposed.

  As a method for determining the degree of arousal, a method using α waves is known. The α wave is a component of a specific frequency (8 to 13 Hz) obtained by Fourier transforming an electroencephalogram signal. Alpha waves are rhythms that are noticeable when closed or in a relaxed state. On the other hand, it is known that the power spectrum value of the α wave increases with a decrease in the arousal level of the subject (for example, the driver in the above driving case) even when the eyes are opened. A determination method that uses this feature to determine that the awakening level of the subject has decreased when the power spectrum value of the α wave increases is generally known.

  On the other hand, as a method of determining the degree of concentration, in recent years, the attention amount of the subject (the degree of concentration of the subject with respect to the visual target) with respect to the visual target being viewed by the subject using the eye-fixation related potential (EFRP) of the electroencephalogram. ) Is being studied. According to this method, for example, it is possible to examine a driver's attention state for driving (concentration level for driving) including a state of consciousness looking aside. “Eye-holding-related potential” is related to the end of rapid eye movement (saccade) when a person is working or looking at an object (ie, the start of eye-holding). A transient potential fluctuation in the brain. A positive component that appears significantly in the occipital region in the vicinity of about 100 milliseconds from the start point of the eyeball retention among the components of the eyeball retention-related potential is referred to as “lambda (λ) reaction”. The lambda response has a feature that it varies depending on the degree of concentration on the visual target. For example, Patent Document 1 discloses using this feature to determine a driver's degree of concentration with respect to driving by measuring an attention amount with respect to a visual target during driving based on an amplitude value of a lambda response. Yes. Specifically, when the amplitude value of the lambda reaction decreases, it is determined that the driver's degree of concentration on driving has decreased.

JP 2007-125184 A

  In the above prior art, the degree of arousal of a subject is determined based only on α waves. Moreover, in the said prior art, a test subject's concentration degree is determined only based on a lambda reaction. However, when the degree of concentration on driving changes due to thoughts, etc., the power spectrum value of the α wave may fluctuate even though the actual arousal level does not change. Judgment may not be possible. Similarly, even when the arousal level changes during driving, the amplitude value of the lambda reaction may fluctuate even though the actual concentration level does not change, and the conventional technique cannot accurately determine the concentration level. there is a possibility.

  The objective of this invention is providing the state determination apparatus and state determination method which can improve the determination precision of a wakefulness degree and the concentration degree.

  A state determination device according to an aspect of the present invention is a state determination device that determines the state of the subject based on a brain wave signal of the subject, and calculates an α wave feature amount from the brain wave signal. Based on the calculation means, the eye movement amount of the subject and the electroencephalogram signal, the eye retention related potential feature quantity calculation means for calculating the feature quantity of the eye retention related potential, and the alpha wave feature quantity calculation means determining means for determining the degree of arousal of the subject or the degree of concentration of the subject based on the feature quantity of the α wave and the feature quantity of the eyeball-related potential calculated by the eyeball-related potential feature quantity calculating means; The structure to comprise is taken.

  A state estimation method according to an aspect of the present invention is a state determination method for determining a state of the subject based on a brain wave signal of the subject, calculating an α-wave feature amount from the brain wave signal, and Based on the amount of eyeball movement and the electroencephalogram signal, a feature amount of the eyeball-related potential is calculated, and based on the calculated feature amount of the α wave and the calculated feature amount of the eyeball-related potential, the subject The degree of awakening or the concentration of the subject is determined.

  According to the present invention, in the determination of the arousal level or the concentration level, the driver's action to increase the concentration on driving against the decrease in the awakening level, and the information processing of the driver accompanying the decrease in the concentration level It becomes possible to determine the degree of arousal or the degree of concentration in consideration of a decrease in load, and the accuracy of determining the degree of arousal and the degree of concentration can be improved.

The block diagram which shows the main structures of the state determination apparatus which concerns on Embodiment 1 of this invention. The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 1 of this invention. A figure showing an example of the electrode position of the electroencephalograph The figure which shows an example of the calculation method of a random reaction amplitude value The block diagram which shows the internal structure of the arousal level determination part which concerns on Embodiment 1 of this invention. FIG. 3 is a block diagram showing experimental conditions according to Embodiment 1 of the present invention. The figure which shows the characteristic of the alpha wave power spectrum value and lambda reaction amplitude value which concern on Embodiment 1 of this invention The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 1 of this invention. The figure which shows the correspondence of the alpha wave power spectrum value which concerns on Embodiment 1 of this invention, a lambda reaction amplitude value, and a wakefulness level The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 2 of this invention. The block diagram which shows the internal structure of the arousal level determination part which concerns on Embodiment 2 of this invention. The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 2 of this invention. The figure which shows the discrimination | determination straight line in the alpha-lambda plane based on Embodiment 2 of this invention. The figure which shows the correspondence of the arousal level evaluation value and arousal level which concern on Embodiment 2 of this invention. The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 3 of this invention. The block diagram which shows the internal structure of the arousal level determination part which concerns on Embodiment 3 of this invention. The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 3 of this invention. The block diagram which shows the main structures of the state determination apparatus which concerns on Embodiment 4 of this invention. The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 4 of this invention. The block diagram which shows the internal structure of the concentration degree determination part which concerns on Embodiment 4 of this invention. The figure which shows the characteristic of the alpha wave power spectrum value and lambda reaction amplitude value which concern on Embodiment 4 of this invention The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 4 of this invention. The figure which shows the correspondence of the alpha wave power spectrum value which concerns on Embodiment 4 of this invention, a lambda reaction amplitude value, and the concentration level The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 5 of this invention. The block diagram which shows the internal structure of the concentration determination part which concerns on Embodiment 5 of this invention. The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 5 of this invention. The figure which shows the discrimination | determination straight line in the alpha-lambda plane based on Embodiment 5 of this invention. The figure which shows the correspondence of the concentration degree evaluation value and concentration degree which concern on Embodiment 5 of this invention. The block diagram which shows the structure of the state determination apparatus which concerns on Embodiment 6 of this invention. The block diagram which shows the internal structure of the concentration degree determination part which concerns on Embodiment 6 of this invention. The flowchart which shows operation | movement of the state determination apparatus which concerns on Embodiment 6 of this invention. Correspondence table related to surrounding environment according to Embodiment 6 of the present invention

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the following, a case will be described in which the “subject” who is the determination target of the arousal level or the concentration level in the state determination device is the “driver” who is driving. However, the “subject” as a determination target of the arousal level or the concentration level in the state determination device is not limited to the driver.

<Embodiment 1>
FIG. 1 is a block diagram showing a main configuration of state determination apparatus 10 according to the present embodiment. The state determination device 10 shown in FIG. 1 determines the state of the driver. In the state determination device 10, the α-wave feature value calculation unit 13 calculates an α-wave feature value (for example, an α-wave power spectrum value) from the electroencephalogram signal, and the eyeball retention-related potential feature value calculation unit 14 Based on the amount of movement of the eyeball and the electroencephalogram signal, a feature amount (for example, lambda response amplitude value) of the eyeball retention-related potential is calculated, and the arousal level determination unit 15 calculates the α wave calculated by the α wave feature amount calculation unit 13. The driver's arousal level is determined based on the feature amount and the feature amount of the eyeball-related potential calculated by the eyeball retention-related potential feature amount calculation unit 14.

  FIG. 2 is a block diagram showing a configuration of the state determination device according to the present embodiment. For example, in a safe driving support system (not shown) that provides driving assistance such as warning or braking to the driver based on the driver's arousal level, the state determination device 10 (FIG. 2) according to the present embodiment is The driver's arousal level is determined as the driver's state. That is, the state determination device 10 according to the present embodiment is a wakefulness determination device that determines the state of the driver's wakefulness.

  The state determination device 10 includes an electroencephalogram signal acquisition unit 11, an eyeball movement amount acquisition unit 12, an α-wave feature amount calculation unit 13, an eyeball retention related potential feature amount calculation unit 14, an arousal level determination unit 15, and an arousal level. And a correspondence table 16.

  The electroencephalogram signal acquisition unit 11 acquires the driver's electroencephalogram signal, and outputs the acquired electroencephalogram signal to the α-wave feature quantity calculation unit 13 and the eyeball retention-related potential feature quantity calculation unit 14.

  For example, the electroencephalogram signal acquisition unit 11 acquires an electroencephalogram signal measured by an electroencephalograph that measures a potential change (electroencephalogram signal) of an electrode attached to the driver's head. The electroencephalograph may be a head-mounted electroencephalograph. In this case, it is assumed that the driver is wearing the electroencephalograph in advance. Alternatively, an electroencephalograph electrode may be embedded in a car seat (mainly a headrest), and when the driver is seated, the electroencephalograph electrode may be in contact with the driver's head. When the electroencephalograph is attached to the driver's head, the electrodes of the electroencephalograph are arranged so as to contact a predetermined position on the driver's head. For example, in the electrode position shown in FIG. 3 (electrode arrangement of the international 10-20 method), the electrode is arranged so as to contact the head at the position of the back of the head (Oz) and the mastoid (A1 or A2). However, it is possible to measure electroencephalograms at sites other than the back of the head (Oz). For example, electrodes of an electroencephalograph may be arranged at a position around Oz such as Pz. For example, the electrode position of the electroencephalograph is determined from the reliability of signal measurement, the ease of wearing, and the like.

  Thereby, the electroencephalogram signal acquisition part 11 can acquire a driver | operator's electroencephalogram signal. The acquired electroencephalogram signal is sampled so that it can be processed by a computer. Data for a predetermined time is temporarily stored in a storage unit inside the electroencephalogram signal acquisition unit 11 and updated as needed. In addition, the influence of the noise mixed in an electroencephalogram signal can be reduced by performing beforehand the low-pass filter process of 15 Hz with respect to the electroencephalogram signal acquired (measured) in the electroencephalogram signal acquisition part 11, for example. In addition, the brain wave signal acquisition unit 11 is not limited to acquiring the driver's brain wave signal by measuring the driver's brain wave, but, for example, reads the driver's brain wave signal measured in advance and recorded in the storage unit. Thus, the driver's brain wave signal may be acquired.

  The eyeball movement amount acquisition unit 12 acquires the driver's eyeball movement amount in a predetermined time by measuring the driver's eyeball movement. The eyeball movement amount acquisition unit 12 outputs the acquired eyeball movement amount to the eyeball retention related potential feature amount calculation unit 14.

  The eye movement amount acquisition unit 12 measures the driver's eye movement based on, for example, an EOG (Electro-oculogram) method. The “EOG method” is a method for measuring eye movement from a change in potential of electrodes arranged on the left and right or above and below the eyeball. The EOG method utilizes the property that the cornea of the eyeball is positively charged with respect to the retina. Using this property, the eyeball movement amount acquisition unit 12 measures the eye movement from the potential change at the left and right or upper and lower electrodes of the eyeball. Moreover, the eyeball movement amount acquisition unit 12 may be a head-mounted measuring device, similar to the electroencephalogram signal acquisition unit 11 (electroencephalograph). Further, the eyeball movement amount acquisition unit 12 is not limited to acquiring the driver's eyeball movement amount by measuring the driver's eyeball movement. For example, the driver's eyeball is measured in advance and recorded in the storage unit. The driver's eyeball movement amount may be acquired by reading the movement amount.

  Further, the eyeball movement amount acquisition unit 12 may calculate the eyeball movement amount by an “corneal reflection method” or the like from an image acquired by a camera that captures the driver's face or eyeball instead of the EOG method. The “corneal reflection method” means that a near-infrared light source (point light source) irradiates the eyeball with near-infrared light, captures an image of the eyeball with a camera, and uses the captured image of the corneal reflection image of the light source on the pupil and corneal surface This is a method for detecting a position. In the “corneal reflection method”, the eyeball movement amount can be calculated from the positional relationship between the pupil and the cornea reflection image calculated in this way. Further, the eyeball movement amount acquisition unit 12 may be a device installed near the steering column in the driver's seat when shooting the driver's face, and is head-mounted when shooting the driver's eyeball. It may be a measuring instrument.

  The α wave feature amount calculation unit 13 calculates the power spectrum value of the α wave (the α wave feature amount) from the electroencephalogram signal acquired by the electroencephalogram signal acquisition unit 11. For example, the α-wave feature quantity calculation unit 13 performs frequency transformation on the electroencephalogram signal obtained by the electroencephalogram signal acquisition unit 11 for a predetermined time to obtain frequency component data, and calculates the power spectrum of the driver's electroencephalogram signal. . In general, in an electroencephalogram signal, a frequency component of 8 Hz or more and less than 13 Hz is called an α wave, a frequency component of 13 Hz or more is called a β wave, and a frequency component of 4 Hz or more and less than 8 Hz is called a θ wave. That is, the α wave feature amount calculation unit 13 calculates a power spectrum value (α wave power spectrum value or α wave feature amount) at 8 to 13 Hz corresponding to the α wave in the frequency component data of the electroencephalogram signal. The α-wave feature amount calculation unit 13 outputs the calculated α-wave power spectrum value to the arousal level determination unit 15.

  Based on the electroencephalogram signal of the predetermined time acquired by the electroencephalogram signal acquisition unit 11 and the eyeball movement amount of the predetermined time acquired by the eyeball movement amount acquisition unit 12, the eyeball retention related potential feature amount calculation unit 14. The lambda response amplitude value starting from the end point is calculated as an eyeball retention-related potential feature quantity. The eyeball retention-related potential feature amount calculation unit 14 outputs the calculated lambda response amplitude value to the arousal level determination unit 15.

  As an example, the processing for calculating the lambda response amplitude value from the electroencephalogram signal from the predetermined time TW to the present time and the amount of movement of the eyeball in the eyeball stop-related potential feature quantity calculation unit 14 will be described with reference to FIG. . FIG. 4 schematically shows a time series change (predetermined time TW) of an electroencephalogram signal and an eyeball movement amount. In FIG. 4, a plurality (six locations) of saccades 91 are detected in a predetermined time TW. In this case, as shown in FIG. 4, the electroencephalogram signal is cut out starting from the end point of each detected saccade 91. The plurality of cut out electroencephalogram signals 92 (five in FIG. 4) contain a lot of noise components due to the influence of the movement of the driver or the influence of electromagnetic waves generated from electronic devices existing in the vicinity. For this reason, the plurality of brain wave signals 92 are averaged. Thereby, the noise component in the electroencephalogram signal is smoothed, and the addition average electroencephalogram signal 93 from which only the feature component is extracted is obtained. The brain wave signal is cut out starting from the end point of the saccade, but FIG. 4 shows an example in which the range of −300 to 600 msec including before the saccade cutting is cut out as the electroencephalogram signal 92. In the addition average electroencephalogram signal 93 shown in FIG. 4, a characteristic reaction 94 that appears in the vicinity of about 100 milliseconds from the end of the saccade (that is, the start time of the addition average electroencephalogram signal 93 (0 milliseconds)) is the lambda reaction. . Therefore, the eyeball retention-related potential feature amount calculation unit 14 calculates the amplitude value of the lambda response 94 as a lambda response amplitude value indicating the amount of attention to visual information. As the lambda response amplitude value, the maximum average value in the section from the saccade end time to 100 milliseconds may be used in the addition average electroencephalogram signal 93, or the maximum value in the same section or the average amplitude in the same section may be used. Also good. The same section may be set from 50 milliseconds to 150 milliseconds from the end of the saccade.

  Further, it is known that when the size of the saccade 91 just before, that is, the amount of movement of the eyeball is large (for example, a predetermined threshold value or more), the calculated lambda response amplitude value also increases. Therefore, a change in the lambda response amplitude value due to the influence of the saccade size may be erroneously determined as a change in the driver's concentration.

  Therefore, the eyeball retention-related potential feature amount calculation unit 14 divides the lambda response amplitude value by the size of the immediately preceding saccade 91 in order to reduce the influence of the saccade size on the lambda response amplitude value (that is, normalization). By doing so, the normalized lambda response amplitude value may be calculated. Thereby, the eyeball retention related potential feature amount calculation unit 14 calculates a lambda response amplitude value (normalized lambda response amplitude value) independent of the size of the immediately preceding saccade 91, and accurately determines the driver's concentration degree. be able to.

  The arousal level determination unit 15 is based on the α wave power spectrum value calculated by the α wave feature amount calculation unit 13 and the lambda reaction amplitude value calculated by the eyeball retention related potential feature amount calculation unit 14. Determine the degree of arousal. Further, when it is determined that the driver's arousal level is low, the arousal level determination unit 15 further refers to the arousal level correspondence table 16 and drives from a level (awake level) indicating a plurality of arousal levels. The level of arousal of the person.

  FIG. 5 is a block diagram showing an internal configuration of the arousal level determination unit 15 according to the present embodiment. As shown in FIG. 5, the arousal level determination unit 15 includes a first arousal level determination unit 101 and a second arousal level determination unit 102.

The first arousal level determination unit 101 determines whether or not the α wave power spectrum value (α wave feature amount) calculated by the α wave feature amount calculation unit 13 is equal to or greater than a predetermined threshold (for example, threshold Th _α ). To do. That is, the first arousal level determination unit 101 performs the arousal level determination process by comparing the α wave power spectrum value with a predetermined threshold value. The first arousal level determination unit 101 outputs a determination result for the α wave power spectrum value to the second arousal level determination unit 102. Specifically, the first arousal level determination unit 101 determines that the driver is awake (a high arousal level) when the α-wave power spectrum value is less than a predetermined threshold.

The second arousal level determination unit 102 determines whether or not the lambda response amplitude value (eyeball retention-related potential feature amount) calculated by the eyeball retention-related potential feature amount calculation unit 14 is equal to or greater than a predetermined threshold value (for example, threshold value Th _λ ). Determine whether. That is, the second arousal level determination unit 102 performs the arousal level determination process by comparing the lambda response amplitude value with a predetermined threshold value.

  For example, the arousal level determination process in the second arousal level determination unit 102 is performed only when the determination result input from the first arousal level determination unit 101 indicates that the α-wave power spectrum value is equal to or greater than a predetermined threshold value. Is called. Specifically, the second arousal level determination unit 102 determines that the driver's arousal level is low when the lambda response amplitude value is equal to or greater than a predetermined threshold.

  In addition, the second arousal level determination unit 102 determines that the driver is awake (a state of high arousal) when the lambda response amplitude value is less than a predetermined threshold. The second arousal level determination unit 102 determines that the driver's arousal level is low as a result of the arousal level determination for the lambda response amplitude value (when the lambda response amplitude value is determined to be equal to or greater than a predetermined threshold). Further, the driver's arousal level is determined from among a plurality of arousal levels by referring to the arousal level correspondence table 16.

  The internal configuration of the arousal level determination unit 15 has been described above.

  The arousal level correspondence table 16 stores information in which a correspondence relationship between a plurality of arousal level levels, an α wave power spectrum, and a lambda response amplitude value is defined.

[Operation of State Determination Device 10]
Next, the operation of the state determination device 10 configured as described above will be described.

  The inventors focused on the fact that the driver's arousal level can be accurately determined based on the α-wave power spectrum value and the lambda response amplitude value.

  First, the arousal level determination method focused on based on new knowledge obtained by experiments by the present inventors will be described.

  The present inventors reproduced the situation in which the driver's arousal level and the driver's concentration level at the same time change during driving on the experimental environment, and when the driver's arousal level or concentration level decreased, the driver An experiment was conducted to measure the features appearing in the EEG.

  FIG. 6 shows combinations (experimental conditions) of states of arousal level and concentration level when extracting the brain wave signal of the driver in the experiment. That is, in the experiment, as shown in FIG. 6, (1) conditions during driving concentration (concentration: high) and high arousal level, (2) distraction (concentration: low), and arousal level In the condition of high (3) driving concentration (concentration: high) and low alertness, and (4) distraction (concentration: low) and low alertness The driver's brain wave signal is extracted.

FIG. 7 shows the result of plotting the α-wave power spectrum value and the lambda response amplitude value calculated from the brain wave signal of the driver extracted according to the conditions shown in FIG. 6 for each combination of the arousal level state and the concentration level state. In FIG. 7, the horizontal axis represents the α-wave power spectrum value [μV ² ], and the vertical axis represents the lambda reaction amplitude value [μV]. In FIG. 7, “■” indicates the result when driving is concentrated and the condition of high arousal (the above condition (1)), and “◆” indicates the condition when distracted and the level of arousal ( The result under the condition (2)) is shown, “△” shows the result when the driver is concentrated and the condition is low (the condition (3) above), “○” shows the distraction, and The result under the condition where the arousal level is low (the above condition (4)) is shown.

  As shown in FIG. 7, in the driving concentration state (concentration for driving: high), the α wave power spectrum value increases when the driver's arousal level changes from a high state to a low state (■ → Δ). I understand that.

  On the other hand, as shown in FIG. 7, in the distraction state (concentration for driving: low), the lambda response amplitude value increases when the driver's arousal level changes from a high state to a low state (◆ → ○). I understand that

  As described above, in the driving concentration state, the α-wave power spectrum value increases as the driver's arousal level decreases as in the conventional knowledge. On the other hand, in the distraction state, a new finding has been obtained that the lambda response amplitude value increases as the driver's arousal level decreases. This is thought to be because the driver's behavior to consciously increase the concentration on driving against the decrease in the arousal level is promoted.

  For this reason, when the driver's arousal level is determined by using only the α wave power spectrum value as in the prior art, the driver's arousal level is determined when the driver is distracted (when the concentration level on driving is low). It may happen that it cannot be accurately determined. Specifically, “♦” shown in FIG. 7 is a result of distraction and a high arousal level. However, since the conventional technique focuses only on the α-wave power spectrum value, the driver's arousal level decreases in the state of “♦” shown in FIG. 7 (when the α-wave power spectrum value is relatively large). May be determined. Therefore, in the conventional technique (when only the α wave power spectrum value is used), when the degree of concentration on driving is low, the accuracy of the driver's arousal level determination is deteriorated.

  Thus, the driver's arousal level and the degree of concentration on driving are closely related, and the mutual state is affected. That is, in the determination of the driver's arousal level, the α wave power spectrum may fluctuate not only when the driver's arousal level but also the concentration level on driving changes. There is a possibility that the arousal level cannot be accurately determined only by the wave power spectrum value.

  Therefore, in the present embodiment, the state determination device 10 determines the driver's arousal level using the α wave power spectrum value, the lambda reaction amplitude value, and a predetermined reference value for each value. Specifically, the state determination apparatus 10 has a relatively large lambda response amplitude value even when the α-wave power spectrum value is relatively large (for example, greater than or equal to a threshold value) based on the new knowledge obtained from the experimental results shown in FIG. When it is small (for example, less than the threshold), it is determined that the driver's arousal level is high (that is, the driver is awake). Thereby, the state determination apparatus 10 can determine the fall of the arousal level with high accuracy regardless of the concentration state (that is, the degree of concentration).

  Next, the operation of the state determination device 10 will be described. FIG. 8 is a flowchart for explaining the operation of the state determination device 10. In addition, here, the state determination apparatus 10 shall perform the arousal level determination process with the predetermined determination interval TS.

  In step S01, the electroencephalogram signal acquisition unit 11 acquires electroencephalogram signals from before the driver's predetermined time TW to the present. The predetermined time TW is a section of an electroencephalogram signal used for determination of the degree of arousal (for example, see FIG. 4), and an arbitrary value such as 30 seconds, 90 seconds, 120 seconds is set. The acquired electroencephalogram signal is recorded in a temporary storage device or the like as time series information of the electroencephalogram signal at the time of TW.

  In step S02, the eyeball movement amount acquisition unit 12 acquires the eyeball movement amount from before the driver's predetermined time TW to the present time. The acquired eye movement amount is, for example, angle information in the horizontal direction of the driver's line of sight. The acquired eye movement amount is recorded in a temporary storage device or the like as time-series information of the angle information in the horizontal direction of the line of sight at the time of TW.

  In step S03, the eyeball movement amount acquisition unit 12 detects a saccade (for example, saccade 91 shown in FIG. 4) using the eyeball movement amount in the predetermined time acquired in step S02. Generally, the time required for the saccade is usually 20 to 70 milliseconds, and the speed of the saccade is 300 to 500 degrees / second in terms of viewing angle. Therefore, the eyeball movement amount acquisition unit 12 has the same eyeball movement direction for a predetermined time (for example, 20 to 70 milliseconds) continuously, and the average angular velocity of the predetermined time is 300 degrees / second or more. Motion can be detected as a saccade.

  In step S04, it is determined whether or not there is a saccade within a predetermined time. When the saccade is not detected in step S03 within the predetermined time TW (step S04: NO), the state determination device 10 ends the process without performing the wakefulness determination process, and after the determination interval TS has elapsed, the process of step S01 is performed again. To start.

  On the other hand, when a saccade is detected in step S03 within the predetermined time TW (step S04: YES), the eyeball movement amount acquisition unit 12 extracts (specifies) the detected end time of each saccade, that is, the eyeball retention start time. .

  In step S05, the α wave feature quantity calculation unit 13 performs Fourier transform on the electroencephalogram signal at the predetermined time TW acquired in step S01, and the power spectrum value (α wave) of the α wave (frequency component of 8 to 13 Hz). Power spectrum value).

  In step S06, the eyeball retention-related potential feature amount calculation unit 14 extracts the electroencephalogram signal from the predetermined time TW before the current time acquired in step S01 to the present time, the eye movement amount acquired in step S02, and the step S04. Based on the end point of the saccade, a lambda response amplitude value is calculated (for example, see FIG. 4).

In step S07, the first arousal level determination unit 101 of the arousal level determination unit 15 compares the α wave power spectrum value calculated in step S05 with a predetermined threshold Th _α . If “α wave power spectrum value ≧ Th _α ” (step S07: YES), the process proceeds to step S08. If not “α wave power spectrum value ≧ Th _α ” (step S07: NO), the process proceeds to step S10. It is assumed that Th _α is set to an optimal value in advance through learning using experimental data. For example, Th _α = 0.012 [μV ² ] may be set from the experimental result of FIG.

In step S08, the second arousal level determination unit 102 of the arousal level determination unit 15 compares the lambda response amplitude value calculated in step S06 with a predetermined threshold Th _λ . If “lambda response amplitude value ≧ Th _λ ” (step S08: YES), the process proceeds to step S09. If it is not “lambda response amplitude value ≧ Th _λ ” (step S08: NO), the process proceeds to step S10. It is assumed that Th _λ is set to an optimal value in advance. For example, Th _λ = 1.4 [μV] may be set based on the experimental result of FIG.

In step S09, the arousal level determination unit 15 determines that the driver's arousal level is low (awakening level reduction state). That is, the awakening degree determination unit 15 determines that “α wave power spectrum ≧ Th _α ” in step S07 and “Lambda reaction amplitude value ≧ Th _λ ” in step S08, and the driver's awakening. It is determined that the degree is low.

  In step S10, the arousal level determination unit 15 determines that the driver has a high level of arousal (awake state). That is, the awakening level determination unit 15 determines that the driver's awakening is determined when “α wave power spectrum <Thα” is determined in step S07 or when “lambda response amplitude value <Thλ” is determined in step S08. It is determined that the degree is high.

  In step S11, when it is determined in step S09 that the arousal level is low, the arousal level determination unit 15 further refers to the α wave power spectrum value, the lambda reaction amplitude value, and the arousal level correspondence table 16. Accordingly, the level of arousal level (arousal level) defined in a plurality of stages is determined.

For example, FIG. 9 is a diagram illustrating a correspondence relationship held in the arousal level correspondence table 16. Specifically, the wakefulness correspondence table 16 shown in FIG. 9, and wave power spectrum value alpha of less than the threshold T _alpha least a threshold T _alpha, and lambda response amplitude values less than the threshold T _lambda above a threshold T _lambda Correspondingly, the arousal level is defined in several stages. In FIG. 9, three levels of arousal level are set: “slightly low”, “low”, and “very low”. The arousal level determination unit 15 compares the α-wave power spectrum value calculated in step S05 and the lambda response amplitude value calculated in step S06 with the threshold values T _α and T _λ for the respective values, and the driver Determine the arousal level. FIG. 9 shows an example in which one threshold value is set for each of the α wave power spectrum value and the lambda response amplitude value. However, a plurality of threshold values are provided for each value to further subdivide the threshold values. You may make it determine the level of the awakening level which became.

As described above, the state determination device 10 determines the driver's arousal level using the α wave power spectrum value and the lambda reaction amplitude value. Specifically, the state determination device 10 determines that the state in which the α wave power spectrum is less than the threshold Th _α (for example, the state of “■” shown in FIG. 7) is high as in the conventional case. To do. Further, the state determination device 10 can also detect a state where the α wave power spectrum is equal to or greater than the threshold Th _α and the lambda response amplitude value is less than the threshold Th _λ (for example, the state of “♦” shown in FIG. 7). It is determined that the degree of awakening is high. That is, the state determination device 10 is in a state where the α-wave power spectrum is equal to or greater than the threshold Th _α and the lambda response amplitude value is equal to or greater than the threshold Th _λ (for example, “Δ” and “◯” illustrated in FIG. 7). Only, it is determined that the driver's arousal level is low.

  By doing so, the state determination device 10 can accurately determine the driver's arousal level without being affected by the change in the degree of concentration of the driver with respect to driving. That is, in the determination of the driver's arousal level, the state determination device 10 determines the driver's action (that is, the change in the concentration level) to increase the driver's concentration against the decrease in the driver's arousal level. The arousal level can be determined in consideration. This makes it possible to perform optimal safe driving support such as information presentation to the driver, warning, or vehicle control.

<Embodiment 2>
FIG. 10 is a block diagram illustrating a configuration of the state determination device 20 according to the present embodiment. In FIG. 10, parts having the same configurations as those of the first embodiment (FIG. 2) are denoted by the same reference numerals and description thereof is omitted. The state determination device 20 illustrated in FIG. 10 is different from the state determination device 10 illustrated in FIG. 2 in that the wakefulness correspondence table is deleted and the operation of the wakefulness determination unit 21 is different.

  In FIG. 10, the arousal level determination unit 21 is obtained from the α wave power spectrum value calculated by the α wave feature amount calculation unit 13 and the lambda response amplitude value calculated by the eyeball retention related potential feature amount calculation unit 14. The driver's arousal level is determined using the arousal level evaluation value.

  FIG. 11 is a block diagram showing an internal configuration of the arousal level determination unit 21 according to the present embodiment. As shown in FIG. 11, the wakefulness determination unit 21 includes a wakefulness evaluation value calculation unit 201 and an evaluation value determination unit 202.

  The arousal level evaluation value calculation unit 201 calculates the arousal level evaluation value from the α-wave power spectrum value and the lambda response amplitude value. Specifically, the arousal level evaluation value calculation unit 201 uses the predetermined boundary line (discrimination line) on the two-dimensional plane composed of the α wave power spectrum value and the lambda response amplitude value to calculate the arousal level evaluation value. calculate. The arousal level evaluation value is a value used to determine the driver's arousal level. For example, the wakefulness evaluation value is calculated according to a wakefulness evaluation value calculation formula set in advance by experiments or the like.

  The evaluation value determination unit 202 compares the wakefulness evaluation value calculated by the wakefulness evaluation value calculation unit 201 with a reference value (threshold value) set in advance by an experiment or the like, and determines the driver's wakefulness.

  Next, the operation of the state determination device 20 according to the present embodiment will be described.

  FIG. 12 is a flowchart for explaining the operation of the state determination device 20. In FIG. 12, parts that are the same as those in the first embodiment (FIG. 8) are assigned the same reference numerals, and descriptions thereof are omitted.

In step S21, the arousal level evaluation value calculation unit 201 of the arousal level determination unit 21 calculates the α wave power spectrum value (αt) at each time point calculated in step S05 and the lambda reaction at each time point calculated in step S06. For the amplitude value (λt), the arousal level evaluation value (D) is calculated according to the arousal level evaluation value calculation formula shown in Formula (1).

  The arousal level evaluation value calculation formula shown in Expression (1) is such that the α-wave power spectrum value increases on a two-dimensional plane of α-λ composed of the α-wave power spectrum value (α) and the lambda response amplitude value (λ). In this case, the lambda response amplitude value decreases. That is, the expression (1) is characterized in that the positive and negative signs of the coefficient a and the coefficient b are the same. The arousal level evaluation value calculation unit 201 calculates the arousal level evaluation value D by substituting the α wave power spectrum value (αt) and the lambda response amplitude value (λt) acquired at each time point into the equation (1).

A method of calculating the arousal level evaluation value D will be described with reference to FIG. FIG. 13 shows the result of performing linear discriminant analysis in order to discriminate between a state with a high arousal level and a state with a low arousal level using the experimental results shown in FIG. 7 as learning data. In FIG. 13, as learning data in a state where the arousal level is high, a state where the arousal level is low using a result when driving is concentrated and the arousal level is high, and a result when distraction is high and the arousal level is high. As the learning data, it is assumed that linear discriminant analysis is performed using a result when driving is concentrated and the arousal level is low and a result when distracted and the arousal level is low. As a result, for example, a discrimination straight line (the predetermined boundary line) shown in FIG. 13 and the following equation (2) is obtained.

And based on the discrimination | determination straight line shown to Formula (2), the arousal level evaluation value calculation formula shown to following Formula (3) is derived | led-out.

In step S22, the evaluation value determination unit 202 of the awareness determination unit 21, and the arousal level evaluation value D calculated in step S21, by comparing the reference value Th _d that is set in advance by an experiment or the like, the driver Determine the degree of awakening. For example, when the awakening degree evaluation value calculation formula is represented by the formula (3), the reference value Th _d is zero. Therefore, if D ≧ 0 (step S22: YES), evaluation value determination unit 202 determines that the driver's arousal level is low in step S09. On the other hand, if D <0 (step S22: NO), evaluation value determination unit 202 determines that the driver's arousal level is high in step ST09. Note that (αt, λt) in the state of D ≧ 0 corresponds to a region above the discrimination line shown in FIG. 13 (including the discrimination line), and (αt, λt) in the state of D <0 is This corresponds to a region below the discrimination line shown in FIG. 13 (not including the discrimination line).

  Further, the evaluation value determination unit 202 learns the relationship between the wakefulness evaluation value D and the driver's wakefulness level in advance by experiment or the like, for example, based on the table showing the correspondence shown in FIG. The arousal level may be determined. In FIG. 14, D <b> 1 and D <b> 2 indicate preset threshold values for the arousal level evaluation value D.

  Thus, state determination apparatus 20 determines the driver's arousal level using the α-wave power spectrum value and the lambda reaction amplitude value, as in the first embodiment (state determination apparatus 10). However, in Embodiment 1 (state determination device 10), the driver's arousal level is determined from a plurality of arousal levels using the arousal level correspondence table, whereas in this embodiment, the state determination is performed. The device 20 determines the driver's arousal level based on the arousal level evaluation value D in the α-λ two-dimensional plane.

  Even in this case, as in the first embodiment, the state determination device 20 can determine the arousal level in consideration of the change in the degree of concentration with respect to the driver's driving in the determination of the driver's arousal level. Therefore, the state determination device 20 can accurately determine the driver's arousal level without being affected by the change in the concentration level of the driver with respect to driving. Thereby, for example, according to the driver's arousal level, it is possible to perform optimum safe driving support such as information presentation to the driver, warning, or vehicle control.

  The discriminant straight line (wakefulness evaluation value calculation formula) in the present embodiment is not limited to a straight line, and is represented by a curve such as a hyperbola or a quadratic curve. May be.

<Embodiment 3>
FIG. 15 is a block diagram showing a configuration of state determination device 30 according to the present embodiment. In FIG. 15, parts having the same configurations as those of the first embodiment (FIG. 2) are denoted by the same reference numerals and description thereof is omitted.

  In FIG. 15, the behavior detection unit 31 detects a behavior (attention behavior) in which the driver consciously increases attention to driving. In general, when the driver's arousal level decreases due to fatigue and monotonous work during driving or monitoring, the driver frequently moves his eyes or blinks frequently to pay attention to the work. You may take action (attention action) to increase consciously. Therefore, the behavior detection unit 31 calculates the amount of eye movement of the driver using the above-described “EOG method” or the like by using a head-mounted measuring device, for example, and pays attention based on frequent eye movement or blinking. An action may be detected. The behavior detection unit 31 may detect the attention behavior by photographing the driver with a camera installed near the steering column and calculating the amount of eye movement using the above-described “corneal reflection method” or the like. .

The reference value correcting unit 32 corrects the determination threshold (reference value) Th _λ of the lambda response amplitude value when the behavior detecting unit 31 detects the attention behavior. When the attention behavior is detected, the behavior to increase the concentration on driving is promoted by the driver, and therefore the lambda response amplitude value may increase. Therefore, the reference value correction unit 32 corrects the determination threshold Th _λ to a higher value than in the normal case (when no attention behavior is detected) according to the detected attention behavior. The reference value correction unit 32 outputs the corrected determination threshold Th _λ to the arousal level determination unit 15a. Thereby, in the state determination apparatus 30, the determination precision of a driver | operator's arousal level can be improved.

As in the first embodiment, the arousal level determination unit 15a includes the α wave power spectrum value calculated by the α wave feature amount calculation unit 13, and the lambda response amplitude value calculated by the eyeball retention related potential feature amount calculation unit 14. Is used to determine the driver's arousal level. However, in the arousal level determination unit 15a, only the point of performing the arousal level determination process according to the determination threshold Th _λ (determination threshold after correction) input from the reference value correction unit 32 in the arousal level determination process for the lambda response amplitude value. Is different from the first embodiment.

FIG. 16 is a block diagram showing an internal configuration of the arousal level determination unit 15a according to the present embodiment. As shown in FIG. 16, the wakefulness determination unit 15a includes a first wakefulness determination unit 101 and a second wakefulness determination unit 102a, and only the operation of the second wakefulness determination unit 102a is performed in the first embodiment. Different from FIG. Specifically, the second arousal level determination unit 102a is calculated by the corrected determination threshold value (for example, threshold value Th _λ ) input from the reference value correction unit 32 and the eyeball retention related potential feature amount calculation unit 14. Compare with lambda response amplitude value.

  Next, the operation of the state determination device 30 according to the present embodiment will be described.

  FIG. 17 is a flowchart for explaining the operation of the state determination device 30. In FIG. 17, parts that are the same as in the first embodiment (FIG. 8) are assigned the same reference numerals, and descriptions thereof are omitted.

  In step S31, the behavior detection unit 31 detects the driver's attention behavior in the predetermined section TW using the electroencephalogram signal and eye movement amount acquired in steps S01 and S02, and the reference value correction unit 32 The number and type of attention actions detected by the detection unit 31 are recorded. Examples of the type of attention behavior include frequent line-of-sight movement and frequent blinking as described above.

In step S32, when an attention behavior is detected in step S31, the reference value correction unit 32 corrects the determination threshold Th _λ of the lambda response amplitude value to a value higher than a normal value. At this time, the reference value correction unit 32 may change the correction amount of the determination threshold Th _λ in accordance with the type and number of attention actions recorded in step S11.

In step S33, the second wakefulness determination unit 102a of the wakefulness determination unit 15a detects the attention action in step S31, and the lambda reaction amplitude value calculated in step S06 and the determination threshold Th corrected in S32. Compare with _λ .

  By doing so, the state determination device 30 can perform the arousal level determination process using an appropriate determination threshold value (reference value) corresponding to the lambda response amplitude value that varies according to the driver's attention behavior. Thereby, in this Embodiment, a driver | operator's alertness can be determined with sufficient precision in consideration of a driver | operator's attention action. Therefore, for example, it is possible to provide optimum safe driving support such as information presentation to the driver, warning, or vehicle control.

<Embodiment 4>
FIG. 18 is a block diagram showing a main configuration of state determination apparatus 40 according to the present embodiment. A state determination device 40 shown in FIG. 18 determines the state of the driver. In the state determination device 40, the α-wave feature value calculator 13 calculates an α-wave feature value (for example, an α-wave power spectrum value) from the electroencephalogram signal, and the eyeball retention-related potential feature value calculator 14 Based on the eye movement amount and the electroencephalogram signal, the feature amount (for example, lambda response amplitude value) of the eyeball retention-related potential is calculated, and the concentration determination unit 41 calculates the α wave calculated by the α wave feature amount calculation unit 13. Based on the feature amount and the feature amount of the eyeball-related potential calculated by the eyeball-related potential feature amount calculation unit 14, the degree of concentration of the driver is determined.

FIG. 19 is a block diagram illustrating a configuration of the state determination device according to the present embodiment. For example,
In a safe driving support system (not shown) that provides driving support such as warning or braking to the driver based on the driver's concentration on driving, the state determination device 40 (FIG. 19) according to the present embodiment includes: As the driver's state, the driver's degree of concentration with respect to driving is determined. That is, the state determination apparatus 40 according to the present embodiment is a concentration degree determination apparatus that determines the state of the degree of concentration with respect to the driving of the driver.

  In FIG. 19, parts having the same configuration as in the first embodiment (FIG. 2) are denoted by the same reference numerals and description thereof is omitted.

  In FIG. 19, the concentration degree determination unit 41 is based on the α wave power spectrum value calculated by the α wave feature amount calculation unit 13 and the lambda reaction amplitude value calculated by the eyeball retention related potential feature amount calculation unit 14. The degree of concentration of the driver with respect to driving is determined. Further, when the concentration level determination unit 41 determines that the concentration level of the driver with respect to driving is low, the concentration level determination unit 41 further refers to the concentration level correspondence table 42 and selects from a plurality of levels (concentration level) indicating the concentration level. Determine the concentration level.

  FIG. 20 is a block diagram showing an internal configuration of the concentration degree determination unit 41 according to the present embodiment. As shown in FIG. 20, the concentration level determination unit 41 includes a first concentration level determination unit 401 and a second concentration level determination unit 402.

The first concentration determination unit 401 determines whether or not the lambda response amplitude value (eyeball retention-related potential feature amount) calculated by the eyeball retention-related potential feature amount calculation unit 14 is equal to or less than a predetermined threshold (for example, threshold Th _λ ). Determine whether. That is, the first concentration level determination unit 401 performs the concentration level determination process by comparing the lambda reaction amplitude value with a predetermined threshold value. The first concentration degree determination unit 401 outputs a determination result for the lambda reaction amplitude value to the second concentration degree determination unit 402. Specifically, the first concentration determination unit 401 determines that the driver is in a concentrated state (a state of high concentration) when the lambda response amplitude value is greater than a predetermined threshold.

The second concentration determination unit 402 determines whether or not the α wave power spectrum value (α wave feature value) calculated by the α wave feature value calculation unit 13 is equal to or greater than a predetermined threshold (for example, threshold Th _α ). To do. That is, the second concentration determination unit 402 performs the concentration determination process by comparing the α wave power spectrum value with a predetermined threshold. The concentration level determination process in the second concentration level determination unit 402 is performed only when, for example, the determination result input from the first concentration level determination unit 401 indicates that the lambda reaction amplitude value is equal to or less than a predetermined threshold value. .

  Specifically, the second concentration determination unit 402 determines that the driver is in a state of distraction (a state of low concentration) with respect to driving when the α-wave power spectrum value is equal to or greater than a predetermined threshold. . On the other hand, when the α-wave power spectrum value is less than a predetermined threshold, second concentration level determination unit 402 determines that the driver is in a concentrated state (a high concentration level). Further, when the second concentration degree determination unit 402 determines that the concentration degree of the driver with respect to driving is low as a result of the concentration degree determination with respect to the α wave power spectrum value (the α wave power spectrum value is equal to or greater than a predetermined threshold value). Further, with reference to the concentration level correspondence table 42, the driver's concentration level is determined from among a plurality of concentration levels.

  The internal configuration of the concentration level determination unit 41 has been described above.

  The degree-of-concentration correspondence table 42 stores information in which the correspondence relation between a plurality of degree-of-concentration levels, an α wave power spectrum, and a lambda reaction amplitude value is defined.

[Operation of State Determination Device 40]
Next, the operation of the state determination device 40 configured as described above will be described.

  As in the first embodiment, the present inventors paid attention to the fact that the driver's degree of concentration with respect to driving can be accurately determined based on the α-wave power spectrum value and the lambda response amplitude value.

  First, the concentration determination method focused on based on new knowledge obtained by experiments by the present inventors will be described.

FIG. 21 shows the α wave power spectrum value and the lambda response amplitude value calculated from the brain wave signal of the driver extracted according to the conditions shown in FIG. 6 (combination of state of alertness and concentration), as in the first embodiment. The results plotted for each combination of arousal level state and concentration level state are shown. In FIG. 21, the horizontal axis indicates the α-wave power spectrum value [μV ² ], and the vertical axis indicates the lambda reaction amplitude value [μV]. In FIG. 21, “■” indicates the result when the driver is concentrated (concentration: high) and the degree of arousal is high, and “◇” indicates the distraction (concentration: low) and arousal. Shows results under high-level conditions, “▲” indicates results when driving is concentrated (concentration level: high) and low-wakefulness conditions, and “○” indicates when distraction is observed (level of concentration: low) And the result in the conditions with a low arousal level is shown. In other words, FIG. 21 is a notation focusing on the driver's concentration on driving with respect to the same result as in FIG. 7 (embodiment 1: noting attention on arousal level).

  As shown in FIG. 21, in a state where the degree of arousal is high, when the driver's concentration on driving changes from a high state to a low state (■ → ◇), the lambda reaction amplitude value decreases, and the α wave power spectrum It can be seen that the value increases.

  On the other hand, as shown in FIG. 21, in a state where the degree of arousal is low, when the driver's degree of concentration with respect to driving changes from a high state to a low state (▲ → ○), the lambda reaction amplitude value decreases, and α waves It can be seen that the power spectrum value increases.

  As described above, regardless of the state of arousal, the lambda response amplitude value decreases as the driver's concentration with respect to driving decreases as in the conventional knowledge. In addition, a new finding was obtained that, regardless of the state of arousal, the α-wave power spectrum value increases as the driver's concentration on driving decreases. This is presumably because the information processing load on the driver decreases with a decrease in the degree of concentration (a decrease in attention to the driver's visual information).

  For this reason, when determining the driver's degree of concentration on driving using only the lambda response amplitude value as in the prior art, for example, in a state where the degree of arousal is high, the degree of concentration cannot be accurately determined. Will happen. Specifically, “■” shown in FIG. 21 is a result when the degree of concentration is high and the degree of arousal is high. However, since the conventional technology focuses only on the lambda response amplitude value, it is determined that the state of “■” shown in FIG. 7 (when the lambda response amplitude value is relatively small) is reduced in the degree of concentration on driving. There is a possibility that. Therefore, in the conventional technique (when only the lambda response amplitude value is used), when the driver's arousal level is high, the accuracy of the concentration level determination is deteriorated.

  Thus, the driver's arousal level and the degree of concentration on driving are closely related, and are affected by each other's state. That is, in determining the driver's degree of concentration with respect to driving, not only the lambda response amplitude value but also the α-wave power spectrum value may change. The degree may not be determined accurately.

  Therefore, in the present embodiment, the state determination device 40 determines the driver's degree of concentration with respect to driving by using the α wave power spectrum value, the lambda reaction amplitude value, and a predetermined reference value for each value. Specifically, the state determination device 40 has a relatively low α-wave power spectrum value even when the lambda response amplitude value is relatively small (for example, below a threshold value) based on the new knowledge obtained from the experimental results shown in FIG. When it is small (for example, less than the threshold), it is determined that the driver's degree of concentration on driving is high. Thereby, the state determination apparatus 40 can determine accurately the fall of the concentration degree with respect to a driving | operation.

  Next, the operation of the state determination device 40 will be described. FIG. 22 is a flowchart for explaining the operation of the state determination device 40. In FIG. 22, parts that are the same as those in the first embodiment (FIG. 8) are assigned the same reference numerals, and descriptions thereof are omitted.

In step S41, the first concentration determination unit 401 of the concentration determination unit 41 compares the lambda reaction amplitude value calculated in step S06 with a predetermined threshold Th _λ . If “lambda response amplitude value ≦ Thλ” (step S41: YES), the process proceeds to step S42. If not “lambda response amplitude value ≦ Thλ” (step S41: NO), the process proceeds to step S44. It is assumed that Th _λ is set to an optimal value in advance. For example, Th _λ = 2.0 [μV] may be set based on the experimental results of FIG.

In step S42, the second concentration degree determination unit 402 of the concentration degree determination unit 41 compares the α wave power spectrum value calculated in step S05 with a predetermined threshold Th _α . If “α wave power spectrum value ≧ Th _α ” (step S42: YES), the process proceeds to step S43. When it is not “α wave power spectrum value ≧ Th _α ” (step S42: NO), the process proceeds to step S44. Note that Th _α is set to an optimal value in advance. For example, Th _α = 0.012 [μV ² ] may be set from the experimental result of FIG.

In step S43, the concentration determination unit 41 determines that the concentration of the driver with respect to driving is low (distracted state). That is, the concentration degree determination unit 41 determines that “lambda response amplitude value ≦ Th _λ ” in step S41 and determines that “α-wave power spectrum ≧ Th _α ” in step S42. It is determined that the degree of concentration is low (distracted state).

In step S44, the concentration degree determination unit 41 determines that the concentration degree of the driver with respect to driving is high (concentrated state). In other words, the concentration determination unit 41 determines that the driver is “lambda response amplitude value> Th _λ ” in step S41 or “α wave power spectrum <Th _α ” in step S42. It is determined that the degree of concentration with respect to driving is high.

  In step S45, when it is determined in step S43 that the concentration level is low, the concentration level determination unit 41 further refers to the α wave power spectrum value, the lambda reaction amplitude value, and the concentration level correspondence table. To determine the level of concentration (concentration level) defined in a plurality of stages.

For example, FIG. 23 is a diagram illustrating a correspondence relationship held in the concentration degree correspondence table 42. Specifically, the degree of concentration correspondence table 42 shown in FIG. 23, and wave power spectrum value alpha of less than the threshold T _alpha least a threshold T _alpha, and lambda response amplitude values less than the threshold T _lambda above a threshold T _lambda Correspondingly, the concentration level is defined in several stages. In FIG. 23, three levels of concentration are set, “slightly low”, “low”, and “very low”. The degree-of-concentration determination unit 41 compares the α-wave power spectrum value calculated in step S05 and the lambda reaction amplitude value calculated in step S06 with the threshold values T _α and T _λ for the respective values to determine the driver. Determine the concentration level for driving. FIG. 23 shows an example in which one threshold value is set for each of the α wave power spectrum value and the lambda response amplitude value. However, a plurality of threshold values are provided for each value to further subdivide the threshold values. It is also possible to determine the level of the concentrated concentration.

Thus, the state determination apparatus 40 determines the driver's degree of concentration with respect to driving using the α wave power spectrum value and the lambda reaction amplitude value. Specifically, the state determination device 40 determines that the state where the lambda response amplitude value is larger than the threshold Th _λ (for example, the state of “▲” shown in FIG. 21) is high in concentration as in the conventional case. Furthermore, the state determination device 40 also concentrates the state where the lambda response amplitude value is equal to or less than the threshold value Th _λ and the α wave power spectrum value is less than the threshold value Th _α (for example, the state of “■” shown in FIG. 21). It is determined that the degree is high. That is, the state determination device 40 is in a state where the lambda response amplitude value is equal to or less than the threshold value Th _λ and the α wave power spectrum value is equal to or greater than the threshold value Th _α (for example, the states of “◇” and “◯” shown in FIG. 7). Only) is determined to be less concentrated.

  By doing so, the state determination device 40 can accurately determine the degree of concentration of the driver with respect to driving without being affected by the change in the driver's arousal level. That is, the state determination device 40 can determine the concentration level in consideration of the decrease in the information processing load of the driver accompanying the decrease in the concentration level in the determination of the concentration level with respect to the driving of the driver. This makes it possible to perform optimal safe driving support such as information presentation to the driver, warning, or vehicle control.

<Embodiment 5>
FIG. 24 is a block diagram illustrating a configuration of the state determination device 50 according to the present embodiment. In FIG. 24, parts having the same configuration as that of the fourth embodiment (FIG. 19) are denoted by the same reference numerals and description thereof is omitted. The state determination device 50 illustrated in FIG. 24 is different from the state determination device 40 illustrated in FIG. 19 in that the concentration level correspondence table is deleted and the operation of the concentration level determination unit 51 is different.

  In FIG. 24, the concentration degree determination unit 51 is obtained from the α wave power spectrum value calculated by the α wave feature amount calculation unit 13 and the lambda response amplitude value calculated by the eyeball retention related potential feature amount calculation unit 14. A concentration degree evaluation value is used to determine a driver's degree of concentration with respect to driving.

  FIG. 25 is a block diagram showing an internal configuration of the concentration degree determination unit 51 according to the present embodiment. As shown in FIG. 25, the concentration degree determination unit 51 includes a concentration degree evaluation value calculation unit 501 and an evaluation value determination unit 502.

  The concentration degree evaluation value calculation unit 501 calculates a concentration degree evaluation value from the α wave power spectrum value and the lambda reaction amplitude value. Specifically, the concentration level evaluation value calculation unit 501 uses a predetermined boundary line (discriminant line) on the two-dimensional plane composed of the α-wave power spectrum value and the lambda reaction amplitude value to calculate the concentration level evaluation value. calculate. The concentration degree evaluation value is a value used to determine the degree of concentration of the driver with respect to driving. For example, the concentration level evaluation value is calculated according to a concentration level evaluation value calculation formula set in advance by experiment or the like.

  The evaluation value determination unit 502 compares the concentration degree evaluation value calculated by the concentration degree evaluation value calculation unit 501 with a reference value (threshold value) set in advance by an experiment or the like, and determines the concentration degree for driving.

  Next, the operation of the state determination device 50 according to the present embodiment will be described.

  FIG. 26 is a flowchart for explaining the operation of the state determination device 50. In FIG. 26, parts that are the same as those in the fourth embodiment (FIG. 22) are assigned the same reference numerals, and descriptions thereof are omitted.

In step S51, the concentration evaluation value calculation unit 501 of the concentration determination unit 51 calculates the α wave power spectrum value (αt) at each time point calculated in step S05 and the lambda reaction at each time point calculated in step S06. For the amplitude value (λt), the concentration level evaluation value (A) is calculated according to the concentration level evaluation value calculation formula shown in Formula (4).

  In the concentration degree evaluation value calculation formula shown in Expression (4), the α wave power spectrum value increases in the two-dimensional plane of α-λ composed of the α wave power spectrum value (α) and the lambda reaction amplitude value (λ). In some cases, the lambda response amplitude value also increases. That is, the expression (4) is characterized in that the positive and negative signs of the coefficient a and the coefficient b are different. The concentration level evaluation value calculation unit 501 calculates the arousal level evaluation value A by substituting the α wave power spectrum value (αt) and the lambda response amplitude value (λt) acquired at each time point into the equation (4).

A method of calculating the concentration evaluation value A will be described with reference to FIG. FIG. 27 shows the result of performing linear discriminant analysis in order to discriminate between a state of high concentration and a state of low concentration using the experimental results shown in FIG. 21 as learning data. In FIG. 27, as the learning data in a high concentration level, the result when driving is concentrated (concentration level: high) and the arousal level is high, and when driving is concentrated (concentration level: high) and the arousal level is low. As the learning data in a state of low concentration using the result of the case, the result of distraction (concentration: low) and high arousal, and the distraction (concentration: low) and arousal Suppose that the linear discriminant analysis is performed using the result of the low case. As a result, for example, a discrimination straight line (the predetermined boundary line) shown in FIG. 27 and the following equation (5) is obtained.

Based on the discriminant straight line shown in Expression (5), a concentration degree evaluation value calculation expression shown in the following Expression (6) is derived.

In step S52, the evaluation value determination unit 502 of the concentration degree determination unit 51 compares the concentration degree evaluation value A calculated in step S51 with a reference value Th _a set in advance through experiments or the like, thereby driving. The degree of concentration on the driver's driving is determined. For example, when the concentration evaluation value calculation formula is expressed by Expression (6), the reference value Th _a is 0. Therefore, if A ≧ 0 (step 52: YES), evaluation value determination unit 502 determines that the degree of concentration is high in step S44. On the other hand, if A <0 (step S52: NO), evaluation value determination unit 502 determines that the degree of concentration is low in step S43. Note that (αt, λt) in the state of A ≧ 0 corresponds to a region above the discrimination line shown in FIG. 27 (including the discrimination line), and (αt, λt) in the state of A <0 is This corresponds to an area below the discrimination line shown in FIG. 27 (not including the discrimination line).

  Further, the evaluation value determination unit 502 learns the relationship between the concentration degree evaluation value A and the concentration level of the driver in advance by experiment or the like, for example, based on the table showing the correspondence shown in FIG. The concentration level may be determined. In FIG. 28, A1 and A2 indicate preset threshold values for the concentration degree evaluation value A.

  Thus, state determination apparatus 50 determines the driver's degree of concentration with respect to driving using the α-wave power spectrum value and the lambda reaction amplitude value, as in the fourth embodiment (state determination apparatus 40). However, in the fourth embodiment (state determination device 40), the concentration level of the driver is determined from a plurality of concentration levels using the concentration level correspondence table, whereas in the present embodiment, the state determination is performed. The device 50 determines the driver's concentration level based on the concentration evaluation value A on the two-dimensional plane of α-λ.

  Even in this case, as in the fourth embodiment, the state determination device 50 can determine the concentration level in consideration of the change in the driver's arousal level in the determination of the concentration level of the driver. Therefore, the state determination device 50 can accurately determine the concentration of the driver with respect to driving without being affected by the change in the driver's arousal level. Thereby, for example, according to the driver's degree of concentration with respect to driving, it is possible to perform optimal safe driving support such as information presentation to the driver, warning, or vehicle control.

  Note that the discriminant straight line (concentration evaluation value calculation formula) in the present embodiment is not limited to a straight line, and is represented by a curve such as a hyperbola or a quadratic curve. May be.

<Embodiment 6>
FIG. 29 is a block diagram showing a configuration of state determination device 60 according to the present embodiment. In FIG. 29, parts having the same configuration as that of the fourth embodiment (FIG. 19) are denoted by the same reference numerals and description thereof is omitted.

  In FIG. 29, the environment detection unit 61 detects the surrounding environment that increases the processing load on the driver's visual information. In general, it is considered that the processing load on the driver's visual information increases as the driving environment becomes more complicated, such as an intersection or a city area, than an expressway or a monotonous straight single road. Therefore, for example, the environment detection unit 61 may acquire the status of the surrounding environment such as the road type (highway / general road) and the road shape (straight line, curve, intersection) from the map information of the car navigation system. Alternatively, the environment detection unit 61 may acquire the situation of the surrounding environment by performing image analysis on a surrounding image acquired by a camera or the like that is installed in the vicinity of the rearview mirror or at the front end of the vehicle and monitors the front of the vehicle. Good.

When the environment detection unit 61 detects a complicated surrounding environment (a surrounding environment that increases the processing load on the driver's visual information), the reference value correction unit 62 sets the α power spectrum value determination threshold (reference value) Th _α . to correct. When a complicated surrounding environment is detected, the processing load on the driver's visual information increases, so it is considered that the appearance of α waves is suppressed. Therefore, the reference value correction unit 62 corrects the determination threshold Th _α to a value lower than that in a normal case (for example, a simple driving environment) according to the detected surrounding environment. The reference value correction unit 62 outputs the corrected determination threshold Th _α to the concentration level determination unit 41a. Thereby, in the state determination apparatus 60, it becomes possible to determine the suppressed α wave with high accuracy, and the determination accuracy of the degree of concentration can be improved.

Similar to the fourth embodiment, the concentration degree determination unit 41a is configured to calculate the α wave power spectrum value calculated by the α wave feature amount calculation unit 13 and the lambda response amplitude value calculated by the eyeball retention-related potential feature amount calculation unit 14. Is used to determine the driver's degree of concentration with respect to driving. However, the concentration determination unit 41a performs the concentration determination process according to the determination threshold Th _α (corrected determination threshold) input from the reference value correction unit 62 in the concentration determination process for the α-wave power spectrum value. Only differs from the fourth embodiment.

FIG. 30 is a block diagram showing an internal configuration of the concentration degree determination unit 41a according to the present embodiment. As shown in FIG. 30, the concentration determination unit 41a includes a first concentration determination unit 401 and a second concentration determination unit 402a, and only the operation of the second concentration determination unit 402a is performed in the fourth embodiment. Different from FIG. Specifically, the second concentration degree determination unit 402 a receives the corrected determination threshold value (for example, threshold Th _α ) input from the reference value correction unit 62 and the α wave calculated by the α wave feature amount calculation unit 13. Compare the power spectrum value.

  Next, the operation of the state determination device 60 according to the present embodiment will be described.

  FIG. 31 is a flowchart for explaining the operation of the state determination device 60. In FIG. 31, parts that are the same as in the fourth embodiment (FIG. 22) are assigned the same reference numerals, and descriptions thereof are omitted.

  In step S61, the environment detection unit 61 acquires a surrounding environment such as a road type and a road shape in the predetermined section TW in which each information is acquired in steps S01 and S02. For example, the environment detection unit 61 may acquire a road congestion condition.

In step S62, the reference value correction unit 62 corrects the α power spectrum value determination threshold Th _α according to the surrounding environment detected in step S61. For example, the reference value correction unit 62 may correct the α power spectrum value determination threshold Th _α based on the correspondence table related to the surrounding environment shown in FIG. In FIG. 32, three items of road type, road shape, and road condition are set as the surrounding environment. Further, in FIG. 32, for each item, an environment with a high visual information processing load required for the driver is associated with an environment with a low visual information processing load required for the driver. In the environment where the visual information processing load required for the driver is high in FIG. 32, the reference value correction unit 62 sets the determination threshold Th _α to be higher than normal (for example, in the case where the visual information processing load required for the driver is low). Correct to a lower value. The reference value correction unit 62 uses the threshold Th when there is an environment that requires a high visual information processing load even in at least one item in the correspondence table (road type, road shape, road condition) shown in FIG. _α may be set lower than usual. Alternatively, the reference location correction unit 62 reduces the determination threshold Th _α in accordance with the number of items that require a high visual information processing load in the correspondence table (road type, road shape, road situation) illustrated in FIG. The value may be changed.

In step S63, the second concentration determination unit 402a of the concentration determination unit 41a compares the α-wave power spectrum value calculated in step S05 with the determination threshold Th _α corrected in S62.

  In this way, the state determination device 60 concentrates using an appropriate determination threshold value (reference value) corresponding to the α-wave power spectrum value that varies according to the magnitude of the driver's visual information processing load with respect to the surrounding environment. Degree determination processing can be performed. Thereby, in this Embodiment, the driver | operator's concentration with respect to a driving | operation can be accurately determined in consideration of the driver | operator's visual information processing load with respect to a surrounding environment. Therefore, for example, it is possible to provide optimum safe driving support such as information presentation to the driver, warning, or vehicle control.

  The embodiments of the present invention have been described above.

  In each of the above embodiments, the case where the arousal level or the concentration level is calculated based on the calculation results of the α wave power spectrum value and the lambda response amplitude value has been described. However, when there are large individual differences between subjects in both the α-wave power spectrum value and the lambda response amplitude value, it is conceivable that the determination accuracy of the arousal level and the concentration level is lowered. Therefore, for both indicators, the difference from the reference value (threshold value) or the ratio with the reference value (threshold value) may be calculated for each subject and used for determination.

  Specifically, both indices immediately before driving or in daily life are calculated in advance as reference values, and the difference between the two calculated indices during driving and the reference value calculated in advance, or the reference calculated in advance. A normalized index (normalized α wave power spectrum value, normalized lambda response amplitude value) may be calculated by calculating a ratio with the value. For both normalized indicators, by using thresholds or discriminants obtained through learning using similarly normalized data, the influence of individual differences among subjects is reduced, and arousal level or concentration level Can be determined.

  In each of the above-described embodiments, the case where the α-wave power spectrum value is used as the α-wave feature amount has been described. However, when there is a large individual difference between subjects in the α wave power spectrum value, the α wave content may be used as the α wave feature amount. The α wave content rate may be the ratio of the power spectrum value of the α wave to the power spectrum value of the entire band, or the ratio of the power spectrum value of the α wave to a specific frequency band such as an α wave or a β wave. Also good.

  The state determination device according to the present invention is useful as a safe driving support system that is mounted on a vehicle or a train for the purpose of preventing a reduction in arousal level or concentration level.

DESCRIPTION OF SYMBOLS 10, 20, 30, 40, 50, 60 State determination apparatus 11 Electroencephalogram signal acquisition part 12 Eyeball movement amount acquisition part 13 Alpha wave feature-value calculation part 14 Eyeball retention related electric potential feature-value calculation part 15, 15a, 21 Arousal degree determination part 16 arousal level correspondence table 101 first arousal level determination unit 102, 102a second arousal level determination unit 201 arousal level evaluation value calculation unit 202 evaluation value determination unit 31 action detection unit 32, 62 reference value correction unit 41, 41a, 51 concentration Degree determination unit 42 Concentration degree correspondence table 401 First concentration degree determination unit 402, 402a Second concentration degree determination unit 501 Concentration evaluation value calculation unit 502 Evaluation value determination unit 61 Environment detection unit

Claims (13)

A state determination device that determines the state of the subject based on a brain wave signal of the subject,
An α-wave feature amount calculating means for calculating an α-wave feature amount from the electroencephalogram signal;
Based on the eye movement amount of the subject and the electroencephalogram signal, an eyeball retention-related potential feature amount calculating unit that calculates a feature amount of the eyeball retention-related potential;
Based on the feature quantity of the α wave calculated by the α-wave feature quantity calculating means and the feature quantity of the eyeball-related potential calculated by the eyeball-related potential feature quantity calculating means, A determination means for determining the degree of concentration of the subject;
A state determination device comprising: The determination means includes
A first arousal level determination means for determining the arousal level by comparing the characteristic amount of the α wave with a first reference value;
By comparing the feature amount of the eyeball-related potential with a second reference value when the first arousal level determination means determines that the feature amount of the α wave is greater than or equal to the first reference value. A second arousal level judging means for judging the arousal level,
The state determination apparatus according to claim 1. The first arousal level determination means determines that the awakening level is high when the feature quantity of the α wave is less than the first reference value,
The second arousal level determination unit determines that the arousal level is high when the feature amount of the eyeball-related potential is less than the second reference value, and the feature amount of the eyeball-related potential is the second If it is equal to or greater than the reference value, it is determined that the arousal level is low.
The state determination apparatus according to claim 2. The level of the arousal level is defined in a plurality of stages by associating the feature amount of the α wave, the feature amount of the eyeball-related potential, the first reference value, and the second reference value. A correspondence table is further provided,
The determination unit refers to the correspondence table when the second arousal level determination unit determines that the feature amount of the eyeball retention-related potential is equal to or greater than the second reference value. From the inside, the level of the arousal level of the subject is determined.
The state determination apparatus according to claim 2. Detecting means for detecting attention behavior that the subject consciously raises attention;
Correction means for correcting the second reference value when the attention behavior is detected by the detection means;
The state determination apparatus according to claim 2. The determination means includes
A first concentration degree determination means for determining the concentration degree by comparing a feature amount of the eyeball retention-related potential and a third reference value;
Comparing the feature quantity of the α wave with the fourth reference value when the first concentration degree judging means judges that the feature quantity of the eyeball-related potential is less than or equal to the third reference value; And a second concentration degree determination means for determining the concentration degree.
The state determination apparatus according to claim 1. The first concentration level determination means determines that the concentration level is high when a feature amount of the eyeball retention-related potential is greater than the third reference value;
The second concentration level determining means determines that the concentration level is low when the α wave feature value is equal to or greater than the fourth reference value, and the α wave feature value is the fourth reference value. If it is less than the value, it is determined that the degree of concentration is high.
The state determination apparatus according to claim 6. The level of concentration is defined in a plurality of stages by associating the feature amount of the α wave, the feature amount of the eyeball retention-related potential, the third reference value, and the fourth reference value. A correspondence table is further provided,
The determination means refers to the correspondence table when the second concentration degree determination means determines that the α-wave feature value is equal to or greater than the fourth reference value, and from among the plurality of stages, Determining the level of concentration of the subject;
The state determination apparatus according to claim 6. Obtaining means for obtaining the surrounding environment of the subject;
Correction means for correcting the fourth reference value when the surrounding environment that increases the visual information processing load on the subject is acquired by the acquisition means;
The state determination apparatus according to claim 6. The determination means includes
A wakefulness evaluation value calculating means for calculating a wakefulness evaluation value using a predetermined boundary line on a two-dimensional plane composed of the feature quantity of the α wave and the feature quantity of the eyeball retention-related potential;
Evaluation value determination means for determining the arousal level based on the arousal level evaluation value calculated by the arousal level evaluation value calculation means and a predetermined reference value;
The predetermined boundary line is characterized in that when the feature amount of the α wave increases, the feature amount of the eyeball-related potential decreases.
The state determination apparatus according to claim 1. The determination means includes
A degree-of-concentration evaluation value calculating means for calculating a degree-of-concentration evaluation value using a predetermined boundary line on a two-dimensional plane composed of the feature quantity of the α wave and the feature quantity of the eyeball retention-related potential;
An evaluation value determining means for determining the concentration based on the concentration evaluation value calculated by the concentration evaluation value calculating means and a predetermined reference value;
The predetermined boundary line is characterized in that when the feature amount of the α wave increases, the feature amount of the eyeball-related potential increases.
The state determination apparatus according to claim 1. The eyeball stop-related potential feature amount calculating means calculates a saccade eyeball displacement amount immediately before saccade end from the eyeball movement amount, normalizes the feature amount of the eyeball stop-related potential by the saccade eyeball displacement amount,
The determination means determines the arousal level or the concentration level by using the normalized characteristic amount of the eyeball retention-related potential.
The state determination apparatus according to claim 1. A state determination method for determining a state of the subject based on a brain wave signal of the subject,
Calculate the feature quantity of the α wave from the brain wave signal,
Based on the amount of eye movement of the subject and the electroencephalogram signal, the feature amount of the eyeball retention-related potential is calculated,
Based on the calculated feature amount of the α wave and the calculated feature amount of the eyeball-related potential, the arousal level of the subject or the concentration level of the subject is determined.
State determination method.

Images