JPH08280637A - Mental activity judging apparatus - Google Patents

Abstract

PURPOSE: To judge human mental activity in real time. CONSTITUTION: This apparatus has a heart bear interval detection means 11 to detect heart bear intervals, a scattering computation means 12 to compute the scattering of a heart beat interval data detected by the heart beat interval detection means 11, a mean computing means 13 to compute a mean of heart beat intervals or the number of heart beats per specified time based on the heart beart intervals detected by the heart beat interval detection means 11 and a mental activity judging means 14 to judge mental activity of a subject based on the scattering of the heart bear intervals computed by the scattering computation means 12 and the average heart beat interval or the average number of heat beart computed by the mean value-computing means 13.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a mental activity judging device for judging a human mental activity based on a heartbeat waveform.

[0002]

2. Description of the Related Art A mental activity determination method for quantifying human mental activity by paying attention to the fluctuation amount of heartbeat intervals is known (for example, JSME Vol. 51, No. 471, 3153-3158).
See "Objective evaluation method for intellectual work", published in November, 1985). FIG. 24 shows an outline of a conventional mental activity determination method, and FIG. 25 shows a heartbeat waveform. The waveform of the heartbeat of the human heart, that is, the heartbeat, includes R waves, Q waves, S waves, T waves, and the like, and the R wave has the highest peak value among these waves. In this specification, unless otherwise specified, the time interval of the R wave in the heartbeat waveform is called the heartbeat interval, and the heartbeat interval from time t to (t + 1) is RRI (t) (R-
R INTERVAL). The RRI detection unit 1 measures a heartbeat waveform and outputs a series of heartbeat interval data RRI (i) (i = 1,
2, ...) is output. The RRV calculation unit 2 calculates the normalized variance RRV (RR VARI) of the heartbeat interval RRI (i) by the following equation.
ANCE) is calculated and defined as the fluctuation amount of the heartbeat interval RRI (i).

[Equation 1] In the calculation of Expression 1, the fluctuation amount R is calculated by the heartbeat interval data RRI (i) (i = 1 to n) of a predetermined sample number n.
Find the RV. The mental activity determination unit 3 determines the mental activity of the subject based on the average fluctuation amount RRV for 20 to 30 minutes, for example. It has been experimentally confirmed that the user is in a tense state when the fluctuation amount RRV of the heartbeat interval is small, and is in a relaxed state when the fluctuation amount RRRV is large.
When the average RV value RRVavr is smaller than a predetermined value, it is determined that the person is in a tense state, and when the average value RRVavr is not less than the predetermined value, it is determined that the person is in a relaxed state.

(2) RRVavr = ΣRRV (i) / T

Further, the fact that the frequency analysis result of the heartbeat interval data RRI (i) correlates with the activity of the sympathetic nervous system and the activity of the parasympathetic nervous system is used to judge the mental activity of humans. The device is known. FIG. 26 shows a frequency analysis result of a series of measured heartbeat interval data RRI (i), where the horizontal axis represents the frequency F and the vertical axis represents the power spectrum Power. Heartbeat interval data RRI
The frequency analysis result of (i) includes two peak components. In the example shown in FIG. 26, there are peaks at 0.025 to 0.14 Hz and 0.14 to 0.5 Hz, the low frequency side peak component is correlated with the activity of the sympathetic nervous system, and the high frequency side peak component is Correlates with parasympathetic activity.

(Equation 3) Here, S (f) is a power spectrum density function obtained by FFT (frequency analysis) processing of RRI (i). Figure 2
7 and FIG. 28 show changes in the activity of the sympathetic nervous system and the parasympathetic nervous system when two subjects were forcibly stressed and relaxed alternately. In the figure, the horizontal axis is time,
The vertical axis represents the activity level calculated by Equation 3. Also,
The box portion indicates the period during which tension is strong, the thin curve indicates the activity of the sympathetic nervous system, and the thick curve indicates the activity of the parasympathetic nervous system. These curves are the heartbeat interval data RRI
It is calculated by Equation 3 based on the frequency analysis result of (i). In general, it is said that the activity of the sympathetic nervous system reflects the tension state, and the activity of the parasympathetic nervous system reflects the relaxed state. Increased activity of the parasympathetic nervous system is seen within.

[0004]

However, in the former mental activity determination method described above, the mental activity is determined by the average RRV for 20 to 30 minutes after measuring the heartbeat interval data. There is a problem that it cannot be used as a monitor for judging mental activity in real time. Further, it has been confirmed by experiments that the fluctuation amount RRV greatly changes from about 10 ^{−4 to} about 10 ⁻² when the psychological change is large and the scene that affects the heartbeat variability frequently occurs. Such a change in the fluctuation amount RRV should be excluded from the calculation target as noise. However, in the above-described conventional mental activity determination method, since the average value is calculated including such fluctuation amount, the conventional The mental activity judgment method can be used only under experimental conditions in which there is no significant psychological change during work.

In addition, in the latter mental activity determination device described above, the measured heartbeat interval data RRI (i) is analyzed by post-processing, and in addition, complicated component calculation in which four frequency threshold values are set is required. Therefore, there is a problem that it cannot be used as a monitor for determining the mental activity of a human being working in real time, like the former mental activity determining method.

An object of the present invention is to provide a mental activity judging device for judging the mental activity of a human in real time.

[0007]

In order to achieve the above-mentioned object, the invention of claim 1 provides a heartbeat interval detecting means for detecting a heartbeat interval and a dispersion of heartbeat interval data detected by the heartbeat interval detecting means. Calculated by the dispersion calculating means, an average value calculating means for calculating an average value of heartbeat intervals or heartbeats at predetermined time intervals based on the heartbeat intervals detected by the heartbeat interval detecting means, and the dispersion calculating means. And a mental activity determining means for determining the mental activity of the subject based on the variance of the heartbeat intervals and the average heartbeat interval or the average heart rate calculated by the average value calculating means. Claim 2
The mental activity determination device of, by the mental activity determination means, develops on the two-dimensional plane the variance of the heartbeat intervals calculated by the variance calculator and the average heart rate calculated by the average value calculator. The mental activity is determined based on the average coordinates. The mental activity determination device according to claim 3 calculates the short-term average heart rate and the long-term average heart rate for a longer time by the average value calculation means, and the mental activity determination means calculates the short-time average heart rate. By comparing the heart rate with the long-term average heart rate, only the short-term average heart rate with a difference between the long-term average heart rate within a predetermined value is used as the average heart rate for the determination of mental activity. is there. The mental activity determination device according to claim 4 is
By the mental activity determination means, the variance of the heartbeat intervals calculated by the variance calculation means and the average heart rate calculated by the average value calculation means are developed on a two-dimensional plane, and based on the change pattern of the coordinates. It is designed to judge mental activity. According to the invention of claim 5, a heartbeat interval detecting means for detecting a heartbeat interval, a frequency analyzing means for performing a frequency analysis of the heartbeat interval data detected by the heartbeat interval detecting means, and a predetermined frequency analysis result from the frequency analyzing means. Frequency component extraction means for extracting low frequency components below a frequency and high frequency components higher than the predetermined frequency, and mental activity of the subject based on the low frequency components and high frequency components extracted by the frequency component extraction means And mental activity determining means for determining. The mental activity determining apparatus according to claim 6 expands the low frequency component and the high frequency component of the frequency analysis result of the heartbeat interval data on the two-dimensional plane by the mental activity determining means, and based on the change pattern of the coordinates. It is designed to judge mental activity.

[0008]

According to the mental activity judging apparatus of the first aspect, the mental activity of the subject is judged based on the variance of the heartbeat intervals and the average heartbeat interval or the average heart rate. In the mental activity determination device according to the second aspect, the variance of the heartbeat intervals and the average heart rate are developed on a two-dimensional plane, and the mental activity is determined based on the average coordinates.
In the mental activity determination device of the third aspect, only the short-term average heart rate whose difference from the long-term average heart rate is within a predetermined value is used as the average heart rate for the determination of mental activity. In the mental activity determination device according to the fourth aspect, the variance of the heartbeat intervals and the average heart rate are developed on a two-dimensional plane, and the mental activity is determined based on the change pattern of the coordinates. In the mental activity determination device according to the fifth aspect, the mental activity is determined based on the low frequency component and the high frequency component extracted from the frequency analysis result of the heartbeat interval data.
In the mental activity determination device according to the sixth aspect, the low frequency component and the high frequency component of the frequency analysis result of the heartbeat interval data are developed on a two-dimensional plane, and the mental activity is determined based on the change pattern of the coordinates.

[0009]

【Example】

First Embodiment FIG. 1 is a diagram showing the configuration of the first embodiment. The microcomputer 5 has a CPU and peripheral parts such as a memory, executes a control program to be described later to determine the mental activity of the subject, and controls according to the determination result. Here, the subject who measures the heartbeat is assumed to be an operator of some device or an observer. The electrocardiogram detector 6 is a device that measures an electrocardiogram signal of a subject. This detector 6 is equipped with electrodes for detecting a potential signal on the subject, but it is assumed that there is no problem because it is assumed that the subject does not have vigorous body movements. In addition, since the purpose is not to accurately measure an electrocardiogram for medical purposes, a simple method can be applied to attach electrodes. In addition, a heartbeat waveform can be similarly measured with a pulse wave that can be simply measured. That is, the pulse wave sensor may be attached to the ear or the hand to detect the pulse wave, and the heartbeat waveform may be measured from these pulse waves. The pulse wave has a waveform close to a sine wave, but the heartbeat interval can be detected similarly to the R wave. The inter-vehicle distance detecting device 7 is a device that measures the inter-vehicle distance to the former by using, for example, an ultrasonic sensor. The engine control device 8 is a device that controls acceleration, deceleration, stop, etc. of the vehicle engine, and the alarm device 9 is a device that sounds a buzzer to issue an alarm. The microcomputer 5 controls the engine control device 8 and the alarm device 9 based on the electrocardiogram signal detected by the electrocardiogram detector 6 and the inter-vehicle distance detected by the inter-vehicle distance detection device 7.

FIG. 2 is a control block diagram of the microcomputer 5. The RRI detection unit 11 includes an electrocardiogram detector 6
A series of heartbeat interval data RRI (i) (i = 1, 2, ...) Is detected by the electrocardiogram signal measured by. R
The RV calculation unit 12 calculates the heartbeat interval RRI according to the above equation 1.
The normalized variance RRV of (i) is obtained, and the heartbeat interval RRI is calculated.
It is output as the fluctuation amount of (i). BEAT calculation unit 13
Calculates the average heart rate BEAT [times / minute] every minute from the heartbeat waveform measured by the RRI detection unit 11.
The mental activity determination unit 14 determines the mental activity of the subject based on the fluctuation amount RRV of the heartbeat intervals calculated by the RRV calculation unit 12 and the average heart rate BEAT calculated by the BEAT calculation unit 13. Further, the control unit 15 performs control according to the determination result of the mental activity determination unit 14.

The processing of the RRI detection unit 11 will be described with reference to the RRI detection processing routine shown in FIG. In step 1, the subject's electrocardiogram signal is input from the electrocardiogram detector 6,
Sampling at a given frequency. In this embodiment, the sampling frequency is 100 Hz, for example. In the following step 2, the sampled heartbeat waveform is filtered to remove the baseline fluctuation. In this embodiment, the heartbeat waveform is subjected to bandpass filter processing of 6 to 30 Hz, for example.
Next, in step 3, a wave exceeding a predetermined threshold value in the heartbeat waveform is detected as an R wave, and its time interval RRI is detected. The detected heartbeat interval RRI is originally 0.6 to 1.
Since there is a fluctuation of about 2 seconds, the heartbeat interval data RRI (i) becomes irregular data in time series as shown in FIG. Therefore, in step 4, the previous value is interpolated at 0.25 second (4 Hz) intervals to perform resampling processing.
As shown in (b), heartbeat interval data RRI (i) regularly sampled at 4 Hz can be obtained.

In the RRV calculator 12, the RRI detector 11
By the heartbeat interval data RRI (i) detected by
The fluctuation amount RRV (t) of the heartbeat interval is calculated every predetermined time. In this embodiment, for example, the fluctuation amount R is generated every 16 seconds.
It is assumed that RV (t) is calculated. Therefore, Equation 1
The number of samples n becomes 64 (= 16 / 0.25), and the fluctuation amount RRV (t) of the heartbeat interval at the time t is calculated by the following formula obtained by modifying the formula 1.

[Equation 4] Although the amount of fluctuation is discussed here in the form of dispersion,
The same can be said when discussing the standard deviation. On the other hand, BE
In the AT calculation unit 13, 16 seconds of the heartbeat interval data RRI (i) detected by the RRI detection unit 11, that is, 64
Based on the average value of the individual heartbeat intervals RRI (i), the average heart rate BEAT is calculated by the following equation.

## EQU00005 ## BEAT (t) = 60 / [{. SIGMA.RRI (t + i-32)} / 64] where .SIGMA. Represents the summation operation of i = 1 to 64. In addition,
In this embodiment, the fluctuation amount RRV of the heartbeat interval is calculated every 16 seconds.
Although (t) and the average heart rate BEAT are calculated, they may be calculated each time an R wave is detected. In that case, irregular sampling data will be obtained, but mental activity can be determined.
Although the average heart rate is explained here, this has an advantage that the RRI fluctuation due to respiration can be taken and the process described later becomes easy. Also, not the average heart rate, 60 /
RRI may be BEAT (t), and mental activity may be determined based on so-called instantaneous heart rate.

Next, the processing of the mental activity determination section 14 will be described with reference to the mental activity determination routine shown in FIG. Step 1
1, the fluctuation amount (-RRV) of the heartbeat interval calculated by the RRV calculation unit 12 and the average heart rate BEAT calculated by the BEAT calculation unit 13 are associated with the coordinates (x, y) on the two-dimensional plane. Two-dimensional information P (-RRV,
It develops as BEAT). In this embodiment, in order to make it easier to understand the change in mental activity, the polarity of the fluctuation amount RRV of the heartbeat interval is reversed.

6 (a) and 6 (b) are RRVs when subject 1 and subject 2 are made to alternately perform work and break.
-BEAT characteristics are shown. In each figure, the vertical axis represents the average heart rate BEAT [times / minute], and the horizontal axis represents the fluctuation amount (-RRV) [× 10 ^-4 ] of the heartbeat intervals. Although the values of -RRV and BEAT are different between the test subject 1 and the test subject 2, the fluctuation amount of the heartbeat interval (-RRV) and the average heart rate BEAT become high at the time of work, and the fluctuation amount of the heartbeat interval at the time of rest ( -RRV) and average heart rate BEAT were confirmed to be low. As a result of conducting such an experiment on a large number of subjects, as shown in FIG. 7, when the mental activity becomes high as during work, the fluctuation amount of the heartbeat interval (-RR
V) and the average heart rate BEAT become large, and when the mental activity becomes low, such as during a break, the fluctuation amount of the heartbeat interval (-RR
V) and the average heart rate BEAT become smaller. That is, it is shown that the mental activity of the subject can be determined based on the fluctuation amount (-RRV) of the heartbeat interval and the average heart rate BEAT. In the following, the degree of mental activity is referred to as mental activity. According to the above-described conventional determination method for determining the mental activity based on only the average fluctuation amount RRV, when the fluctuation amounts RRV of the heartbeat intervals have the same value, it is not possible to distinguish and the mental activity cannot be determined. However, according to this example,
Since the average heart rate BEAT is considered in addition to the fluctuation amount RRV of the heartbeat interval, it is possible to accurately determine the mental activity in any situation.

In step 12 of FIG. 5, it is determined whether or not to prepare data as a criterion for mental activity. The two-dimensional data, which is the reference data, differs from person to person, so it is indispensable when determining mental activity. Therefore, if there is no reference data, it is not possible to determine the mental activity, so the process proceeds to step 13 and new reference data is created. Even if there is reference data, it is not possible to accurately determine mental activity if it is older than a certain time since the reference data was created.
Go to 3 and update the reference data. If the predetermined time has not elapsed since the reference data was created, the reference data is used for the determination, and step 1
Go to 4. When the reference data is created or updated, in step 13, P (-RRV, BE during the predetermined time T0.
AT mean) B (x0, y0) and variance (σx, σy)
Is calculated. When the already created reference data is used, in step 14, the determination routine by the averaging process shown in FIG. 8 is executed, the variance of the heartbeat interval data and the average heart rate are developed on a two-dimensional plane, and Determine mental activity based on average coordinates. In the following step 15, in FIG.
The determination routine based on the change pattern shown in 2 is executed, the variance of the heartbeat interval data and the average heart rate are developed on a two-dimensional plane, and the mental activity is determined based on the change pattern of the coordinates. The mental activity may be determined by either the averaging process in step 14 or the change pattern in step 15.

The determination process of the mental activity by the averaging process will be described with reference to FIG. Two-dimensional data P (-RRV, BE
AT) changes greatly when the state of concentration is released (hereinafter referred to as relaxed transition) and when the mind is suddenly concentrated (hereinafter referred to as concentrated transition). In the determination of the mental activity by the averaging process, it is more accurate to exclude such data P. Therefore, in step 21, the change point data deletion routine shown in FIG. 9 is executed to delete the data P (hereinafter referred to as change point data) whose value has greatly changed from the series of data strings P (t), and to create new data. Row P
Create n (t).

In step 31 of FIG. 9, the average value PL of the data P at the long predetermined time T1 is calculated by the following equation.

[Mathematical formula-see original document] PL (t) = [Sigma] P (i) / T1 Here, [Sigma] represents the summation operation of the data P (i). In the following step 32, a predetermined time T2 shorter than T1 is calculated by the following equation.
The average value PS of the data P at (T2 <T1) is calculated.

## EQU00007 ## PS (t) =. SIGMA.P (i) / T2 where .SIGMA. Represents the summation operation of the data P (i). Next, at step 33, the difference (PL-P
It is determined whether or not the absolute value of S) is greater than or equal to a predetermined value d. In step 35, the above processing is performed on all the data P to determine whether all the change point data have been deleted. If the processing has been completed, the process returns to step 22 in FIG. If there is, return to step 31 and repeat the above processing.
As a result, a new data string P that does not include change point data
n (t) is created.

From step 22 of FIG. 8 after the return, the mental activity determination process is performed based on the new data string Pn (t). In the determination by the averaging process, (1) the fluctuation amount RRV of the heartbeat interval at time t and the average heartbeat BEAT
Instantaneous determination (steps 22 to 23) based on the data Pn (t) of (1) and (2) Global determination (step 24) to perform global determination using the average value of the data Pn in a certain long predetermined time T3. ~ 26) and (3) T3
Data P at a shorter predetermined time T4 (T4 <T3)
Spot determination (steps 27 to 29) is performed to make a spot-like determination using the average value of n. In the instant determination, in step 22, the data P at time t
Differences Vx and Vy between (t) and the reference average data B (x0, y0) described above are calculated by the following equation.

Vx = x−x0, Vy = y−y0 where x is the fluctuation amount (−R) of the heartbeat interval at time t.
RV), y is the average heart rate BEAT at time t. In step 23, the mental activity determination routine shown in FIG. 10 is executed to make an instantaneous determination of mental activity based on the difference Vx, Vy.

In step 41 of FIG. 10, the difference Vx,
When it is determined that Vy is larger than the reference variance data σx and σy, respectively, the routine proceeds to step 42, where it is determined that the degree of mental activity is considerably high. Further, in steps 41 and 43, the differences Vx and Vy are the reference variance data σx, respectively.
When it is determined that it is less than or equal to σy and greater than σx / 2 and σy / 2, the process proceeds to step 44 and it is determined that the degree of mental activity is high. Furthermore, when it is determined in step 45 that the differences Vx and Vy are smaller than (−σx / 2) and (−σy / 2), respectively, it is determined in step 46 that the degree of mental activity is low.

Next, in the global judgment, in step 24, the data Pn (t) at a certain long predetermined time T3
The average value PL (t) of In the following step 25,
The average values PLx and PLy and the reference average data B (x
0, y0) and the difference Vx, Vy is calculated by the following equation.

## EQU9 ## Vx = PLx-x0, Vy = PLy-y0 In step 26, the mental activity determination routine shown in FIG. 10 is executed, and the global determination of mental activity is performed based on the difference Vx, Vy as described above. .

In the spot-like determination, in step 27, the data P for a short predetermined time T4 (T4 <T3)
The average value PS (t) of n (t) is calculated. In the following step 28, differences Vx, Vy between the average values PSx, PSy and the above-mentioned reference average data B (x0, y0) are calculated by the following equation.

Vx = PSx-x0, Vy = PSy-y0 In step 29, the mental activity determination routine shown in FIG. 10 is executed, and the mental activity spot determination is performed based on the difference Vx, Vy as described above. .

Next, a method for determining mental activity based on the change pattern will be described. FIG. 11 shows experimental data when a test run of a vehicle consisting of a straight road and a bank road was test-run at a speed of about 160 k / h. FIG. 7A shows heartbeat data when traveling on a bank road, and FIG. 7B shows heartbeat data when traveling on a straight road. The abscissas of the graphs (a) and (b) represent the fluctuation amount (-RRV) [× 10 ^-4 ] of the heartbeat intervals, and the ordinate represents the average heartbeat Beat [times / min]. (C) shows a change pattern of mental activity. (D) shows the occurrence frequency of relaxed transition and concentrated transition, the horizontal axis represents time t, the upper part of the vertical axis represents the occurrence frequency of relaxed transfer [%], and the lower part of the vertical axis represents the occurrence frequency of concentrated transfer [. %]. As is clear from these experimental data, immediately after entering the bank road and the straight road, the frequency of the intensive transition in which the data P (-RRV, BEAT) changes to the left and then to the right increases. . Then, after a while, the data P
The frequency of relaxation transition, which changes to the left and then to the right, increases. In this experiment, the degree of mental concentration was high immediately after the entry into the bank road and the straight road, and the experimental data shown in FIG. 11 agrees with the self-reported sensory evaluation of the subject. That is, the fluctuation amount RRV of the heartbeat interval
It can be said that the change pattern of the average heart rate BEAT and the change of the average heart rate BEAT accurately represent the instantaneous heart movement and psychological movement of the subject. According to such a mental activity determination method, various states of the subject can be accurately determined in real time.
For example, when the subject is dozing, the average heart rate B
Since the EAT is small and the fluctuation amount RRV of the heartbeat interval is relatively large, it is possible to determine the dozing state by setting a predetermined criterion.

FIG. 12 shows the mental activity determination process based on the change pattern. In the determination based on this change pattern, the time interval T at time (t-dt), t, (t + dt)
Average value P0 of the data P (-RRV, BEAT) of
Based on (t), P1 (t), and P2 (t), it is determined whether the relaxation transition or the intensive transition. The time interval T is 1
It may be 0 to 16 seconds or 1 minute, but if a too long time is set, the instantaneous characteristics are lost. Step 51
At, the average values P0 (t), P1 (t), P2 (t) of the data P (-RRV, BEAT) of the time interval T at the times (t-dt), t, (t + dt) are calculated. In the following step 52, the average value P0 (t) is changed to the average value P1.
Whether or not the change to (t) is lower left than the predetermined value d is determined by the following equation.

## EQU11 ## P0x (t)> P1x (t) + d, P0y (t)> P1y (t) + d If leftward, proceed to step 53, otherwise proceed to step 55. In step 53, the average value P1
Whether or not the change from (t) to the average value P2 (t) falls to the right larger than the predetermined value d is determined by the following equation.

## EQU12 ## P1x (t) <P2x (t) -d, P1Y (t)> P2y (t) + d If it is a downward slope, the process proceeds to step 54. If not, the program returns to the program shown in FIG. In step 54, the average value of the data P of the time interval T at time (t-dt) and time t changes to the left downward from P0 (t) to P1 (t), and at time t and time (t + dt). The average value of the data P of the time interval T is from P1 (t) to P2
Since it has changed to the lower right toward (t), FIG.
As shown in, it is determined that the transition is a relaxed transition.

On the other hand, when it is determined in step 52 that the angle is not lowering to the left, the routine proceeds to step 55, where the average value P0 (t)
From the average value P1 (t) to the average value P1 (t) is determined by the following equation as to whether or not the change is greater to the left than the predetermined value d.

## EQU13 ## P0x (t)> P1x (t) + d, P0y (t) <P1y (t) -d If leftward, go to step 56, otherwise return to the program shown in FIG. Step 56
Then, it is determined by the following equation whether or not the change from the average value P1 (t) to the average value P2 (t) is an upward increase larger than the predetermined value d.

## EQU14 ## P1x (t) <P2x (t) -d, P1y (t) <P2y (t) -d If it is rising to the right, the process proceeds to step 57. If not, the process is terminated and shown in FIG. Return to the program.
In step 57, the average value of the data P of the time interval T between the time (t-dt) and the time t is from P0 (t) to P1.
It changes to the left upward to (t), and at time t and time (t +
The average value of the data P of the time interval T in dt) is P1.
Since it is changing to the right upward from (t) to P2 (t), it is determined to be the intensive shift as shown in FIG. 11 (c).

FIG. 13 shows a modification of the mental activity determination process based on the change pattern. Based on the average values P1 (t) and P2 (t) of the data P (-RRV, BEAT) of the time interval T at the time t and the time (t + dt), it is determined whether the transition is the relaxed transition or the concentrated transition. The time interval T may be 0. In step 61, time t and time (t + dt)
At time interval T in P (-RRV, BEAT)
The average values P1 (t) and P2 (t) of are calculated. At step 62, the number of consecutive changes of P1x> P2x and P1y> P2y is counted, that is, the number of consecutive falling times KD1 of the data P is counted, and at the following step 63, whether the count value KD1 is the predetermined value N1 or more. To determine. KD1 ≧
If it is N1, the process proceeds to step 64, and if not, the process proceeds to step 67. In step 64, P1x <P2x,
The number of consecutive changes of P1y> P2y, that is, the number of consecutive downward falling times KD2 of the data P is counted, and in a succeeding step 65, it is determined whether or not the count value KD2 is equal to or more than a predetermined value N2. If KD2 ≧ N2, step 66
If not, the process ends. Assuming that the predetermined value N1 is set to 5 and the predetermined value N2 is set to 4, the leftward decrease from P1 (t) to P2 (t) continues for 5 times or more, and further from P1 (t). When the downward descent to P2 (t) continues four times or more in succession, step 66
Then, it is determined that the transition is a relaxation transition during (KD1 + KD2).

On the other hand, when the leftward downward does not continue for N1 times or more, in step 67, P1x> P2x, P1y <P2
How many times the change of y is continuous, that is, the number of times KU1 of upward rising of the data P is counted, and the following step 6
At 8, it is determined whether the count value KU1 is equal to or greater than the predetermined number N1. If KU1 ≧ N1, the process proceeds to step 69, and if not, the process ends. In step 69, P1
How many times x <P2x and P1y <P2y are continuously changed, that is, the number of times KU2 of upward rising of the data P is counted, and in the subsequent step 70, it is determined whether or not the count value KU2 is equal to or more than a predetermined value N2. . If KU2 ≧ N2, the process proceeds to step 71. If not, the process ends.
When the upward ascending from P1 (t) to P2 (t) continues for 5 times or more and the upward ascending from P1 (t) to P2 (t) continues for 4 or more times continuously, step 71 so,
During (KU1 + KU2), it is determined that the shift is concentrated.

FIG. 14 is a flowchart showing the automatic speed control of the control unit 15. The control unit 15 controls the engine control device 8 and the alarm device 9 according to the determination result of the mental activity by the mental activity determination unit 14. In step 71, it is determined whether or not the vehicle-to-vehicle distance detected by the vehicle-to-vehicle distance detecting device 7 becomes less than or equal to a predetermined value and the vehicle approaches the front vehicle. When the vehicle approaches the front vehicle, the routine proceeds to step 72, where it is determined whether or not the mental activity determination unit 14 determines that the mental activity level is high or the concentration shifts. If the degree of mental activity is high or the shift is concentrated, it is determined that the occupant recognizes the approach to the preceding vehicle, and the process returns to step 71. When the determination is performed by the averaging process described above, the determination result of any one of the instantaneous determination, the global determination and the spot determination or a plurality of determination results are used. On the other hand, if the mental activity is low even though the vehicle is approaching the front vehicle, and the vehicle does not shift to the concentrated state, it is determined that the occupant does not recognize the front vehicle approaching, the process proceeds to step 73, and the engine control device 8 drives the traveling speed. Is decelerated, and an alarm device 9 gives an alarm.

In this way, the mental activity of the subject is judged based on the fluctuation amount RRV of the heartbeat intervals and the average heart rate BEAT. In the determination, the fluctuation amount RRV of the heartbeat interval and the average heart rate BEAT are developed on a two-dimensional plane, and the mental activity is judged based on the average coordinates, or the fluctuation amount RRV of the heartbeat interval and the average heart rate are determined. BEAT is developed on a two-dimensional plane, and mental activity is judged based on the change pattern of the coordinates. Thereby, the mental activity of the subject can be accurately determined in real time. Also, when the judgment result of mental activity is low when the vehicle is running ahead and the mental activity level is low or it is not concentrated transition,
Since it is determined that the occupant of the vehicle does not recognize the approach to the former, the vehicle is automatically decelerated and the alarm is issued. Therefore, it is possible to avoid the abnormal approach of the occupant to the former due to the drowsiness of the occupant.

In the configuration of the first embodiment described above, the electrocardiogram detector 6 and the RRI detector 1 of the microcomputer 5 are used.
Reference numeral 1 is a heartbeat interval detecting means, RRV calculating section 12 of the microcomputer 5 is distributed calculating means, BEAT calculating section 13 of the microcomputer 5 is heart rate detecting means, and mental activity determining section 14 of the microcomputer 5 is mental activity determining section. Each means is configured.

Second Embodiment In the above-mentioned embodiment, the mental activity is determined based on the fluctuation amount RRV of the heartbeat interval and the average heart rate BEAT, but based on the frequency component of the frequency analysis result of the heartbeat interval data. A second embodiment for judging mental activity will be described. The configuration of the second embodiment is similar to that of the first embodiment shown in FIG. 1, and illustration and description thereof will be omitted.

FIG. 15 is a control block diagram of the microcomputer of the second embodiment. The first shown in FIG.
The same elements as those in the control block diagram of the embodiment will be designated by the same reference numerals, and the description will focus on the differences. RRI detector 1
1 is a series of heartbeat interval data RRI (i) (i = 1, 2, ...) According to the electrocardiogram signal measured by the electrocardiogram detector 6.
・ ・) Is detected. The frequency component calculation unit 21 uses the heartbeat interval data RRI (i) detected by the RRI detection unit 11.
Is frequency analyzed to calculate a predetermined frequency component. The mental activity determination unit 22 determines the mental activity of the subject based on the frequency components calculated by the frequency component calculation unit 21.
Further, the control unit 15 performs control according to the determination result of the mental activity determination unit 21.

The detailed processing in the RRI detection unit 11 is the same as the processing shown in FIG.
The processing of the frequency component calculation unit 21 will be described with reference to the frequency component calculation routine shown in FIG. In step 81, the analysis section is cut out prior to the frequency analysis of the heartbeat interval data RRI (i) detected by the RRI detection unit 11. In this embodiment, the analysis interval is 64 seconds (2 seconds) in consideration of the real-time determination of mental activity and the accuracy of information.
56 samples), and this analysis section is shifted every 5 seconds. As a result, it is unavoidable to wait for the processing result for data acquisition for the first about 1 minute, but once the processing is started, the processing result is obtained every 5 seconds.
In step 82, frequency analysis is performed on the cut out heartbeat interval data RRI (i) of the analysis section. For frequency analysis, it is desirable to use a high-speed FFT analyzer to improve the real-time determination of mental activity.

FIG. 17 shows an example of the frequency analysis result, in which the horizontal axis represents frequency [Hz] and the vertical axis represents power spectrum Power. In step 83, a frequency component equal to or higher than the predetermined frequency Fo and a frequency component lower than Fo are extracted from the frequency analysis result. The predetermined frequency Fo is 0.1
It has been experimentally confirmed that ~ 0.15 Hz is optimum. In the conventional mental activity determination device described above, 0.0
Since it takes time to extract the frequency components of 25 to 0.14 Hz and 0.14 to 0.5 Hz, this embodiment only extracts the frequency components above and below the single predetermined frequency Fo. Therefore, the calculation time is about 1/4 of that of the conventional device, the burden on the microcomputer can be reduced, and the real-time mental activity can be improved. In step 84, the analysis section is shifted by 5 seconds and the process proceeds to step 85. It is determined whether or not there is a new analysis section of 64 seconds, 256 samples or more. If a new analysis section can be secured, the process returns to step 81 and the above processing is repeated. If a new analysis section cannot be secured, the process ends.

The mental activity judging section 22 judges the mental activity based on the frequency components calculated by the frequency component calculating section 21. Heartbeat interval data R as shown in FIG.
In the frequency analysis result of RI, the low frequency component is defined as the instantaneous mental activity, and the high frequency component is defined as the average mental activity, which are calculated by the following equations.

(Equation 15) Here, S (f) is a power spectral density function obtained by FFT processing the heartbeat interval data RRI.

FIG. 18 and FIG. 19 show changes in the instantaneous mental activity and the average mental activity when two subjects are forcibly stressed and relaxed alternately. In the figure, the horizontal axis represents time, and the vertical axis represents the activity level calculated by Expression 15. Further, the box portion is a period in which the subject is allowed to perform a predetermined task with the aim of achieving the highest grade and is in a tense state, and the other portion is in a relaxed state in which such work is released. The thin curve in the figure shows the instantaneous mental activity of the low frequency component. Since the low frequency component is a component close to the direct current component, it reflects a large change in mental activity. As can be seen from the figure, the low frequency components increase rapidly before and after the box, that is, when the subject starts or finishes performing the task, and reflects the instantaneous degree of mental activity. . That is, the increase of the low frequency component indicates the changing process of the mental activity, and the decrease of the low frequency component indicates the steady state of the mental activity. On the other hand, the thick curve in the figure shows the average mental activity of the high frequency components. The high-frequency component reflects not a large change in mental activity, but a small change and an average change in mental activity that frequently occurs. As is clear from the figure, the high-frequency component decreases in the tensioned box and increases outside the box, so an increase in the high-frequency component indicates a relaxed state and a decrease reflects the tensioned state. There is.

It can be seen that the above-mentioned changes in the instantaneous mental activity and the average mental activity have occurred before the transition to the tense state, and reflect the preparation state of the subject. That is, it is possible to predict what kind of state the subject will transition to based on the instantaneous mental activity and the average mental activity, and determine the subject's mental activity according to the determination and the determination result. It is possible to provide effective information for real-time control.

20 and 21 show the low frequency component and the high frequency component, which are shown in FIGS. 18 and 19, respectively, which are divided into two-dimensional coordinates P (x, y), and the x axis is the low frequency component. It represents the power spectrum of the component, and the y-axis represents the power spectrum of the high frequency component. In these figures, two convergence areas are observed in the area on the left of the two-dimensional plane. Since the low frequency component is small and the high frequency component is large in the relaxed state as shown in FIGS. 18 and 19, the upper convergence area corresponds to the relaxed state. On the other hand, as shown in FIGS. 18 and 19, both the low-frequency component and the high-frequency component are small in the tensioned state, so the lower convergence point corresponds to the tensioned state. In addition, the upper converging area showing a relaxed state is swelled to the right side to the lower converging area showing a tense state. Corresponding to the changing process of. That is, FIG.
As shown in, the transition from the relaxation area to the tension area is the tension transition, and conversely, the transition from the tension area to the relaxation area is the relaxation transition. In this way, by expanding the low frequency component and high frequency component of the frequency analysis result of the heartbeat interval data on the two-dimensional plane, even if there is a difference in the power spectrum of the frequency component of each subject, the mental activity of each subject Can be accurately evaluated.

FIG. 23 is a flow chart showing mental activity determination processing in the mental activity determination unit 22. In step 91, data P (x; low frequency component, at time interval T at time (t-dt), t, (t + dt),
y; high frequency component) average values P0 (t), P1 (t),
Calculate P2 (t). In the following step 92, the average value P
Whether or not the change from 0 (t) to P2 (t) is the transition from the tension area to the relaxation area shown in FIG. 22 is determined by the following equation.

P1y (t)> P0y (t) + d, P2y (t)> P1y (t) + d, P1x (t)> P0x (t), P1x (t)> P2x (t) In the affirmative, the routine proceeds to step 93, where it is determined that the subject's mental activity is a relaxation transition. On the other hand, if the determination in Expression 16 is denied, the process proceeds to step 94 to determine whether the change from the average value P0 (t) to P2 (t) is the transition from the relax area to the tension area shown in FIG. Determined by

P1y (t) <P0y (t) -d, P2y (t) <P1y (t) -d, P1x (t)> P0x (t), P1x (t)> P2x (t) If the determination is affirmative, the routine proceeds to step 95, where it is determined that the subject's mental activity is a transition to tension.

As described above, the low frequency component and the high frequency component extracted from the frequency analysis result of the heartbeat interval data are developed on the two-dimensional plane, and the mental activity is determined based on the change pattern of the coordinates. Therefore, the mental activity of the subject can be accurately determined in real time.

In the second embodiment described above, the electrocardiogram detector 6 and the RRI detection section 11 of the microcomputer are heartbeat interval detection means, the frequency component calculation section 21 of the microcomputer is frequency analysis means, and the mental activity of the microcomputer. The judging section 22 constitutes a mental activity judging means.

In the above-mentioned embodiment, an example in which the speed of the vehicle is controlled according to the determination result of mental activity is shown.
The application example of the determination result of the mental activity is not limited to the above embodiment. For example, it can be applied to monitoring work in factories and airplanes. During the series of steps, a time zone is set beforehand so that the observer must not make any mistakes, and if the mental activity of the observer does not shift intensively during that time, first refresh the room temperature or scent. If it tries, and if it does not work, it will give an additional alarm. This can reduce the influence of getting used to and getting tired of the alarm, and can contribute to the prevention of human error due to rut.

[0042]

As described above, according to the present invention, the mental activity of the subject is judged based on the variance of the heartbeat interval data and the average heartbeat interval or the average heartbeat rate. It can be accurately determined in real time. In the determination, the variance of the heartbeat interval data and the average heartbeat interval or the average heart rate may be developed on a two-dimensional plane and the mental activity may be determined based on the average coordinates, or the heartbeat interval data may be determined. May be developed on the two-dimensional plane and the average heartbeat interval or the average heartbeat rate, and the mental activity may be determined based on the change pattern of the coordinates. Further, since the low frequency component and the high frequency component extracted from the frequency analysis result of the heartbeat interval data are developed on a two-dimensional plane and the mental activity is determined based on the change pattern of the coordinates, the subject's Mental activity can be accurately determined in real time.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment.

FIG. 2 is a control block diagram of the first embodiment.

FIG. 3 is a flowchart showing RRI detection processing of the first embodiment.

FIG. 4 is a diagram showing heartbeat interval data.

FIG. 5 is a flowchart showing mental activity determination processing of the first embodiment.

FIG. 6 is a diagram showing the characteristics of the fluctuation amount of the heartbeat interval and the average heart rate when the subject is made to alternately perform work and break.

FIG. 7 is a diagram illustrating a method for determining mental activity based on the characteristics of fluctuation amount of heartbeat interval and average heart rate.

FIG. 8 is a flowchart showing a determination process by the averaging process of the first embodiment.

FIG. 9 is a flowchart showing change point data deletion processing according to the first embodiment.

FIG. 10 is a flowchart showing mental activity determination processing according to the first embodiment.

FIG. 11 is a diagram showing heartbeat data when a test run consisting of a straight road and a bank road is tested at high speed.

FIG. 12 is a flowchart showing a determination process based on a change pattern according to the first embodiment.

FIG. 13 is a flowchart showing a modification of the determination process according to the change pattern of the first embodiment.

FIG. 14 is a flowchart showing automatic speed control according to the first embodiment.

FIG. 15 is a control block diagram of the second embodiment.

FIG. 16 is a flowchart showing frequency component calculation according to the second embodiment.

FIG. 17 is a diagram showing a frequency analysis result of heartbeat interval data.

FIG. 18 is a diagram showing changes in the instantaneous mental activity level and the average mental activity level when the subject is alternately stressed and relaxed.

FIG. 19 is a diagram showing changes in the instantaneous mental activity and the average mental activity when the tension and the relaxation are alternately applied to other subjects.

20 is a diagram in which the instantaneous mental activity and the average mental activity of FIG. 18 are developed on a two-dimensional plane.

FIG. 21 is a diagram in which the instantaneous mental activity and the average mental activity of FIG. 19 are developed on a two-dimensional plane.

FIG. 22 is a diagram illustrating a method for determining mental activity according to the second embodiment.

FIG. 23 is a flowchart showing mental activity determination processing according to the second embodiment.

FIG. 24 is a diagram showing an outline of a conventional mental activity determination method.

FIG. 25 is a diagram showing a heartbeat waveform.

FIG. 26 is a diagram showing a frequency analysis result of heartbeat interval data.

FIG. 27 is a diagram showing changes in activity of the sympathetic nervous system and the parasympathetic nervous system when a subject is alternately forced to relax and relax.

FIG. 28 is a diagram showing changes in activity of the sympathetic nervous system and the parasympathetic nervous system when tension and relaxation are alternately applied to other subjects.

[Explanation of symbols]

 5 Microcomputer 6 Electrocardiogram detector 7 Inter-vehicle distance detection device 8 Engine control device 9 Alarm device 11 RRI detection part 12 RRV calculation part 13 BEAT calculation part 14 Mental activity determination part 15 Control part 21 Frequency component calculation part 22 Mental activity determination part

Claims (6)

[Claims] 1. A heartbeat interval detecting means for detecting a heartbeat interval, a dispersion calculating means for calculating a variance of heartbeat interval data detected by the heartbeat interval detecting means, and a heartbeat interval detected by the heartbeat interval detecting means. An average value calculating means for calculating an average value of the heartbeat intervals or heartbeats for each predetermined time based on the above; a variance of the heartbeat intervals calculated by the variance calculating means; and an average heartbeat interval or an average calculated by the average value calculating means A mental activity judging device comprising: a mental activity judging means for judging a mental activity of a subject based on a heart rate. 2. The mental activity determination device according to claim 1, wherein the mental activity determination means is a variance of heartbeat intervals calculated by the variance calculation means and an average heart rate calculated by the average value calculation means. Is developed on a two-dimensional plane, and the mental activity determination device is characterized by determining mental activity based on the average coordinates. 3. The mental activity determining device according to claim 2, wherein the average value computing means computes a short-term average heart rate and a long-term average heart rate for a longer time, and the mental activity determining means. Compares the short-term average heart rate with the long-term average heart rate, and uses only the short-term average heart rate within a predetermined value as a difference from the long-term average heart rate to determine mental activity. A mental activity determination device characterized by the above. 4. The mental activity determination device according to claim 2, wherein the mental activity determination means is a variance of heartbeat intervals calculated by the variance calculation means and an average heart rate calculated by the average value calculation means. A mental activity determination device characterized in that the mental activity is determined on the basis of a change pattern of the coordinates developed on a two-dimensional plane. 5. A heartbeat interval detecting means for detecting a heartbeat interval, a frequency analyzing means for performing a frequency analysis of heartbeat interval data detected by the heartbeat interval detecting means, and a predetermined frequency or less from a frequency analysis result by the frequency analyzing means. Frequency component extraction means for extracting a low frequency component and a high frequency component higher than the predetermined frequency, and the mental activity of the subject is determined based on the low frequency component and the high frequency component extracted by the frequency component extraction means. A mental activity determination device, comprising: 6. The mental activity determination device according to claim 5, wherein the mental activity determination means develops a low frequency component and a high frequency component of a frequency analysis result of heartbeat interval data on a two-dimensional plane, A mental activity determination device characterized by determining a mental activity based on a coordinate change pattern.