JP2003325467A - Intention communication assisting device and software for assisting intention communication - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device or the like for intention communication based on the change in brain wave relevant to an alpha wave. <P>SOLUTION: Brain wave data measured by an electrode part 1 provided on the head part are input into an artificial neural network (ANN) model software operated by a control device 7 after the brain wave data are pre-treated by a pre-treatment method (PTM). An ANN model is acquired by learning data on event-related desynchronization in advance. The software performs appropriate output functions after correcting a frequency due to aging (equal to or more than 60 years old), if necessary. Output can be confirmed by an amplification of a power value of a beta wave. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication support device and communication support software.

[0002]

2. Description of the Related Art After the war, the number of vehicles increased dramatically, and along with that, the number of traffic accidents also increased. The number of traffic injuries and casualties has increased by about 30% in the last decade, reaching almost 1 million casualties per year. Due to such traffic accidents and diseases such as muscular dystrophy and amyotrophic lateral sclerosis, many people lose their motor functions and language functions. Among them, it is almost impossible for a person with severe motor dysfunction to communicate with a nurse, and thus the nurse must carefully care.

Further, in Japan, the birth rate is declining and the life expectancy is increasing, so that a rapidly aging society is approaching. Therefore, in the future, the increase in the number of nurses will catch up with the increase in the number of nurses. It is thought to disappear. Various electric assisting devices have been developed because a supporting device using a computer is required for the purpose of reducing the labor of nursing.

In this regard, in recent years, research called functional electric stimulation (FES), that is, research for reconstructing the motor function lost by electrical stimulation has been underway. Currently, the interface, that is, the interface between a person with a disability and a computer, uses the remaining few functions such as voice, eye movements, and breathing. For example, if the brain does not receive the signal from the brain because the spinal cord is damaged due to a traffic accident, etc., and the desired movement cannot be performed, control commands such as voice and eye movements are input to the FES system and remain. By applying electrical stimulation from the FES system to peripheral nerves, it becomes possible to perform the desired movement. Normally, voices, eye movements, movements of remaining operable parts, and the like are used as control commands, but a severely paralyzed person requires much load and time even for such movements. However, even in these severely paralyzed persons, the brain is presumed to be functioning normally, so if the brain waves can be used as an interface with a computer, it may be a useful method. Therefore, the present inventors have paid attention to the electroencephalogram as a new interface that can be used even more easily by persons with severe disabilities.

[0005]

A computer interface using an electroencephalogram is called a brain-computer interface (Brain Computer Interface: BCI). B
CI is a communication system that discriminates various EEG patterns and uses the results to generate a predetermined control signal. It is said that the electroencephalogram pattern caused in the motor cortex changes depending on the preparation and execution of actions, or imagination of actions.

An example of an intention communication device to which BCI is applied is disclosed in Japanese Patent Laid-Open No. 7-20774. In the device disclosed in this publication, β waves (20 Hz to 40 H
Based on the change of z), it tries to convey the intention of the subject through a computer. However, β
Waves have been difficult to use stably because of the large variability between days in multiple humans and in the same person. The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a device or the like that communicates based on changes in brain waves corresponding to α waves.

[0007]

[Means for Solving the Problems, Actions of the Invention, and Effects of the Invention] As a result of diligent studies, the present inventors have measured brain waves in both the left and right hemispheres and evaluated event-related desynchronization of the brain wave data. As a result, they have found that information can be transmitted, and basically completed the present invention.

The "electroencephalogram" is a voluntary change in electric potential from the brain, which is an electroencephalogram or EEG.
Also called. To measure an electroencephalogram, an electrode is fixed to a predetermined site on the scalp with conductive glue or the like, and a weak potential fluctuation generated from the brain is amplified by an electroencephalograph amplifier and continuously recorded. To derive the electroencephalogram, there are two methods: the reference derivation method that measures the potential difference (electroencephalogram) with the electrode provided on the head using the part of the earlobe and other parts away from the brain as the reference for the electric potential. There is a bipolar derivation method that measures the potential difference of the electrode opening. In general, an electroencephalogram includes an element generated in a limited area of the brain and an element similarly recorded from a relatively wide range. In the bipolar derivation method, when the two electrodes have a small electrode spacing, the same recordings from a wide range of the brain are canceled out and hardly appear in the electroencephalogram recording. On the other hand, potential fluctuations that are localized in the vicinity of the area where the two electrodes are placed and that appear closer to one of the electrodes are subtracted from the portion common to both electrodes. Often recorded more clearly than in law. In the present invention, the number of electrodes to be mounted on the scalp and the positions thereof are not limited, but, for example, the ten-twenty electrode syste
m) can be used. In this case, the recommended electrode positions are left and right somatosensory areas (between C3 and P3, C4
And P4), T5, T6, F3, F4, P3, P4, F
There are a total of 10 channels of 7 and F8, and 6 channels of left and right somatosensory areas, P3, P4, T5 and T6 are preferable.

In general, evoked potentials (E)
P) refers to transient voltage fluctuations recorded at various sites (spinal cord, brain stem, cerebrum, etc.) from the input of various sensory stimuli to receptors until reaching the cerebral cortex,
This is also called an evoked response. The evoked potentials include cerebral evoked potentials such as visual, auditory and somatosensory, brain stem evoked potentials, spinal cord evoked potentials, and the like. This evoked potential is not only a passive electrical response of the nervous system to sensory stimuli, but also a component that changes due to psychological activities such as attention, recognition, problem solving, voluntary movements related to sensory stimuli ( P300, concomitant negative variation, and exercise-related potential) have been revealed, and these are called event related potential (ERP). P300 is a stimulus that contains useful information for the subject,
A so-called task-related stimulus is given at a low frequency for the target stimulus and a high frequency for the non-target stimulus, and only in the case of the target stimulus, the potential evoked at that time, 300 ms after the stimulus
It is a large positive potential that appears in the vicinity.

The present invention utilizes one of the event-related potentials related to the cognitive processing process called "event related desynchronization (hereinafter, ERD)". This is a phenomenon in which brain waves change in the sensorimotor cortex by preparation for movement or imagination of movement.
As an actual electroencephalogram change, for example, in the motion of bending and stretching the right elbow, the μ wave (7 to 13 Hz, which is a little faster than the α wave, which is slightly faster than the α wave) of the brain hemisphere which is the opposite side of the hand to move before the motion μ-shaped wave) and β wave (when you are nervous, etc. 13
The amplitude of fast waves (~ 30Hz) decreases. ERD is said to occur in the somatosensory and motor areas of the brain. According to the study by the present inventors, a frequency band of 7 Hz to 25 Hz, preferably 7 Hz to 20 Hz
The fluctuations in the frequency band of Hz, more preferably in the frequency band of 10 Hz to 19 Hz are evaluated. In addition, in elderly people in their 60s and above, the frequency of α-wave shifts to the slow side, so 8Hz-9
It is preferable to appropriately correct and use the electroencephalogram in the frequency band of Hz (for example, using the equations (4) and (5) described later).

"Information" specifically means that 1-bit (for example, 1/0, Yes / No, or right / left discrimination) data can be transmitted in one trial. By repeating the trial, complex information can be assembled. "Evaluate" means to calculate and evaluate event-related desynchronization using a computer, and to output an appropriate output. As a computer calculation method, for example, an artificial neural network (AN
N), hidden Markov model (HMM), self-organizing model (SOM), time-delayed NN (TDNN), self-remembering NN, vector quantization model, etc. can be used. Of these, ANN is preferred. ANN is a mathematical model of a neural network in the brain of a living body, and the most basic one is the perceptron. The perceptron was able to add a learning function to the computer, but its problem was pointed out because it was limited to linear separable problems. In 1986, Roomheart (Rumelhart) et al. Proposed the BP method (backpropagation) that extends the learning rule of the perceptron to a multilayer network.
The research on s has become popular. ANN has many excellent features such as parallel processing, learning function, and self-organizing ability. It is also capable of associative memory, and is good at information processing such as pattern recognition and comprehensive judgment.

In general, an NN (neural network) has a network structure in which neurons that perform signal conversion in a non-linear or linear manner are used as units, and its structure is a recurrent ANN in which neurons are connected to each other and a signal having a hierarchical structure. Can be divided into multi-layered NNs in which the flow is in one direction. It is said that the recurrent NN is suitable for processing time series data, and the multi-layer NN is suitable for pattern recognition. The hierarchical NN is configured to form multiple layers by using a large number of such units, and is a nonlinear multi-input multi-output system as a whole.

The BP method can automatically identify the nonlinear input / output relation given to such a network by changing the coupling weight. This operation is called learning, and the desired human output relationship is finally realized by repeating this learning many times. Using this BP method,
By learning, NN can automatically acquire knowledge.

To input EEG data into the ANN model,
It is preferable to process the data using, for example, the Fast Fourier Transform (FFT). The famous principle of the Fourier series, "Any periodic function can be represented by the sum of series of sine and cosine waves", was announced by French math and physicist Fourier. The Fourier transform is a combination of a sine wave and a cosine wave with a fundamental frequency that is the frequency of the period and a harmonic frequency that is an integral multiple of that frequency, even if the waveform looks irregular at first glance. Based on the principle of the Fourier series, which is defined by, the ratio of each component is calculated. By obtaining this “ratio of each component (referred to as Fourier coefficient)”, it is possible to find the “hidden feature” of a seemingly irregular and elusive waveform. The amplitude of each frequency component can be obtained from the value of the Fourier coefficient obtained for each frequency by Fourier transforming the wave, and the value obtained by squaring the amplitude is called power. The change of the Fourier coefficient with respect to the change of frequency is called the spectrum, the characteristic of the amplitude with respect to the frequency is the amplitude spectrum of the periodic function, the square of the amplitude is the power, and the one showing the change of the power with respect to the frequency is the power spectrum. . Any of these data may be used,
Most preferably, a power spectrum is used. Furthermore, AN
It is preferable that the data input to N is subjected to an appropriate pretreatment. As such a pretreatment method, for example, PTM1 to PTM3 described later can be used. It is also preferable to enter the age in the ANN. As described above, since the electroencephalogram data analyzed by the present invention (particularly, the frequency data corresponding to α waves) changes depending on the age of the subject, it is preferable to analyze the changes in order to obtain good results. This is because it is preferable.

Further, the change of β wave can be used as an auxiliary for the transmission of information. That is, when the output result of the computer is incorrect, the recognition error can be notified by increasing the β wave power value. Traditionally,
It has been said that it is difficult to use the electroencephalogram data as a means for transmitting information because of the great variation between days and people. However, according to the present invention, it becomes possible to convey information by evaluating the pattern of event-related desynchronization (ERD) of the electroencephalogram of the subject by a computer. As a result, even a person who has lost most of his / her motor function due to a serious accident or illness can communicate.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Next, one embodiment of the present invention will be described in detail with reference to the drawings, but the technical scope of the present invention is not limited to the following embodiments, and The present invention can be implemented with various modifications without changing the gist. Moreover, the technical scope of the present invention extends to an equivalent range.

<Example 1> 1. Test method 1. Electroencephalogram collection method Electroencephalography was performed on two healthy male subjects (subject A: 25 years old, and subject B: 23 years old). The electroencephalogram was collected when the subject lay on his / her back in the shielded room and had his eyes rested. The time schedule for collecting EEG is shown in FIG. As shown in the figure, two seconds after the start of the experiment, the test subject hears the voice instruction “left” or “right” from the examiner. After listening to the voice message, the subject makes a motion image of bending and stretching the instructed arm for 3 seconds, and then actually bends and extends by the instruction of "yes".

The electroencephalogram data for 2 seconds from the start of the experiment was used as the reference data, and the data for 3 seconds during the motion image was used as the analysis data. ERD is a phenomenon in which an electroencephalogram changes depending on the preparation for an action or an image of the action.
Therefore, for analysis, we did not use the data during operation, but used the data at the time of imaging. The electroencephalogram was collected by partially changing the electrode arrangement based on the International 10-20 electrode system. The electrode arrangement diagram is shown in FIG. The reference electrode is Cz, and the electroencephalogram is generated on a total of 10 channels of left and right somatosensory areas (existing between C3, C4 and P3, P4), T5, T6, F3, F4, P3, P4, F7, and F8. Recorded.

For subject A, 44 data collected on the same day were set as the first set (DS1), and 44 data collected on another day were set as the second set (DS2). For subject B, 44 sets of data were collected from the third set (DS3). In addition, the data set of subject A, DS1 and D
A fourth set (DS4) was obtained by combining with S2.

2. Processing of EEG data Frequency analysis was performed on EEG data at a sampling interval of 10.0 [ms] and a sampling time of 2.0 [s] using the fast Fourier transform (FFT). In this research, we analyzed using the power spectrum value every 1Hz. In this analysis, almost 1Hz
Data analysis can be performed in the frequency range of 50 Hz. Here, considering the wave of 1 Hz to 6 Hz corresponding to the movement of the eye and the wave of 25 Hz to 31 Hz corresponding to the electromyogram, 7H is used for data analysis.
I decided to use the frequency band of z ~ 25Hz. Furthermore, 1Hz ~ 6H
If the power spectrum value of z and the frequency component of 25 Hz to 31 Hz exceeds a certain threshold value, it is considered that noise is mixed in other frequency bands as well, so such data is not used. I decided.

Specifically, the power value (R) for 2 seconds in the motion image before actually bending or stretching the right elbow or the left elbow and the reference power value for 2 seconds which is considered to be unrelated to the imagination of motion ( The R / P value, which is the ratio with P), was used. The electroencephalogram data was Fourier transformed to obtain a power spectrum for each 1 Hz.
This power spectrum value was subjected to three kinds of pretreatment methods (PTM). Table 1 shows each pretreatment method.

[0022]

[Table 1]

In the table, R is the power spectrum value in the analysis section, P is the power spectrum value in the reference section, and the subscripts r and l represent the right hemisphere or the left hemisphere. The electroencephalogram data was subjected to fast Fourier transform to obtain a power spectrum value. In the pretreatment method 1 (PTM1), the rate of change of the power spectrum value was used. That is, for each frequency of each channel, R and P
The ratio (R / P) was input to the ANN as an input value. Pretreatment method 2
In (PTM2), the change pattern is used instead of the change rate value.
Used as input for ANN. That is, for each frequency in each channel, if R> P, set the input value to 1 and R <
In the case of P, the input value was set to 0. The input value was set to 1 when Rr> Rl, and the input value was set to 0 when Rr <Rl.

Further, in the preprocessing method 3 (PTM3), the idea of PTM2 is further advanced, and the patterns of the left and right magnitudes of the rate of change of the power spectrum are input. That is,
For each frequency of each channel, if the ratio of R and P in the right hemisphere (Rr / Pr) is greater than the ratio of R and P in the left hemisphere (Rl / Pl), set the input value to 1, In the opposite case, the input value was set to 0.

3. ANN structure ANN is of a hierarchical type, and has three layers: an input layer (Input Layer), an intermediate layer (Hidden Layer), and an output layer (Output Layer).
It is a layered structure (see FIG. 3). As the learning method of ANN, a sequential correction method in which inertia is introduced into a generally widely used BP (back progation) algorithm is used. When the processed electroencephalogram data is input to the ANN, 0 is output in the left motion and 1 is output in the right motion. To construct the model, Ns of N DSs were used as ANN learning data, and the remaining Nes were used as evaluation data. In this study, the ratio of Ns and Ne is
It was set to about 2: 1. The input value (Iij) and the teacher value (Ti) of the ANN are standardized to be a value between 0.1 and 0.9 between the maximum value and the minimum value in N DS.

Those having an absolute error of 0.3 or more with respect to the learning data were counted, and this number was used as the evaluation index J. In order to optimize the network structure, the number of intermediate layer units was varied from 1 to the maximum number of intermediate layer units (the number of input layer units + 10), and the structure with the smallest J was selected.

Since there is a possibility that a correct model cannot be constructed if there are redundant input variables, the variable increase method (PIM) was used as a method for selecting the optimum input variable. In this method, first select one input variable that minimizes J, and
As a step, next, one input variable is newly added to this input variable, and the input variable that minimizes J is selected. By this procedure, the input variables are sequentially increased and repeated until the evaluation index J does not decrease even if a new input variable is added.

4. Evaluation Method of Motion Discrimination Ability As described above, the ANN is trained to output a value of 1 for right motion and 0 for left motion. Therefore, when data is input to ANN, the output corresponding to it is 0. .4
In the following cases, the left side is recognized, in the case of 0.6 or more, the right side is recognized, and when a value between 0.4 and 0.6 is output, it is not recognized. At this time, the learned ability of the ANN was expressed by the recognition rate (Equation 1) and the correct answer rate (Equation 2).

[0029]   Recognition rate = (the number of data whose output is 0.4 or less + the number of data whose output is 0.6 or more) / ( All test times) Formula (1)   Correct answer rate = (number of times "right" or "left" was correctly recognized) / (output is 0.4 or less Number of data + the number of data whose output is 0.6 or more) Formula (2)

Two. Test results and discussion 1. A model using the data obtained by processing the EEG from the somatosensory cortex with PTM1 was constructed using the data obtained by processing the EEG from the two channels of the left and right somatosensory cortex with PTM1. (1) Discrimination between right and left in subject A's model DS1 collected on different days by subject A (learning data 30
Individual, 14 for evaluation) and DS2 (30 data for learning, for evaluation)
14 pieces) were processed by PTM1 (38 input items), an ANN model was created with each data, and it was determined which of the left and right elbows was bending or stretching. The results are shown in Table 2.

[0031]

[Table 2]

Looking at the left and right discrimination results in the ANN model constructed from the waveform data collected by the same subject on the same day, although the recognition rate for the DS2 learning data was 93.3%, the overall recognition rate was And the legitimacy rate is very high,
The correct answer rate was 100% for the learning data and the evaluation data, and it was possible to completely discriminate between the left and right.

(2) Effect of EEG changes due to days EEG changes greatly with time, and can a model created from data on one day show a good correct answer rate for data collected on another day? I didn't know so I confirmed it.
In subject A, two data sets DS1 and DS2 collected on different days were used. That is, using DS1 as learning data and DS2 as evaluation data, the recognition rate and correct answer rate in data on different days were confirmed. The results are shown in Table 3.

[0034]

[Table 3]

Although the correct answer rate was as high as 89.8%,
The recognition rate was not high enough at 63.6%, and the model created from the data collected on one day was not sufficient to distinguish the data on another day. From this, it was found that the day-to-day variation of the electroencephalogram is large and it is difficult to discriminate with the same ANN model, and it is necessary to learn the ANN again.

(3) Effects of changes in EEG between different subjects It is known that EEG changes significantly from person to person even in a healthy person. Therefore, according to conventional research, the ANN model is applied to each person. Is required. For this reason,
If one ANN model is
If both the high recognition rate and the correct answer rate are shown, the model can be said to be a model with high generality. Then, using the ANN model learned from the data (DS4) of the subject A, the subject B
It was tried to determine whether the data (DS3) collected in 1. could be distinguished.
The results are shown in Table 4.

[0037]

[Table 4]

Although the correct answer rate was as high as 94.3%, the recognition rate was very low as 38.6%, and it was not possible to sufficiently determine which of the left and right arms was bending and stretching. From this result, it was found that it is difficult for the ANN model created by one person's data to discriminate another person's data. Previous studies (eg G. Pfurtscheller et al .; "On-line EEG cl
assificationduring externally-paced hand movements
using a neural network-based classifier ", Electro
enceph. Clin. Neurophysiol, vol.99, pp.416-425, 19
Also in 96), the EEG changes during the motion image are
Since it varies considerably from person to person, we make a distinction by creating a model for each individual.

From the above results, a high recognition rate and correct answer rate could be obtained by using ERD to create an ANN model suitable for each individual each time. However,
When PTM1 was used to create an ANN model using electroencephalograms from the somatosensory area, it was considered difficult to create an ANN model with high generality. Therefore, we tried to create a model with high generality by examining the electrode position and the pretreatment method.

2. Examination of electrode position and pretreatment method (1) PTM1 of 7Hz-25Hz collected by model 10 channel using data processed by PTM1 with data of 10 channel PTM1
Processed in. D1 for the ANN model learned by subject A's DS1 (30 learning data, 14 evaluation data)
By inputting S2 (44 pieces) as the evaluation data, it was confirmed how much discrimination can be performed on the same person on different days. In addition, the subject A who learned in DS4
With respect to the NN model, DS3 collected by another subject B was input as evaluation data, and it was confirmed to what extent the different human data could be discriminated. The respective results are shown in Table 5.

[0041]

[Table 5]

The recognition rates were 83.6% and 79.5%, respectively. The correct answer rate was 94.4% and 88.7%, respectively. The recognition rate was improved as compared with the discrimination result of the ANN model using the electroencephalogram obtained only from the somatosensory area.

(2) Examination of pretreatment method The data of 7 Hz to 25 Hz collected on 10 channels was used for PTM2 and PTM.
By using the ANN model that was processed in 3 and trained on the DS1 of the subject A to determine the DS2, it was confirmed how much the EEG data collected by the same person on different days could perform the determination. The recognition rate and the correct answer rate when each pretreatment method is used are shown in the upper part of Table 6.

[0044]

[Table 6]

In each pretreatment method, the correct answer rate was 94.3%. The recognition rate increased in the order of PTM1, PTM2, and PTM3, and the correct answer rate was 100% especially in PTM3. Using the ANN model of subject A trained in DS4, another subject B
By discriminating the DS3 collected in Section 3, it was confirmed how much discrimination can be performed by the electroencephalogram data of different persons. The recognition rate and the correct answer rate when each pretreatment method is used are shown in the lower side of Table 6. The recognition rate increased in the order of PTM1, PTM2, and PTM3, and the recognition rate in PTM2 and PTM3 was 85% or more. Also,
The correct answer rate was higher in PTM2 and PTM3 than in PTM1.
The recognition rate for PTM2 and PTM3 was over 90%.

Since the input value processed by PTM2 and PTM3 is not the power spectrum value of each frequency but the pattern value of the magnitude of the power spectrum, it becomes irrelevant to the power spectrum value of the electroencephalogram. It was thought that they were less affected.

(3) Examination of input items The results of discrimination by the data processed by PTM2 and PTM3 were very good, but there were a large number of input variables, 285 and 95, respectively. , 340 hours and 72 hours are required for the calculation by the variable increase method, respectively. In order to improve this situation, we examined the input items with the aim of reducing the number of input variable candidates. Therefore, when learning the ANN model by each preprocessing method, the input items selected by the variable increase method are summarized, and the distribution chart of the selected items is shown in FIG. Looking at Figure 4, the frequency band 10Hz ~ 19Hz
Is highly selected. Also, in the shaded portion, that is, in the left and right somatosensory cortex, P3, P4, T5, T6 channels, the frequency range of 10 Hz to 19 Hz is highly selected,
About 70% of all selected data were included. Therefore, as input items, left and right somatosensory areas, P3, P4, T5, T6
It was decided to be 10Hz ~ 19Hz in 6 channels of. As a result, the number of input items processed by PTM2 and PTM3 was 90 and 30, respectively, and the learning time of the ANN model could be greatly shortened.
Table 7 shows the results of analysis by the ANN model performed by the variable increasing method using these items.

[0048]

[Table 7]

Compared to the result (Table 6) of discriminating 10 channels of 7 Hz to 25 Hz data as input items, the correct answer rate may be lower, but the overall recognition rate is 90%.
As described above, it was found that sufficiently high discrimination accuracy can be obtained. From this, it seems that the six channels of left and right somatosensory cortex, P3, P4, T5, and T6 are sufficient as channels for acquiring EEG data. Although the recognition rate and correct answer rate of PTM3 are slightly lower than those of PTM2,
The recognition rate and correct answer rate are 90% or more, which is almost the same accuracy as PTM2. From these results, as input variables, 10Hz-19 of 6 channels of left and right somatosensory cortex, P3, P4, T5, T6
It was judged that the value obtained by processing the power spectrum value of Hz with PTM3 is sufficient.

Example 2 Next, it was examined whether or not the electroencephalogram data could be processed in many subjects including persons with spinal cord injury. one. Test method 1. Eight healthy males A, B, C, D, E,
F, G and H (Age of each subject is 25 years old, 2 respectively
They were 3, 34, 40, 56, 60, 72 and 72 years old. ) Three healthy women I, J and K (ages of each subject were 23, 20 and 20 years old) and 3 patients L, M and N with spinal cord injury.
(Age of each patient was 17, 25, and 46 years old.) A total of 14 people were selected. EEG data collection is
The time schedule was the same as in Example 1 (see FIG. 1).

The electrodes were installed according to the international 10-20 system. Recording was performed using a total of 6 channels of the left and right somatosensory areas, T5, T6, P3, and P4, using the linked bimastoid of both mastoids as a reference electrode (see FIG. 2). The electroencephalogram data of 45 image movements of the elbow was collected for all the subjects on the same day.

2. Data processing The data processing, the data pre-processing method, and the ANN model were performed in the same manner based on the result of the above-described first embodiment, and therefore, are omitted to avoid redundant description. 3. Recognition Confirmation Using β-Waves In order to avoid erroneous left-right recognition, we examined the process of confirming recognition results. That is, the subject was trained to increase the power of the β-wave to signal that it was not accurate if the recognition results were different.

As training for that purpose, two kinds of mental labor were used. That is, (A) mental arithmetic and (B) association. Among them, (A) mental arithmetic is composed of the following problems. 1. Calculation of arithmetic progression. 2. Repeat the subtraction of 1000 to 7. 3. Calculation of geometric series with common ratio 2. The (B) association is composed of the following problems. 1. Shiritori. 2. Says the names of animals, plants, vegetables, and colors. 3. Impress yourself with the image of disliked people or unpleasant things.

Two. Test Results and Discussion The ANN model was constructed using the somatosensory cortex, P3-P4, and T5.
-EEG data from 6 channels of T6 was used. 10Hz-19
The EEG data in the frequency band of Hz was processed by PTM3 and then input to the ANN model. First, the ANN model was trained by the data of each subject. Table 8 shows the results of evaluating other subject data using the ANN model after learning.

[0055]

[Table 8]

In Table 8, as typical subjects, B,
When the F, H, K, and M data were used for ANN model learning, the recognition rate and the correct answer rate when other subject data were evaluated were shown (Note that this table shows the other subject data at the same time. Is used for ANN model training, B, F,
It can also be read as showing the recognition result when evaluating the H, K and M data). In healthy subjects younger than 50 years (ie A, B, C, D, I, J and K), the recognition rate was
It was 86.7% or more, and the correct answer rate was 83.3% or more.

Most of the recognition rates between patients (L, M and N) and young healthy subjects were 84.4% or more, and the correct answer rate was 8
It was 1.5% or more. The precision was slightly lower in 3 patients. Also, subjects aged 50 and over (ie E, F,
Using the data of G and H) for learning or evaluation,
The recognition rate was 75.5% or higher, and the correct answer rate was 70.7% or higher. These results suggest that EEG data is affected by the age of the subjects.

1. Testing the effects of aging on EEG data A comparison between EEG data of the elderly and EEG data of young adults has been previously pointed out that the frequency of α waves is lower in the former than in the latter. The frequency of the α wave is
They are about 10.8 Hz, about 9.0 Hz, about 9.0 Hz, and about 8.0 Hz, respectively. That is, due to aging, the frequency of the α wave is slowed down. Power values of 10Hz-19Hz were used in the above analysis, but this range may be a good range for young adults. However, for older people, 8.0 Hz and 9.0
The Hz data seems to be important. Therefore, the present inventors attempted correction using the power value of the low frequency by using the following formula (3).

[0059]

[Equation 1]

In the equation (3), P _i represents a power value at each frequency, and D _F represents a frequency distribution ratio, that is, a function D _F = f (F, N) of the frequency F and the age N. Report of Okuma (Teruo Okuma: Clinical EEG, Medical Institute, Tokyo, 199)
Based on 1), the distribution ratio was determined as shown in FIG. (1) Construction of frequency distribution ratio function The distribution ratio function has an asymmetric double S as shown in equation (4).
Estimated as a character function.

[Equation 2]

In equation (4), u, ν, w1, w2 and w
3 is a parameter. Table 9 shows the estimated parameters in the equation (4).

[0062]

[Table 9]

(2) Parameter (u, ν, w1, w2, w3) and age
Estimating the function between (N) and the parameters shown in Table 9 and age were regression analyzed to determine the coefficient of the function f in the following equation (5). f (u, ν, w1, w2, w3) = aN ² + bN + c Formula (5) The results are shown in Table 10.

[0064]

[Table 10]

In each subject, the distribution ratio of 8 Hz to 13 Hz was calculated, and after correcting the value of α wave based on the ratio,
The corrected power value was input to the ANN again. Table 11 shows
ANN model was made to learn each subject data, and the result when other subject data was evaluated was shown (however, Table 11
Indicates the percentage of correct answers when the data of B, F, H, K, and M are used for learning in order to correspond to Table 8.

[0066]

[Table 11]

As shown in this table, the elderly (E, F, G and
Even when the data of H) were used for learning or evaluation, both the recognition rate and the correct answer rate were as high as 90% or more. From this, it was clarified that the change including the frequency information on the α wave works well to correct the power value, and the influence of aging can be reduced by the corrected power value. In order to further improve the correct answer rate, all the data obtained from the subjects A, D, H, I and M are used for learning, and the recognition rate and the correct answer rate when the data of other subjects are evaluated are shown in Table 12. Indicated.

[0068]

[Table 12]

The recognition rate and the correct answer rate were almost 100%. Since the ANN model based on the electroencephalogram data obtained from a plurality of people contains a lot of information about the features of the electroencephalogram data, it is considered that a high correct answer rate was obtained. In this embodiment, the frequency distribution is corrected according to the age of the subject before the electroencephalogram data is input to the artificial neural network (ANN). However, as another embodiment of the present invention, the brain wave data regarding the subjects of various ages is input to the ANN together with the ages of the subjects in advance, and the learning is performed, thereby changing the ages of the subjects as in the above embodiment. It is also possible to make a determination according to age without performing frequency distribution correction.

2. Confirmation of recognition using β-wave In this research, β-wave was used to confirm the recognition result. During mental work, β-wave power values increased around the Pz site, mainly at C3 and C4. Table 13 shows 14
The averaged percentage increase in C3, C4, P3 and P4 is shown for the power values in the frequency band Hz-20Hz.

[0071]

[Table 13]

The β-wave power value remarkably increased when the associative problem was performed, compared with when the mental arithmetic problem was performed. In particular, the increase rate showed a high value of 50% or more in each of (1) Shiritori, (4) plants, and (6) vegetables. As a result, it was found that the increase in the β-wave power value related to the associative problem (1), (4), or (6) can be used for confirmation. That is, if the output result of the ANN model is incorrect, the test subject can remember the plant name or vegetable name by shiritori and notify the recognition error through the increase of the β wave power value. For example, if the average percentage increase in C3, C4, P3 and P4 is greater than 50%, it can be determined that the output result of the ANN model is wrong.
Thus, the β wave is used to confirm that the recognition of the ANN model is incorrect, and the final correct answer rate D _c is given by the following equation (6).

D _c = D _rate + p (100 − D _rate ) (%) Expression (6) In the expression, D _rate is the correct answer rate before confirmation by β waves, and p
Is the rate of increase in the correct answer rate when confirmation is made by increasing β waves. By this procedure, the correct answer rate was significantly improved, and the legitimacy of the system ability was secured.

We confirmed the ANN model by using the corrected power value as an input value and increasing the β wave. By this measure, the correct answer rate could be increased to almost 100% in both healthy subjects and patients with spinal cord injury.

<Communication Supporting Device> Next, an example of a communication supporting device for carrying out the present invention will be described with reference to FIG. Reference numeral 1 is an electrode unit attached to the head for detecting a current generated in the head. The position where the electrode portion 1 is attached is, for example, the left and right somatosensory areas (C3 and P3).
Located between C4 and P4), T5, T6, F3, F4, P
A total of 10 channels of 3, P4, F7, and F8 (or 6 channels of left and right somatosensory areas, P3, P4, T5, and T6) are exemplified (see FIG. 2).

Reference numeral 2 is an amplifier for amplifying the weak current captured by the electrode section 1. The electrode unit 1 and the amplifier 2 are electrically connected. 3 is for converting an analog signal into a digital signal for improving the accuracy of frequency peak component extraction A
It is a / D converter. Reference numeral 4 is a frequency band to be used, mainly a frequency band filter of 7 Hz to 25 Hz. By passing through the filter 4, the data of the frequency band required for the ANN analysis is extracted from the electroencephalogram measured by the electrode unit 1.

Reference numeral 5 is a circuit for integrating the peak current.
An output amplifier 6 amplifies the signal of 5 into a stable signal. Reference numeral 7 denotes a control device (computer), which is not shown in the drawing, but is internally provided with a ROM, a RAM, a storage device and the like. Further, the control device 7 can operate the ANN and software described later. An output device 8 outputs the electroencephalogram data processed by the ANN model by the operation of the control device 7. Specific examples of the output device 8 include a display and audio output.

<Software> Next, an example of software for implementing the present invention will be described with reference to FIG. First, the electroencephalogram data is collected (S100), and the electroencephalogram data is subjected to fast Fourier transform (FFT) (S110). Next, it is determined whether or not the subject who collected the electroencephalogram data is 60 years old or older (S120), and if YES, the frequency distribution is corrected (S130), and then the process proceeds to step 140. Meanwhile, step 1
If the subject is under the age of 60 at 20, the process proceeds directly to step 140.

In step 140 (S140), the data is preprocessed (for example, PTM3 can be used). next,
The pre-processed data is input to the ANN (S150), and confirmation is performed using β waves (S160). Note that the ANN is preliminarily trained using the training data. Here, if the confirmation is affirmative, an output from the ANN is performed (S170), and the routine ends. On the other hand, if the confirmation by the β wave is negative, the routine is finished as it is. When a plurality of signals are input, the above steps 100 to 170 can be sequentially repeated.

As described above, according to this embodiment, it is possible to communicate with a patient suffering from amyotrophic lateral sclerosis (ALS). For example, the patient can control the movement of the cursor on the video screen. The patient can then type in the appropriate letter or select the appropriate comment from the video screen. The system is also useful as a prosthetic hand control method for patients with severe movement disorders. Similarly, this system can be introduced into studies that reconstruct lost motor function, functional electrical stimulation (FES).

[Brief description of drawings]

FIG. 1 is a timing chart when collecting brain wave data.

FIG. 2 is a diagram showing electrode positions when viewed from the top of the head.

FIG. 3 is a diagram showing an outline of an ANN model.

FIG. 4 is a diagram showing a distribution of selected items by collecting input items selected by the variable increasing method when learning an ANN model by the preprocessing method.

FIG. 5 is a graph for correcting the frequency distribution of α waves according to age.

FIG. 6 is a block diagram of a communication support device according to the present embodiment.

FIG. 7 is a flowchart of software in the present embodiment.

Continued front page    (72) Inventor Taizo Hanai             4-24-9 Hasumatsu Higashi-ku, Fukuoka City, Fukuoka Prefecture Flow             Lens Kaizuka 104 (72) Inventor Shin Hibino             4-1101, Ikeba, Tenpaku-ku, Nagoya-shi, Aichi             Stone Heights Ishiyakushi 40B (72) Inventor Tatsuaki Shirataki             4-503 Takinomizu, Midori Ward, Nagoya City, Aichi Prefecture             Ntion Takinomizu 311 F term (reference) 4C027 AA03 CC00 DD07 FF01 FF02                       FF05 GG01 GG11 KK03

Claims (10)

[Claims] 1. An electroencephalogram measuring device capable of measuring electroencephalograms of the left and right hemispheres, and a computer for evaluating event-related desynchronization of electroencephalographic data measured by the electroencephalogram measuring device and outputting information. Communication support device. 2. The communication support apparatus according to claim 1, wherein the evaluation by the computer uses an artificial neural network (ANN). 3. The communication support device according to claim 2, wherein the subject's age is also input to the ANN. 4. The communication support device according to claim 1, wherein the frequency distribution of the electroencephalogram data is corrected according to the age of the subject. 5. The communication support device according to claim 1, wherein the output of the information by the computer is confirmed by the change of the β wave. 6. EEG data measured in both the left and right hemispheres,
Software for inputting to an artificial neural network (ANN) learned by electroencephalogram data relating to event-related desynchronization and performing output corresponding to the electroencephalogram data. 7. The electroencephalogram data is subjected to a pre-processing after being converted by using a fast Fourier transform and then compared with reference data before event-related desynchronization and analysis data after event-related desynchronization. 7. The software according to claim 6, wherein the software is input to the ANN after being processed. 8. The software according to claim 6, wherein the age of the subject is also input to the ANN in addition to the electroencephalogram data. 9. The software according to claim 6, wherein the frequency distribution of the electroencephalogram data is corrected according to the age of the subject. 10. The output corresponding to the electroencephalogram data is β
7. The method according to claim 6, wherein the change is confirmed by a change in the wave.
~ Software according to any one of claims 9 to 10.