JP2011076177A - Method and device for controlling equipment using brain wave induced by contact of teeth - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an equipment control device for controlling a control object by highly precisely estimating the movement of teeth on the basis of a brain wave induced by the contact of the teeth obtained by non-invasion from the head of a user without any special training. <P>SOLUTION: The equipment control device is configured to estimate the movement of the teeth of a user from the brain waves of a plurality of channels induced by the movement of the teeth of the user obtained at a plurality of places on the head skin of the user, and to generate a command to control the object equipment according to the result of estimation, and provided with: a band pass filter for extracting the components of a prescribed frequency band from each of the brain waves of the plurality of channels; a featured value calculation part 152 for calculating the pattern of a predetermined featured values from the components of the frequency band of the brain waves; a learned pattern classifier 154 for, when the pattern of the featured value is given from the featured value calculation part 152, obtaining output showing the movement of the teeth of the user according to the featured value; and a command output part 156 for generating a command in response to the output of the pattern classifier 154, and for giving the command to the object equipment. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a BMI (Brain-Machine Interface), and more particularly, to a device control apparatus that can control a target object in real time in a non-invasive manner and with a signal from a brain without imposing an excessive burden on a user.

  Conventionally, as a technique for measuring an electroencephalogram with high accuracy, a technique for embedding an electrode in the head is known. For example, Patent Document 1 discloses an electrode structure for this purpose. However, such an invasive technique has a heavy burden on the subject, and the applicable scenes are particularly limited for humans.

  On the other hand, there is a method for detecting electrical changes in the brain accompanying brain activity from the scalp without embedding electrodes in the head. Such a technique is called a non-invasive technique. The non-invasive technique is characterized by a wide range of application because it does not place a heavy burden on the subject.

  In particular, BMI, which controls machines and determines the psychological state of human beings by non-invasively measured human brain waves, has recently attracted attention and has made great technological progress. In the background of this advancement, the increase in information processing ability due to the development of modern computer technology, the development of machine learning theory based on mathematical science, and the stable detection of specific information due to the development of signal processing technology play a major role.

  As phenomena promising in non-invasive BMI, there are phenomena called ERD (Event Related Desynchronization) and ERS (Event Related Synchronization). ERD means that specific frequency components such as mu waves (8-13 Hz) and beta waves (14-30 Hz) in an electroencephalogram decrease when a certain person is preparing for a certain movement. ERS means that a specific frequency component increases after a certain person moves.

  ERD, ERS, etc. reflect brain activity related to exercise. It is known that an electroencephalogram performs the same activity as when actually exercising not only when actually exercising, but also when imagining exercise. Therefore, it may be possible to control a robot arm, a mouse cursor (pointer) of a computer, an electric wheelchair, and various other devices by using an electroencephalogram when a user imagines a certain movement. Conceivable. Some experiments have already been carried out, and some success has been achieved. In this way, a technique using an electroencephalogram when an image of movement is used is called a movement image paradigm (Motor Image Paradigm).

  In the motion image paradigm, an electroencephalogram is measured non-invasively by providing an electrode outside the scalp in order to reduce the burden on the user. In this case, what is measured is not the state of firing of a specific nerve cell, but a movement of a certain portion of nerve cells. And since there is a skull between the nerve cell and the electrode, the signal measured by the electrode contains a lot of noise. For this reason, there is a problem that the accuracy is lowered as compared with an invasive method in which electrodes are arranged in the head. In order to overcome the problem of accuracy, many methods of reducing noise using a mathematical model are used. However, in the case of the motion image paradigm, when a user imagines motion, there is a problem that the signal detected by the electroencephalogram is not constant every time and the range of fluctuation is large. For this reason, the user needs to make the signal detected when performing each imaginary motion as constant as possible through repeated training (training) for a long period of time and to improve the accuracy. In this case, however, user training takes a long time and a lot of effort.

Special table 2009-527303 gazette

  In the case of the conventional exercise image paradigm, as described above, a long training time and a lot of effort are required for the user to obtain a certain degree of accuracy, and there is a problem that a burden is imposed. Moreover, even though such a long training time and a lot of effort are required, most users cannot give a certain signal when imagining exercise. Therefore, in the conventional motion image paradigm, users who can use BMI are greatly limited.

  Therefore, an object of the present invention is to control a control object with high accuracy based on a signal obtained from electrical fluctuations of a user's head without being subjected to special training. It is providing the apparatus control method and apparatus control apparatus which can be performed.

  The apparatus control apparatus which concerns on 1st aspect of this invention estimates the contact state of a user's tooth | gear from the electrical signal of the several channel each obtained in several places on a user's scalp, and estimated user A device control device for generating a command for controlling a target device in accordance with the contact state of a tooth, a filter means for extracting a component of a predetermined frequency band from each of the signals of the plurality of channels, and a filter When the feature amount calculation means for calculating a predetermined feature amount pattern from the components of the predetermined frequency band of the signals of the plurality of channels extracted by the means, and the feature amount pattern is given from the feature amount calculation means, In accordance with the feature amount, a pattern classifying unit that has been learned in advance so as to obtain an output indicating the contact state of the user's teeth, and in advance according to the output of the pattern classifying unit And a command generating means for providing the target device generates a command has been fit.

  The filter means extracts a component of a predetermined frequency band from each of a plurality of channel signals obtained by a noninvasive technique from a plurality of locations on the user's scalp. From these extracted components of a plurality of channels in a predetermined frequency band, the feature amount calculating means calculates a predetermined feature amount and gives it to the pattern classification means. The pattern classification means has been learned in advance, and provides an output indicating the contact state of the user's teeth to the command generation means in accordance with the given feature amount pattern. The command generation means generates a predetermined command according to the output of the pattern classification means and gives it to the device to be controlled.

  The device can be controlled using signals obtained from the head non-invasively. No special training is required. As a result, it is non-invasive and can control the control object with high accuracy based on the signal obtained from the electrical fluctuation of the user's head without overloading the user. An apparatus can be provided.

  Preferably, the pattern classification means is a pattern that has been learned in advance according to a feature amount pattern obtained from a user when the tooth contact state is maintained to be one of a plurality of predetermined states. Includes a classifier.

  By using the pattern classifier, the contact state of the user's teeth can be detected with high accuracy from a signal with low spatial resolution obtained by a non-invasive technique.

  More preferably, the pattern classifier includes a support vector machine (SVM).

  More preferably, the filter means includes a band-pass filter that extracts only an arbitrary band component within a band of 0.5 Hz or more and 500 Hz or less from each of the plurality of channel signals.

  More preferably, the bandpass filter includes a bandpass filter that extracts only a band component of 50 Hz or more and 250 Hz or less from each of the plurality of channel signals.

  A feature amount is obtained from an arbitrary band component within a band of 0.5 Hz to 500 Hz, preferably a band component of 50 Hz to 250 Hz, of a signal obtained from a plurality of channels. By using this feature amount, the contact state of the teeth can be estimated from the electroencephalogram with high accuracy. The electroencephalogram generated by tooth contact has a much larger amplitude than the surrounding noise, and a constant signal can be obtained from all users, so anyone can get high accuracy without special training .

  The multi-channel signal may be obtained from electrodes arranged on the front of the head on the user's scalp and on both sides of the head.

  When the user touches all of the upper and lower teeth, when not contacting any of the upper and lower teeth, when contacting only the upper and lower front teeth, when contacting only the right tooth, and only contacting the left tooth As described above, it was confirmed that the signals can be classified with high accuracy from the signals obtained from the front part of the head and the heads of both sides as described above.

  The feature amount calculating means includes a framing means for dividing the frequency band components of the signals of a plurality of channels into predetermined length frames, and a feature amount extracting means for extracting a predetermined first feature quantity from each frame of each channel. The first feature quantity extracted by the feature quantity extraction means is converted, and the feature quantity conversion means for maximizing the ratio of the variance of the feature quantities obtained from different states of the brain activity.

  Thus, by converting the feature quantity so that the variance of the feature quantity is maximized, the user's state can be efficiently classified from the signal obtained from the user's head.

  The device control method according to the second aspect of the present invention estimates a contact state of a user's teeth from signals of a plurality of channels respectively obtained at a plurality of locations on the user's scalp, and A device control method for generating a command for controlling a target device according to a contact state of a tooth, wherein electrodes are arranged at a plurality of locations on a user's scalp, and a signal of a plurality of channels for detecting a user's brain activity is received. A step of generating, a step of extracting a component of a predetermined frequency band from each of the signals of the plurality of channels, and a predetermined feature amount from a component of the predetermined frequency band of the signals of the plurality of channels extracted by the extracting step The step of calculating the pattern and the feature amount pattern are given in advance so that an output indicating the contact state of the user's teeth can be obtained according to the feature amount. The pattern classification means is provided with the pattern of the feature quantity obtained in the step of calculating the pattern and receives an output indicating the contact state of the user's teeth as the classification result, and in advance according to the output of the pattern classification means Generating a predetermined command and giving it to the target device.

1 is a diagram schematically showing a configuration of a robot controller system according to an embodiment of the present invention. FIG. It is a figure for demonstrating the characteristic of the amplitude of the time-sequential electrode signal obtained from the several places of the head when a user contacts the whole upper and lower teeth. It is a figure for demonstrating the characteristic of the amplitude of the time-sequential electrode signal obtained from the several location of the head when the user is not making contact | abutting on any of the upper and lower teeth. It is a figure for demonstrating the characteristic of the amplitude of the time-sequential electrode signal obtained from several places of the head when a user contacts only the tooth | gear (anterior tooth) of a center part among upper and lower teeth. . It is a figure for showing the characteristic of the amplitude of the time series electrode signal obtained from two or more places of a head when a user makes a right tooth contact among upper and lower teeth. It is a figure for showing the characteristic of the amplitude of the time-sequential electrode signal obtained from two or more places of a head when a user makes a left tooth contact among upper and lower teeth. It is a hardware block diagram of the robot controller 24 shown in FIG. It is a hardware block diagram of PC68 shown in FIG. It is a flowchart which shows the control structure of the main routine of the program for BMI implementation | achievement performed by CPU120 shown in FIG. It is a flowchart which shows the control structure of the program routine which implement | achieves the feature-value extraction process performed by step 152 of FIG. It is a flowchart which shows the control structure of the program for performing the learning of the pattern classifier used by step 154 of FIG.

  Hereinafter, how to implement the present invention will be described with reference to an embodiment. In the following description, the same parts are denoted by the same names and reference numerals. Their functions are also the same. Therefore, detailed description thereof will not be repeated.

  Referring to FIG. 1, the robot controller system according to this embodiment is mounted on the head of a user 26 who wants to control a robot 26 that can be controlled and that can be wirelessly controlled and that can be controlled by radio. The head electrode cover 22 that converts electrical changes at a plurality of locations on the head of the user 20 into electrical signals and outputs them as a time-series signal of a plurality of channels, and a time series of a plurality of channels from the head electrode cover 22 Based on the brain wave obtained from these time series signals, the psychological state of the user 20 is determined, feedback based on the psychological state is performed to the user 20, and a plurality of channels from the head electrode cover 22 are provided. Based on the time series signal, the intention of the user 20 regarding the control of the robot 26 is determined, and the robot 26 is controlled according to the determination result. Wirelessly and generates a command for including a robot controller 24 for performing a process of transmitting to the robot 26.

  In the present embodiment, the robot controller 24 uses a component in the 0-50 Hz band (EEG: electroencephalogram. Among the time-series signals of a plurality of channels obtained from the user 20, this is referred to as “first electroencephalogram”. Feedback control is performed on the basis of the frequency of the component (which is also an EEG (electroencephalogram), which is also referred to as a “second electroencephalogram signal” in the present embodiment). Use. One feature of the present embodiment is that the second electroencephalogram signal is obtained using the head electrode cover 22 for measuring the electroencephalogram, and the robot 26 is controlled based on the second electroencephalogram signal. In this embodiment, in particular, what operation the user 20 performs on his / her teeth is determined from the change in the electroencephalogram signal from the head electrode cover 22, and a command is generated according to the determination result.

  Referring to FIG. 2 (1), when the user brings the entire upper and lower teeth into contact with each other, the electroencephalogram signal measured by the electrode has the following characteristics. Here, the relationship between the electroencephalogram signals obtained from the frontal lobe A, the left temporal lobe B, and the right temporal lobe C shown in FIG. That is, in this case, the amplitude of the electroencephalogram signal obtained from the frontal lobe A (FIG. 2 (3A)) is small, and the amplitude of the electroencephalogram signals obtained from the left and right temporal lobes B and C (FIG. 2 (3B) and (3C)). Is big.

  Referring to FIG. 3 (1), when the user does not contact all the upper and lower teeth with each other, as shown in FIGS. 3 (3A)-(3C), frontal lobe A, left and right temporal lobes B, and The electroencephalogram signals obtained from C all have a small amplitude.

  Referring to FIG. 4 (1), when the user contacts the upper and lower front teeth 42 and the other teeth are not in contact, the electroencephalogram signal obtained from the frontal lobe is large as shown in FIG. 4 (3A). Indicates the amplitude. On the other hand, as shown in FIGS. 4 (3B) and (3C), the amplitudes of the electroencephalogram signals obtained from the left and right temporal lobes B and C are small.

  Referring to FIG. 5 (1), in a state where the user contacts the right back tooth 44 and the other teeth are not in contact, the electroencephalogram obtained from the right temporal lobe B as shown in FIG. 5 (3B). The signal shows a large amplitude. On the other hand, as shown in FIGS. 5 (3A) and (3C), the amplitudes of the electroencephalogram signals obtained from the frontal lobe A and the left temporal lobe C are small.

  Referring to FIG. 6 (1), when the user contacts the left back tooth 46 and the other teeth are not in contact, the electroencephalogram obtained from the left temporal lobe C as shown in FIG. 6 (3C). The signal shows a large amplitude. On the other hand, as shown in FIGS. 6 (3A) and (3B), the amplitudes of the electroencephalogram signals obtained from the frontal lobe A and the right temporal lobe B are small.

  The above is a general tendency, and in reality, it is not easy to distinguish these amplitude differences. In the present embodiment, learning data is prepared by causing a user to imagine controlling the robot 26 in advance and attaching a label corresponding to a control command to an electroencephalogram signal obtained at that time. SVM learning is performed using this learning data. In actual operation, this SVM is used as a pattern classifier, and a control command of the robot 26 by the user is estimated from the movement of the user's teeth.

  Conventionally, there has been an example in which tooth movement is determined from an electromyographic signal measured with an electrode attached to the cheek, but there is no example in which tooth movement is estimated from an electroencephalogram measured from above the scalp as in this embodiment. It was.

  Referring to FIG. 7, a robot controller 24 receives an A / D converter 62 for receiving time-series signals of a plurality of channels from a plurality of electrodes 60 of the head electrode cover 22 and converting them into digital signals of different channels. A band pass filter (BPF) 64 for passing only the components in the frequency band of 0-50 Hz for each of the digital signals of a plurality of channels output from the A / D converter 62, and the A / D converter 62 BPF 66 for passing only the components in the frequency band of 50 to 250 Hz, the digital first electroencephalogram signal of a plurality of channels from BPF 64, and the digital signals of the plurality of channels from BPF 66 Receiving the second EEG signal, and using the first EEG signal, the user's psychological or physical By detecting the state (emotion, fatigue level, etc.) and performing signal processing for performing feedback suitable for the soccer game by the robot to the user and the second brain wave signal, the user 20 controls the robot 26. PC (personal computer) PC68 for performing the signal processing which determines an intention and outputs a determination result as a command which controls the robot 26 is included.

  The robot controller 24 further includes a feedback device 70 for performing feedback processing according to the situation for the user 20 according to the feedback signal that the PC 68 processes and outputs the first brain wave signal, and the PC 68 includes the second brain wave. A wireless communication device 72 for receiving a command obtained by processing the signal and transmitting the command to the robot 26 by wireless communication.

  Referring to FIG. 8, PC 68 has a configuration similar to that of a general configuration PC. CPU (Central Processing Unit) 120 and bus 134 connected to CPU 120 are both connected to bus 134. An electrically readable / writable ROM (Read-Only Memory) 122 accessible by the CPU 120 by designating a predetermined range of addresses, and a RAM (Random Access Memory) 124 capable of writing and reading data at any time. A soccer game controlled by an I / O interface 126 that receives a time-series signal of a plurality of channels from the A / D converter 62, an I / O interface 128 to which the wireless communication device 72 shown in FIG. A sound box that outputs sound effects for the user 20 as feedback to the user 20 De 130 and a removable semiconductor memory 116, includes a memory port 132 to be mounted.

  Although not shown here, the PC 68 may further include a network adapter board that provides a connection to a local area network (LAN). A computer program for causing the PC 68 to function as the robot controller 24 is stored and distributed, for example, in the semiconductor memory 116, and is copied to the ROM 122 under the control of the CPU 120 when the semiconductor memory 116 is attached to the memory port 132. It may be transferred from another computer on the network to the PC 68 through a network adapter board (not shown) and written in the ROM 122. The program is loaded into the RAM 124 at the time of execution.

  This program includes a plurality of instructions that cause the PC 68 to function as the robot controller 24 according to this embodiment. Some of the basic functions necessary for realizing this function are provided by an operating system (OS) or a third-party program running on the PC 68 or modules of various tool kits installed on the PC 68. Therefore, this program does not necessarily include all functions necessary for realizing the system and method of this embodiment. This program is an appropriate program "tool" from within a "tool set" consisting of a set of functions or a set of functions for a computer in a controlled manner to obtain the desired result. It is only necessary to include a command for realizing the function as the robot controller 24 described above by calling. Since the operation of PC 68 is well known, details thereof will not be repeated here.

  Referring to FIG. 9, a program for realizing the function of controlling the robot 26 in accordance with the intention of the user 20 among the robot controller 24 of the PC 68 is a time series of a plurality of channels having a frequency band of 50 to 250 Hz output from the BPF 66. Step 150 for framing the signal with a predetermined frame length that does not overlap with each other, Step 152 for extracting a predetermined feature amount from each frame of each channel of the electroencephalogram signal following Step 150, and Step 152 Subsequently to Step 154, which generates and outputs a command for controlling the robot 26 intended by the user 20 by using a pattern classifier composed of SVM that has been learned in advance, using the feature amount as an input. The generated command is sent to the robot 26 via the wireless communication device 72. And and a step 156 to wirelessly transmit. After step 156, control returns to step 150, and the series of processes described above is repeated.

  As already described, in the time-series signal obtained from the head electrode cover 22, the spatial resolution of the signal obtained from each part of the brain is very low. Therefore, in order to make a highly accurate determination, it is necessary to extract a component effective for the determination from the obtained signal and reduce the dimension of the feature amount. In step 152, a matrix consisting of raw feature values obtained from the original time-series signal is obtained by using a technique called CSSD (Common Spatial Decomposition) (also referred to as CSP (Common Spatial Pattern)) described later. By multiplying the mapping matrix, it is converted into a set of feature quantities that are considered to be most effective for determination, and at the same time the dimension of the feature quantities is reduced. This mapping matrix is calculated in advance during SVM learning. The method will be described later.

  Referring to FIG. 10, the program routine for realizing the feature amount extraction process in step 152 includes steps 170 and 170 for calculating the amplitude of each signal as a feature amount from each frame of the time-series signal of each channel. The feature that is input to the pattern classifier is obtained by multiplying the feature matrix obtained for each channel from the raw feature obtained for each frame of each channel by the mapping matrix calculated in advance. And a step 172 of calculating the quantity and terminating this routine.

  As already described, the pattern classifier needs to learn in advance using learning data. In the present embodiment, a feature value mapping matrix is also calculated during the learning. Hereinafter, the learning process will be described. Learning requires the subject (user) to maintain the teeth in the state as shown in FIG. 2 to FIG. 6, and the output from the head electrode cover 22 during that time is shown in FIG. A label according to either one is attached and used as learning data.

  Referring to FIG. 11, the learning processing program inputs a label about what tooth shape the subject has taken, and following step 200, the tooth shape specified by the subject is entered. Step 202 is a step of framing a time-series signal having a predetermined time length obtained from the head electrode cover 22 while maintaining the above into a frame having the same frame length as that at the time of actual processing and not overlapping each other for each channel. A step 206 for calculating a predetermined feature amount (amplitude) from each frame of each channel obtained in step 202; and a feature amount obtained for the signal of each channel in step 206, for each channel and each frame. It is determined whether or not the step 206 for storing each time and the process for collecting learning data have been completed, and the flow of control is branched according to the determination result. And a step 208 that. In step 208, when the process for collecting learning data is not completed, the control returns to step 200.

  This program is further executed when it is determined in step 208 that the process for collecting learning data has been completed, and the mapping matrix is calculated by CSSD based on the data collected by the process of steps 200-208. Step 210 to be performed, and subsequent to step 210, the learning data accumulated in the processes up to steps 200 to 208 are arranged in a matrix format in which channels are arranged in each row and samples obtained from each frame are arranged in each column. By converting and multiplying this matrix by the transformation matrix obtained in step 210, the feature quantity obtained in step 212 and the step 212 for reducing the number of dimensions of the feature quantity are input and input in step 200. Learning the pattern classifier consisting of SVM with the correct label as correct data Ryosuru and a step 214.

-Reducing the number of feature dimensions by CSSD-
Here, in order to optimally separate each data group, the dimension of the raw feature amount is reduced using spatial filtering. As described above, this filtering technique is called CSSD or CSP, and in short, refers to a process of extracting an axis that can optimally separate the brain waves of two tasks. Since this method is essentially parallel and uses only scalar products, the calculation cost is low and it is optimal for real-time processing.

  A measurement sample of the EEG signal obtained by one data collection process at the time of learning is represented by a matrix E of N rows and T columns. N is the number of channels (number of electrodes), and T is the number of samples per channel. Here, the average amplitude obtained from one frame is one sample.

For example, for the sample _El obtained when the left tooth is brought into contact with the sample _Er obtained when the right tooth is brought into contact, the spatial covariance of the EEG is normalized as follows: Obtained from.

ただし、行列の右肩の「Ｔ」記号は行列の転置を表し、「ｔｒａｃｅ（Ｅ）」は行列Ｅの対角成分の和を表す。Ｅ_ｌ及びＥ_ｒはそれぞれいずれも同じ条件で、歯の接触条件のみを変えて測定されるので、サンプル行列Ｅ_ｌ及びＥ_ｒはは以下のように表すことができる。

However, the “T” symbol on the right shoulder of the matrix represents the transpose of the matrix, and “trace (E)” represents the sum of the diagonal components of the matrix E. Since E _l and E _r are both measured under the same conditions and changing only the tooth contact conditions, the sample matrices E _l and E _r can be expressed as follows.

ただしＳ_ｌは左側の歯を接触させることに特有の信号源を表し、Ｃ_ｌは対応する空間パターンを表す。Ｓ_ｒは右側の歯を接触させることに特有の信号源を表し、Ｃ_ｒは対応する空間パターンを表す。Ｓ_ｃはいずれにも共通の信号源を表し、Ｃ_ｃは対応する空間パターンを表す。

Where S _l represents a signal source specific to contacting the left tooth and C _l represents the corresponding spatial pattern. S _r represents a signal source specific to contacting the right tooth, and C _r represents the corresponding spatial pattern. S _c represents a common signal source, and C _c represents a corresponding spatial pattern.

となる。Ｗ_ｌは左側の歯を接触させることに対応する空間フィルタであり、Ｗ_ｒは右側の歯を接触させることに対応する空間フィルタである。この空間フィルタＷ_ｌ及びＷ_ｒはトレーニングデータを用いた学習時にＣＳＳＤにより算出される。 The purpose of CSSD is to obtain the spatial filters (mapping matrices) W _l and W _r necessary to extract the signal source components S _l and S _r . That is,

It becomes. W _l is a spatial filter corresponding to contacting the left tooth, and W _r is a spatial filter corresponding to contacting the right tooth. The spatial filters W _l and W _r are calculated by CSSD during learning using training data.

  Let ￣Cl and ￣Cr (“￣” should be written immediately above the character immediately after) be the average normalized covariance when the left and right teeth are in contact, respectively.

ただしＮ_ｌ及びＮ_ｒはそれぞれ、学習時における、左側及び右側の歯を接触させた状態に対応する学習データの収集試行回数を示し、Ｃ_ｌ（ｉ）及びＣ_ｒ（ｉ）はそれぞれ、ｉ回目の学習データの収集試行で得られた、左側及び右側の歯を接触させた状態のときの正規化共分散を示す。

N ₁ and N _r indicate the number of learning data collection attempts corresponding to the state in which the left and right teeth are in contact with each other during learning, and C _l (i) and C _r (i) are respectively i The normalized covariance obtained when the left and right teeth are in contact with each other, obtained in the second learning data collection trial, is shown.

ただしＵ_ｃは固有ベクトルからなる行列であり、λ_ｃは固有値からなる対角行列である。以下の説明では、固有値は降順にソートされているものとする。 If the sum of C ₁ and C _r is C, C can be decomposed as follows.

However, U _c is a matrix composed of eigenvectors, and λ _c is a diagonal matrix composed of eigenvalues. In the following description, it is assumed that the eigenvalues are sorted in descending order.

  Then, the whitening conversion is formulated as follows.

この結果、￣Ｃ_ｌ及び￣Ｃ_ｒは次のように変形できる。

As a result, ¯C _l and C _r can be modified as follows.

Ｓ_ｌ及びＳ_ｒはそれぞれ、以下のように分解できる。

Each of S _l and S _r can be decomposed as follows.

ここで、Ｓ_ｌ及びＳ_ｒは共通の固有ベクトル行列Ｕを持ち、対角固有値行列λ_ｌ及びλ_ｒは以下の関係を満たす。

Here, S _l and S _r have a common eigenvector matrix U, and the diagonal eigenvalue matrices λ _l and λ _r satisfy the following relationship.

ただしＩは単位行列である。行列λ_ｌ内の最大の固有値は、行列λ_ｒ内に、対応する最小の固有値をもち、その逆も成立する。行列Ｕ内において、その最初の固有ベクトルをＵ_ｌ（１行Ｎ列）とし、最後の固有ベクトルをＵ_ｒ（１行Ｎ列）とする。ベクトルＵ_ｌ及びＵ_ｒは２種類の歯の接触条件を識別するための最適化された固有ベクトルとなる。このとき、左側及び右側の歯を接触させた状態にそれぞれ対応する空間フィルタＷ_ｌ及びＷ_ｒは、それぞれ以下のようになる。

Where I is a unit matrix. The largest eigenvalue in matrix λ _l has the corresponding smallest eigenvalue in matrix λ _r and vice versa. In the matrix U, the first eigenvector is U _l (1 row N columns), and the last eigenvector is U _r (1 row N columns). The vectors U _l and U _r are optimized eigenvectors for identifying the contact conditions of the two types of teeth. At this time, the spatial filters W _l and W _r respectively corresponding to the state in which the left and right teeth are in contact are as follows.

学習データの収集時のサンプル行列Ｅは、次のように分解（マッピング）される。

The sample matrix E at the time of learning data collection is decomposed (mapped) as follows.

ただしＷ＝［Ｗ_ｌ；Ｗ_ｒ］であり、ＥはＮ行Ｔ列の行列である。

However, W = [W ₁ ; W _r ], and E is a matrix of N rows and T columns.

The signal group Z _p (p = 1... 2 m) that maximizes the variance of the two signal groups is the signal group corresponding to the maximum eigenvalues λ _l and λ _r . According to the calculation of the matrix W, these signals are the first m rows and the last m rows of the matrix Z after mapping.

After selecting the signal group Z _p, we calculate the normalized variance. Then, it calculates the feature amount f _p by the following equation, giving a linear pattern classifier on you to follow the normal distribution with the logarithm.

ＣＳＳＤは、２つの信号群の分類にしか利用できない。複数の信号群を分類するために、２つの信号群の各々に対するＣＳＳＤ特徴量を加算して得られた合計ＣＳＳＤ特徴量を用いる。パターン分類器への入力としてもこの合計ＣＳＳＤ特徴量を用いる。

CSSD can only be used to classify two signal groups. In order to classify a plurality of signal groups, a total CSSD feature value obtained by adding the CSSD feature values for each of the two signal groups is used. This total CSSD feature quantity is also used as an input to the pattern classifier.

[Operation]
The robot controller 24 according to the above embodiment operates as follows. The operation of the robot controller 24 has two phases. The first phase is the calculation of the mapping matrix W and the learning of the pattern classifier. The second phase uses this pattern classifier to determine the state of contact of the teeth by the user 20, and commands according to the state Is an operation phase for outputting.

-Learning phase-
Learning is performed as follows. The subject (user 20) wears the head electrode cover 22 on the head, and each electrode 60 of the head electrode cover 22 is connected to an input of the A / D converter 62 of the robot controller 24.

  The learning instructor selects one of the states shown in FIGS. 2 to 6 and instructs the subject to set the tooth state to that state. When the subject takes the state, the experimenter inputs a label indicating the state (step 200 in FIG. 11). Further, reading each of the signals from the plurality of electrodes 60 is repeated for a predetermined time, and the signals of the plurality of channels are framed into frames of a predetermined length (step 202). A feature amount is calculated for each frame of each channel (step 204). Specifically, the average amplitude of the signal in each frame is calculated.

  Subsequently, for each frame of each channel, the feature amount calculated in step 206 is stored in a storage medium (step 206).

  The above processing is repeated until sufficient learning data is obtained. Although sufficient learning data has been obtained, in step 210, the above-described CSSD (CSP) is used to calculate the mapping matrix and store it in the storage medium. Further, in step 212, using this mapping matrix, the feature amount obtained in step 206 is converted into the CSSD feature amount described above for each channel. The CSSD feature value obtained in this way is accumulated as learning data together with the label input in step 200 to obtain learning data.

  Finally, learning is performed on the pattern classifier (SVM in the present embodiment) using the learning data obtained in step 212, and the process ends when the learning is completed (step 214). This completes the learning process.

-Operation phase-
In the operation phase, the same user 20 as the subject in the learning phase wears the head electrode cover 22 on the head. Each electrode 60 is connected to an A / D converter 62. When the wireless communication connection between the wireless communication device 72 and the target robot 26 is established and the initialization of the feedback device 70 and the like is completed, the game by the robot 26 can be executed.

  When the game is started, the user 20 changes the state of the teeth so as to be in a contact state associated with an instruction to be performed on the robot 26. At this time, a time series signal is given from each electrode 60 to the A / D converter 62. The A / D converter 62 converts the time series signals of the plurality of channels into digital signals and supplies them to the BPF 64 and the A / D converter 62. Of these signals, the BPF 64 passes only the 0-50 Hz band component and gives it to the PC 68 as the first electroencephalogram signal. The BPF 66 passes only the 50-250 Hz band component of the input signal and gives it to the PC 68 as the second electroencephalogram signal.

  The PC 68 identifies the psychological state (emotion, fatigue level, etc.) of the user 20 based on the signal given from the BPF 64 as the first electroencephalogram signal, and gives feedback to the user 20 in a preprogrammed manner. For example, when the user 20 is calm even if the score is entered, a stimulus is given to increase the excitement, or when the score is given to the other party and the user 20 is very excited, the user 20 is calmed. Or playing music using headphones 114. Since the control of the feedback device 70 by the PC 68 is not directly related to the present invention, a detailed description thereof will not be given here.

  On the other hand, the PC 68 generates a command for controlling the robot 26 using the second electroencephalogram signal given from the BPF 66 as follows. The PC 68 divides each of the plurality of channels of the second electroencephalogram signals obtained from the head electrode cover 22 provided from the BPF 66 into frames of a predetermined length (step 150 in FIG. 9), and for each frame of each channel. A feature amount is calculated (step 152). Specifically, the PC 68 calculates the average amplitude as a feature amount for each frame of each channel (step 170 in FIG. 10), and then uses the mapping matrix obtained at the time of learning to calculate the feature amount. The CSSD feature quantity with reduced dimensions is calculated (step 172).

  This CSSD feature quantity is given to the learned pattern classifier (step 154 in FIG. 9). From the pattern classifier, a value indicating which of the teeth of the user 20 is shown in FIGS. 2 to 6 is output. The PC 68 selects a command previously assigned to each state, and transmits it to the robot 26 via the wireless communication device 72.

  The robot 26 performs processing corresponding to this command and continues the soccer game. As described above, according to this embodiment, it is also possible to obtain information such as the user's psychological state and fatigue level from the electrode worn on the head, and at the same time, the contact state of the user's teeth, that is, the user It is also possible to estimate the intention. It is not necessary to wear an electrode for obtaining an electromyogram on the cheek for the purpose of knowing the state of the teeth. Moreover, it has been found that when the contact state of the user's teeth is estimated by machine learning using CSSD as described above, the accuracy becomes very high. Conventionally, the second electroencephalogram signal component is considered to be a disturbing component for the measurement of the electroencephalogram and is excluded from the measurement signal. Since such a signal component can be used to estimate the user's intention with high accuracy, the present invention can be efficiently applied particularly in a scene where an electroencephalogram is measured. However, it is not always necessary to implement the present invention together with the measurement of the electroencephalogram, and it is needless to say that the processing using the electroencephalogram need not be performed.

  In the above embodiment, the intention of the user 20 to control the robot is estimated from the electrical signals of a plurality of channels obtained from the user's 20 head. However, the present invention is not limited to such an embodiment. The present invention can always be effectively applied in a situation where it is required to transmit some command to an object to be controlled without using a hand. For example, it is obvious that the present invention can also be applied to control of an electric wheelchair, control of a game, mouse cursor in a computer, and control for generating some action.

  Furthermore, the present invention can also be applied to a case where an operation for controlling an object cannot be performed using both hands for some reason. For example, when performing maintenance of a complicated machine, there may be an application in which a work manual is presented to a mechanic using virtual reality technology. When performing machine maintenance, you may not be able to move your hands away from the machine. It will be apparent to those skilled in the art that the present invention can be used when, for example, a situation arises in which it is desired to advance the manual to the next page. In addition, when a firefighter, police officer, soldier, etc. holds a device in both hands and communicates or talks with another person, or it is necessary to operate a device other than the device held in both hands Also, the present invention can be applied.

  In the above embodiment, an electroencephalogram signal associated with the contact state of the teeth is obtained from the output of the head electrode cover 22, and the intention of the user 20 is estimated. However, the present invention is not limited to such an embodiment, and can be similarly applied to an organ that can take a plurality of states in the head of the user 20. For example, it is considered that the intention of the user 20 can be estimated based on the same principle with respect to a plurality of states of the eye such as crushing one of the eyes or repeating blinking.

  In the above embodiment, SVM is also used as the pattern classifier. However, the present invention is not limited to the one using SVM. For example, an artificial neural network (neural network), a Kalman filter, a hidden Markov model, a linear separator, or the like may be used.

  In the above embodiment, the tooth contact state is estimated from the brain wave component of 50 Hz to 250 Hz, but this is only the frequency range that can be estimated with the highest accuracy, and is 50 Hz or less or 250 Hz or more, for example, 0.5 Hz or more and 500 Hz. It is possible to estimate the contact state of teeth from the following electroencephalogram components in any band.

  In the above embodiment, the feature amount is extracted using the spatial filter obtained by the CSSD method and the tooth contact state is estimated. However, the present invention is not limited to such an embodiment. It is also possible to filter the electroencephalogram signal measured from the electrodes in a specific frequency band, and use this as a feature value to directly estimate the contact state of the teeth using a separator such as an artificial neural network or SVM. .

  Further, in the above embodiment, the CSSD method is used to obtain the coefficient of the spatial filter. However, the present invention is not limited to such an embodiment, and an axis that can optimally separate the brain waves of two tasks is extracted. Any method may be used as long as it is possible.

  Further, after converting the electroencephalogram signals measured in a plurality of channels into the frequency domain, it is possible to employ a method of comparing the power of each channel and estimating the contact state of the teeth.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

20 User 22 Head electrode cover 24 Robot controller 26 Robot 60 Electrode 62 A / D converter 64, 66 BPF
68 PC
72 Wireless communication device 120 CPU

Claims (8)

Estimate the user's tooth movement from multiple channels of EEG signals that detect the user's brain activity obtained at multiple locations on the user's scalp, and select the target device according to the estimated tooth movement. A device control device for generating a command to be controlled,
Filter means for extracting a component of a predetermined frequency band induced by the movement of the user's teeth from each of the signals of the plurality of channels;
Feature quantity calculation means for calculating a predetermined feature quantity pattern from the frequency band components of the signals of the plurality of channels extracted by the filter means;
Given a feature amount pattern from the feature amount calculation means, a pattern classification means that has been learned in advance so as to obtain an output indicating the movement of the user's teeth according to the feature amount;
A device control device including command generation means for generating a predetermined command according to an output of the pattern classification means and giving the command to the target device. The pattern classification means includes
A pattern classifier that has been learned in advance according to the pattern of the feature amount obtained from the user when the contact state of the teeth is maintained to be one of a plurality of predetermined ones. The apparatus control apparatus as described in. The apparatus control apparatus according to claim 2, wherein the pattern classifier includes a support vector machine (SVM). The said filter means contains the band pass filter which extracts the arbitrary band components in the band of 0.5 Hz or more and 500 Hz or less from each of the said multiple channel signal. Equipment control device. The device control apparatus according to claim 4, wherein the band-pass filter includes a band-pass filter that extracts only a band component of 50 Hz or more and 250 Hz or less from each of the plurality of channel signals. The apparatus control device according to claim 1, wherein the signals of the plurality of channels are obtained from an electrode disposed on a front part of the user's scalp and both immediate heads. The feature amount calculating means includes:
Framing means for dividing the frequency band components of the signals of the plurality of channels into frames of a predetermined length;
Feature quantity extraction means for extracting a predetermined first feature quantity from each frame of each channel;
And a feature value conversion means for converting the first feature value extracted by the feature value extraction means so that a ratio of variance of feature values obtained from different states of brain activity is maximized. The apparatus control apparatus in any one of Claims 1-6. Estimate the user's tooth movement from multiple channels of EEG signals that detect the user's brain activity obtained at multiple locations on the user's scalp, and select the target device according to the estimated tooth movement. A device control method for generating a command to be controlled,
Arranging electrodes at the plurality of locations on the user's scalp to generate a plurality of channels of signals for detecting the activity of the user's brain;
Extracting a component of a predetermined frequency band induced by the movement of the user's teeth from each of the signals of the plurality of channels;
Calculating a predetermined feature amount pattern from the frequency band components of the signals of the plurality of channels extracted by the extracting step;
Given a pattern of feature amounts, the pattern classification means that has been learned in advance so as to obtain an output indicating the movement of the user's teeth according to the feature amount, the pattern obtained in the step of calculating the pattern Providing a pattern of feature values and receiving the output of the classification results;
And a step of generating a predetermined command according to the output of the pattern classification means and giving the command to the target device.

Images