JP6759496B2 - EEG pattern classification device, EEG pattern classification method, EEG pattern classification program and neurofeedback system - Google Patents

Description

The present invention presents an electroencephalogram pattern classification device for classifying the state of brain activity from a user's (user) brain wave signal, an electroencephalogram pattern classification method, an electroencephalogram pattern classification program, and a neuro for executing neurofeedback to the user. It relates to the configuration of the feedback system.

With the declining birthrate and aging society becoming a problem in many countries including Japan, there is an increasing demand for assistive devices that apply robotics technology. On the other hand, robots capable of balancing and walking have been developed. For example, there is a robot that can optimally distribute the acting force required for exercise to any plurality of contact points in space and generate torque for each joint in the same manner as a human (see Patent Document 1).

In recent years, there has been an increasing demand for the development of robots that support rehabilitation, such as exoskeleton robots that aim to support upper limb, lower limb, and trunk movements. For example, an exoskeleton type robot is used for a patient such as a stroke in rehabilitation that promotes an independent living of the patient (see Patent Documents 2 and 3).

In this way, in robot control using biological signals for compensation and recovery of motor function, surface electromyogram (sEMG) and electroencephalogram (Electroencephalogram) are used from the viewpoint of burden on the patient and simplicity of the system. : EEG) and other non-invasive detections of biological signals have been reported (see Non-Patent Document 1, Non-Patent Document 2, Non-Patent Document 3, and Non-Patent Document 4).

In particular, the use of brain signals to control a device as an assist robot is becoming a popular research direction (Non-Patent Document 5).

In this way, the Brain Machine Interface (BMI), which uses the measurement results of human brain activity directly to control external devices, allows the user to control the robot as a part of his or her body. It is a promising interface for wearable devices.

In addition, it has been demonstrated that the use of non-invasive BMI for exercise support by an exoskeleton type robot is useful for improving intention / motion control damaged by a stroke, for example (Non-Patent Document 6). ..
Such rehabilitation procedures use brain signals to activate devices that support exercise. It is thought that such supportive exercise based on brain activity causes plasticity of the central nervous system and leads to restoration of normal motor control.

In other words, endogenous BMI based on sensorimotor rhythms (SMR) can be used in motor neurorehabilitation as an effective tool for promoting the plasticity of nerve cells in nerve and muscle pathways. It has been suggested (Non-Patent Document 7). Such a rehabilitation system can be said to be a kind of "neurofeedback system" because the brain activity of the user is fed back to the support of the subject's movement by a robot or the like.

For example, with respect to SMR, the so-called "motor imagination", in which the user imagines movement or kinesthetic sensation, typically results in a decrease in the power of μ rhythm (7-13 Hz) and β rhythm (13-30 Hz) in the brain wave. Known to accompany, such power loss is called event-related desynchronization (ERD), especially for the exercising and contralateral brain areas.
It has been shown that an ERD-based BMI system can be used to control external devices such as neuroprostheses (eg, artificial limbs). More specifically, clinical trials have also been conducted to confirm the effectiveness of an ERD-based Brain Robot Interface (BRI) for recovery of motor control (Non-Patent Document 8). ..

WO2007 / 139135 Japanese Unexamined Patent Publication No. 2014-104549 Japanese Unexamined Patent Publication No. 2016-61302

T. Kagawa and Y. Uno. Gait pattern generation for a power-assist device of paraplegic gait. The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 633-638, 2009. Tomoyuki Noda, Norikazu Sugimoto, Junichiro Furukawa, Masa aki Sato, Sang-Ho Hyon, and Jun Morimoto. Brain-controlled exoskeleton robot for bmirehabilitaion. IEEE / RAS International Conference on Humanoid Robotics (Humanoids2012), Osaka, pp.21-27, 2012. Tomoyuki Noda, Jun ichiro Furukawa, Tatsuya Teramae, and Jun Morimoto. An electromyogram based force control coordinated in assistive interaction. IEEE International Conference on Robotics and Automation (ICRA2013), Karlsruhe, Germany, p. WeC6.4, May 2013. Tomohiro Hayashi, Hiroaki Kawamoto, and Yoshiyuki Sankai. Control method of robot suit hal working as operator's muscle using biological and dynamical information. IEEE / RSJ International Conference on Inteligent Robots and Systems, pp. 3063-3068, 2005. X. Perrin, R. Chavarriaga, F. Colas, R. Siegwart, and J. del R. Milln, “Brain-coupled interaction for semi-autonomous navigation of an assistive robot,” Robotics and Autonomous Systems, vol. 58, no . 12, pp. 1246 --1255, 2010, intelligent Robotics and Neuroscience. L. F. Nicolas-Alonso and J. Gomez-Gil, “Brain computer interfaces, a review.” Sensors (Basel, Switzerland), vol. 12, no. 2, pp. 1211-1279, Jan 2012. N. Mrachacz-Kersting, SR Kristensen, IK Niazi, and D. Farina, “Precise temporal association between cortical potentials evoked by motor imagination and afference induces cortical plasticity,” The Journal of Physiology, vol. 590, no. 7, pp. 1669-1682, 2012. K. Shindo, K. Kawashima, J. Ushiba, N. Ota, M. Ito, T. Ota, A. Kimura, and M. Liu, “Effects of neurofeedback training with an electroencephalogram-based brain-computer interface for hand paralysis in patients with chronic stroke: a preliminary case series study. ”Journal of rehabilitation medicine, vol. 43, no. 10, pp. 951-957, Oct 2011.

However, when trying to apply BMI to a rehabilitation system or the like, there are the following problems.

First, in general, electroencephalographs ensure electrical contact by applying gel to the user's scalp, so electroencephalograph systems using such gels require a long setup time.

Secondly, the signal from the electroencephalograph generally has a large non-stationarity, and there is a problem that the signal fluctuation within or between one session in which BMI rehabilitation is performed is large.

Third, users generally tend to be "insufficient" about BMI and require preliminary training for a significant proportion of users.

Due to these problems, it takes a considerable amount of preparation time to be able to perform BMI-based neurofeedback training for rehabilitation, etc., and the system also requires a considerable amount of preparation time within or between training sessions. There were problems such as the need for readjustment.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to perform a process of non-invasively detecting an electroencephalogram signal and classifying an electroencephalogram pattern. It is an object of the present invention to provide an electroencephalogram pattern classification device, an electroencephalogram pattern classification method, and an electroencephalogram pattern classification program capable of shortening the time until the start of use by the user.

Another object of the present invention is to provide a neurofeedback system capable of shortening the time until the start of use by the user even when performing neurofeedback using an electroencephalogram signal.

According to one aspect of the present invention, it is a brain wave pattern classification device for asynchronously classifying brain wave patterns based on the subject's motion imagination, and detects the activity of the subject's brain at a plurality of locations on the subject's scalp. Brain wave measuring means for acquiring a plurality of channels of brain wave signals, a filter means for extracting a component of a predetermined frequency band induced by motion imagination by a subject from each of the signals of the plurality of channels, and a filtering means. A feature amount calculation means for adaptively calculating a feature amount by a spatial filter by processing for each sliding window along the time from the components of the frequency band of the signal of a plurality of channels extracted by The calculation means are a spatial filter updating means that adaptively updates the spatial filter derived from the covariance matrix of the brain wave signals of multiple channels up to now by calculating the exponential moving average of the covariance matrix, and the exponential moving average. Including the oblivion coefficient determining means for determining the oblivion coefficient in the above based on the ratio of the step width of the sliding window to the time constant of the exponential moving average, the feature amount is obtained by the classification process based on the feature amount from the feature amount calculating means. It is provided with a classification processing means for outputting a signal indicating that the subject has performed motion imagination accordingly.

Preferably, the derivation of the spatial filter is by the CSP (Common Spatial Patterns) method, and the spatial filter updating means is used for the projection matrix in the CSP method derived in the training set during the learning period, and the electroencephalogram signal at the present time. It includes an adaptive whitening processing means for calculating an updated spatial filter from an adaptive projection matrix obtained by multiplying the whitening matrix of the above and the inverse matrix of the whitening matrix calculated for the training set.

Preferably, the forgetting coefficient determining means determines the time constant from a plurality of candidates for the time constant of the forgetting coefficient by n-fold cross-validation (n: a natural number of 2 or more) with respect to the output of the classification processing means.

Preferably, the feature amount calculating means further includes an adaptive feature processing means that converts the feature amount derived by the updated spatial filter into a feature amount of zero average.

Preferably, the electroencephalogram measuring means includes a wireless headset with dry electrodes.

According to another aspect of the present invention, an arithmetic unit and an arithmetic unit based on a signal from a brain wave measuring device for acquiring a multi-channel brain wave signal for detecting the activity of the subject's brain at a plurality of locations on the subject's scalp. This is a brain wave pattern classification method for causing a computer having a storage device to perform a process of asynchronously classifying brain wave patterns based on the subject's motion imagination. Each of the signals of a plurality of channels is stored in the storage device, and the stored plurality of signals are stored. From the signal of the channel, the arithmetic unit extracts by filtering the components of a predetermined frequency band induced by the motion imagination by the subject, and the arithmetic unit extracts by filtering by the processing for each sliding window along the time. The step of calculating the feature amount is provided with the step of adaptively calculating the feature amount by the spatial filter from the components of the frequency band of the signal of the plurality of channels extracted in the step of performing the step. The step of adaptively updating the spatial filter derived from the covariance matrix by calculating the exponential moving average of the covariance matrix, and the oblivion coefficient in the exponential moving average of the sliding window for the time constant of the exponential moving average . The subject performed motion imagination according to the feature amount by the classification process based on the feature amount calculated in the step of calculating the feature amount by the arithmetic unit including the step of determining based on the step width ratio . It includes a step of outputting a signal indicating that.

According to yet another aspect of the present invention, an arithmetic unit is based on a signal from a brain wave measuring device for acquiring a multi-channel brain wave signal for detecting the activity of the subject's brain at a plurality of locations on the subject's scalp. A brain wave pattern classification program for causing a computer having a storage device to perform a process of asynchronously classifying brain wave patterns based on the subject's motion imagination. The program stores each of a plurality of channels of signals in the storage device. From the stored multi-channel signals, the arithmetic unit extracts the components of a predetermined frequency band induced by the motion imagination by the subject by filtering, and the arithmetic unit processes each sliding window over time. , A step of adaptively calculating a feature amount by a spatial filter from the components of the frequency band of a multi-channel signal extracted in the step of extracting by filtering, and a plurality of steps for calculating the feature amount up to now. The step of adaptively updating the spatial filter derived from the covariance matrix of the brain wave signal of the channel by calculating the exponential moving average of the covariance matrix, and the oblivion coefficient in the exponential moving average are the time constants of the exponential moving average. The subject moves according to the feature amount by the classification process based on the feature amount calculated in the step of calculating the feature amount by the arithmetic unit, including the step of determining based on the ratio of the step width of the sliding window to Have the computer perform a process that includes a step of outputting a signal indicating that the imagination has been performed.

According to still another aspect of the present invention, it is a neurofeedback system that supports the movement of the subject in response to the detection of the movement imagination in the subject's brain, and is a brain wave by the movement imagination in the subject's brain. A brain wave pattern analyzer for classifying patterns asynchronously is provided, and the brain wave pattern analyzer is for acquiring multi-channel brain wave signals for detecting the activity of the subject's brain at a plurality of locations on the subject's scalp. The frequency of the multi-channel signal extracted by the brain wave measuring means, the filter means for extracting the component of the predetermined frequency band induced by the motion imagination by the subject from each of the multi-channel signals, and the filter means. It is equipped with a feature amount calculation means for adaptively calculating the feature amount by a spatial filter by processing each sliding window along the time from the band component, and the feature amount calculation means is a multi-channel brain wave up to now. Spatial filter updating means that adaptively updates the spatial filter derived from the covariance matrix of the signal by calculating the exponential moving average of the covariance matrix, and the oblivion coefficient in the exponential moving average, the time constant of the exponential moving average. It has an oblivion coefficient determining means that determines based on the ratio of the step width of the sliding window to, and the subject performs motion imagination according to the feature amount by the classification process based on the feature amount from the feature amount calculating means. It includes a classification processing means that outputs a signal indicating that, and a threshold determination means that generates a trigger signal when the output of the classification processing means exceeds a predetermined threshold value, and is an object according to the trigger signal. It is equipped with an assist robot that supports the movement of the person.

According to the electroencephalogram pattern classification device, the electroencephalogram pattern classification method, and the electroencephalogram pattern classification program of the present invention, it is possible to deal with the non-stationarity of the feature quantity from the electroencephalogram signal in an asynchronous state by using an adaptive algorithm. It is possible to improve the classification performance of the classifier.

In the neurofeedback system of the present invention, by using the above-mentioned electroencephalogram pattern classification method, the classification of the user's sensorimotor imagination is performed with high reliability, and rehabilitation by exercise support by the assist robot can be performed. Is.

It is a conceptual diagram for explaining the structure of the neurorehabilitation system using the neurofeedback about the lower limbs. It is a figure which shows the configuration example of the exoskeleton type robot 1 in this embodiment. It is a figure which shows the structure of the degree of freedom of an exoskeleton type robot 1. This is an example of a block diagram of the exoskeleton type robot 1. It is a functional block diagram explaining the structure for executing the classification process of an electroencephalogram pattern. It is a flowchart when the asynchronous and adaptive CSP algorithm is executed online. It is a figure for demonstrating the time variation of the adapted feature quantity. It is a figure which shows the relationship between the output of a classifier and a threshold. It is a figure explaining the state of robot control. It is a figure which shows the result of cross-validation at the time of performing the pseudo-online processing. It is a figure which shows the spatial pattern automatically selected for each subject over each division process of cross-validation. It is a figure which shows the sliding window output of one trial of the subject S1. It is a figure which compares the pseudo-online performance of cross-validated all-or-nothing by three adaptation methods. FIG. 5 is a conceptual diagram showing learning and online processing of the electroencephalogram pattern analysis unit 40 in the first embodiment. FIG. 5 is a conceptual diagram showing learning and online processing of the electroencephalogram pattern analysis unit 40 in the second embodiment.

Hereinafter, the configuration of the exoskeleton type robot according to the embodiment of the present invention and the electroencephalogram pattern classification device for controlling the operation of the exoskeleton type robot will be described with reference to the drawings. In the following embodiments, the components and processing steps having the same reference numerals are the same or equivalent, and the description thereof will not be repeated if they are not necessary.

Further, as an actuator for driving the joint of the exoskeleton type robot, the "electrostatic hybrid type actuator" described below will be described as an example.

Therefore, in the present embodiment, the exoskeleton type robot using the static-electric hybrid actuator for walking / posture rehabilitation will be described below.

However, the static-electric hybrid exoskeleton type robot of the present invention can be used not only as an exoskeleton type robot for assisting the movement of the lower limbs but also as an exoskeleton type robot for assisting the movement of the upper limbs. It is possible.

Further, in the following description, an exoskeleton type robot that assists the movement of the lower limbs as a pair will be described, but the exoskeleton type that assists the movement of either one of the lower limbs or one of the upper limbs. It can also be used as a robot.

Further, the static-electric hybrid exoskeleton type robot of the present invention can assist the movement of the target human musculoskeletal system, as described above, "at least one of the lower limbs, or the upper limb. The movement is not limited to at least one of them, and may be, for example, assisting only the movement of the shoulder of the target human, or assisting the movement of the ankle when standing or walking. It may be a thing. In the present specification, such assistance of the target human musculoskeletal movement is collectively referred to as "support (assist) of the target human musculoskeletal movement".

The exoskeleton type robot of the present embodiment has an exoskeleton. The "exoskeleton" is a skeletal structure possessed by a robot corresponding to a human skeletal structure. More specifically, the "exoskeleton" refers to a frame structure that externally supports a part of the human body that wears an exoskeleton type robot.

The frame structure is further provided with joints for moving each part of the frame structure in response to movement based on the human skeletal structure.

In particular, the exoskeleton type robot that assists the movement of the lower limbs is a robot that has a base and a lower body and has joints with active 6 degrees of freedom at the left and right positions of the ankles, knees, and hips. The six joints are statically driven hybrid drive joints. Hereinafter, in the exoskeleton type robot, the joint driven by the actuator to give the support force to the user's joint is referred to as an "active joint".
[Embodiment 1]
FIG. 1 is a conceptual diagram for explaining the configuration of a neurorehabilitation system using neurofeedback for the lower limbs of the first embodiment.

An exoskeleton type robot 1 as described later is attached to the lower limbs of the user 2. The operation of the exoskeleton type robot 1 is controlled in response to a command from the external control device 20. Depending on the situation, the user 2 may be assisted by the physiotherapist 3 during the rehabilitation.

An electroencephalograph 300 for measuring the brain activity of the user 2 is attached to the head of the user 2.

The measurement data from the electroencephalograph 300 can be transmitted to the external control device 20 by a wireless interface such as Bluetooth (registered trademark) in addition to the wired interface. Based on the signal from the electroencephalograph 300, information indicating the activity status in the brain and information indicating the status of neurofeedback are displayed on the display device 212 by the external control device 20.

The electroencephalograph 300 may use a wet (gel type) electrode, but is not particularly limited here, but the electroencephalograph 300 will be described as a dry type (dry type) using a wireless electroencephalograph. To do.

As mentioned above, when using EEG signals acquired by a dry wireless headset to control an exoskeleton robot, the setup time is shorter and the setup time is shorter than in a wet system using gel. Difficulties caused by cable routing can be eliminated. Such a dry wireless electroencephalograph is disclosed in the following documents, for example.

Known Documents 1) PMR Reis, F. Hebenstreit, F. Gabsteiger, V. von Tscharner, and M. Lochmann, “Methodological aspects of EEG and body dynamics measurements during motion.” Frontiers in human neuroscience, vol. 8, p. 156 ， Mar 2014.
Further, in the following, as an example of the exercise supported by the exoskeleton type robot attached to the lower limbs, a squat exercise is assumed. This is because squat exercises are also used in rehabilitation programs because they require functional coordination of the ankles, knees and hips.

FIG. 2 is a perspective view showing an extracted main part of the exoskeleton type robot 1 according to the present embodiment. The exoskeleton type robot 1 has 10 degrees of freedom.

Regarding the configuration example of such an exoskeleton type robot, a similar configuration is also disclosed in Patent Document 2 and Patent Document 3 described above.

In FIG. 2, the exoskeleton type robot 1 has a frame structure corresponding to both legs, a backpack 101, a flexible seat 102, a HAA antagonist muscle 103, an HFE extensor muscle 104, an HFE motor 111, a KFE extensor muscle 105, a KFE motor 106, and an AFE. It includes an extensor / AAA antagonist muscle 107, an AFE flexor muscle 108, a universal joint 109, and a rotary joint 110 with a pulley provided in a frame structure.

In FIG. 2, the backpack 101 is directly attached to the structure that supports the exercise, but the backpack 101 may be removed from this structure.

Further, for example, an optical encoder is attached to the rotation shaft of the universal joint 109 to measure the joint angle. An optical encoder is also attached to the rotary joint 110 with a pulley. The optical encoder may be configured to read the moving direction and the amount of movement of the belt wound around the shaft instead of being attached to the shaft. In the HFE and KFE joints, which are hybrid joints, the joint angle may be measured using an encoder attached to the motor.

FIG. 3 is a diagram showing the configuration of the degree of freedom of the exoskeleton type robot 1.

In FIG. 3, in each joint, the indication "R_" indicates that it is the right joint, and the indication "L_" indicates that it is the left joint.

With reference to FIGS. 2 and 3, the HFE joint and the KFE joint are hybrid-driven out of the total 10 degrees of freedom. Further, in FIG. 2, out of all 10 degrees of freedom, the left and right AFE joints employ antagonistic drive by extensor muscles and flexor muscles. Joints other than hybrid drive and antagonistic drive are passive drive. However, more joints, for example all joints, may be hybrid driven.

In FIG. 2, a posture sensor is mounted on the body portion to which both legs are connected to detect the posture of the base portion. In addition, wire-type encoders (or encoders attached to motors) are attached to all joints so that joint angles can be measured. By detecting the joint angle, the target torque generated in each joint can be calculated.

In addition, a floor reaction force sensor is mounted on the sole of the foot, which is used as an auxiliary to determine whether or not the sole of the foot, which is supposed to be in contact, is actually in contact, and to correct model errors included in the Jacobian matrix. It may be configured to be used.

In addition to the controller, the backpack 101 contains an air muscle valve and an electric motor driver.

In addition, a battery, a squeezed air cylinder (or a CO ₂ gas cylinder may be used), and a regulator are mounted in the backpack 101 to enable short-time autonomous driving in case the power line and air supply are interrupted. It may be.

FIG. 4 is an example of a block diagram of the exoskeleton type robot 1.

A command for controlling the exoskeleton type robot 1 is given to the exoskeleton type robot from the external control device 20 via a communication path. Although not particularly limited, the external control device 20 can use a general-purpose personal computer, and an Ethernet (registered trademark) cable can be used as the communication path. Of course, as the communication path, in addition to the wired communication path of other standards, a wireless communication path, for example, a wireless LAN (Local Area Network) or a wireless communication standard may be used.

The external control device 20 is a non-volatile storage device in which an input unit 208 that receives an instruction input from a user, a program for generating a command, and data required for control such as various control parameters are recorded. Commands are issued according to the program, the 206, the memory 204 including the ROM (Read Only Memory) in which the firmware for starting the external control device 20 is stored, the RAM (Random Access Memory) that operates as the working memory, and the like. Under the control of the arithmetic unit 210 that executes the generated process, the interface (I / F) unit 202 for transmitting commands to the outer skeleton type robot via the communication path, and the arithmetic unit 210, the outer skeleton type It is provided with a display device 212 for displaying information and the like regarding the state of control of the robot 1.

As described above, when the external control device 20 is a general-purpose personal computer, the arithmetic unit 210 is composed of a CPU (Central Processing Unit), and the non-volatile storage device 206 includes a hard disk drive, a solid state drive, or the like. Can be used. However, a part or all of the functional blocks of the external control device 20 may be configured by dedicated hardware.

Further, the external control device 20 performs a process of configuring an estimation model for estimating joint torque from the joint angle and the like at the time of calibration for a user to which the exoskeleton type robot is mounted, for example. Information about the constructed estimation model is stored in a storage device such as a storage device 132.

The exoskeleton type robot 1 includes an exoskeleton portion 12 and an internal control device 10.

The exoskeleton portion 12 includes a base 121, a lower body 122, an active joint 123, and a detection mechanism 124. Further, the active joint 123 includes an air muscle 1231 (not shown) and an electric motor 1232 (not shown).

Further, the internal control device 10 includes an I / F unit 11, a recording device 131, a storage device 132, a measuring device 133, a control unit 134, and an output device 135.

The I / F unit 11 can receive a torque, a position command, or the like commanded from the external control storage 20.

The base 121 may be considered to include the skeleton at the waist position and the active joint 123 at the waist position, or may be considered to be only the skeleton at the waist position.

The lower limbs 122 may be considered to include a skeleton at the position of the thigh or foot, an active joint 123 at the position of the thigh or foot, or may be only a skeleton at the position of the thigh or foot.

The active joint 123 is an active joint located at each position of the left and right ankles, the left and right knees, and the left and right hips. Here, the active joint 123 is a joint that can be actively operated by an actuator. That is, the active joint 123 includes an actuator.

Further, one or more active joints 123 here are of a hybrid type. That is, at least a part of the active joint 123 is a hybrid type including an air muscle 1231 and an electric motor 1232. The actuator has a function of receiving a torque value which is a control target value as a drive signal and controlling the actuator based on the received torque value.

When a servomotor is used as the actuator, the actuator has, for example, a drive circuit capable of controlling current, and the servomotor that generates torque proportional to the current is set to a torque value input as a control target value. Torque control that generates the input torque is realized by multiplying the torque constant determined by the ratio and instructing the drive circuit. In particular, by disposing a torque sensor on the active joint 123 and feeding back the value detected by the torque sensor to the drive circuit, highly accurate torque control becomes possible.

The detection mechanism 124 detects the state of the robot. The detection mechanism 124 is, for example, an encoder arranged in each joint, a floor reaction force sensor arranged in the palm, a gyro sensor arranged in the pelvis for posture detection, and a load cell for detecting the driving force of each air muscle. And so on. The detection mechanism 124 may be an angle sensor that detects the angle of the joint, a posture sensor that acquires the posture of the robot, an external force sensor, or the like.

The internal control device 10 operates the active joint 123. The internal control device 10 operates the active joint 123 in response to a target torque or a position command received by the I / F unit 11.

The measuring device 133 receives various signals (data) indicating the detection result from the detection mechanism 124 such as a sensor.

The control unit 134 performs various calculations such as calculation of a control target value. The operations performed by the control unit 134 for the operation of the exoskeleton are described in the following public documents in addition to Patent Document 2 and Patent Document 3.

Known documents 2) B. Ugurlu, P. Forni, C. Doppmann, and J. Morimoto, “Torque and variable stiffness control for antagonistically driven pneumatic muscle actuators via a stable force feedback controller,” in Intelligent Robots and Systems (IROS), 2015 IEEE / RSJ International Conference on, Sept-Oct 2015, p. In press.
The output device 135 outputs a control signal to the active joint 123. The output device 135 outputs, for example, a target air muscle pressure value or a motor control value to the active joint 123. More specifically, such a control value is output to a motor driver, a pressure control valve, or the like, and physical energy (compressed air or current) is supplied to the motor or air muscle.
[Classification process of brain wave pattern]
In this embodiment, an asynchronous and adaptive BMI based on a dry wireless electroencephalograph headset will be described in order to induce squat movement of the exoskeleton by imagining the movement of the foot.
Such asynchronous and adaptive BMI can reduce the setup time of BRI and mitigate the effects of in-session non-stationarity.

Here, the "asynchronous BMI" refers to a BMI that does not have an external stimulus such as a visual stimulus, and that operates at a timing intended by the BMI user himself / herself.

(Asynchronous adaptive feature extraction to mitigate non-stationarity)
The following describes an adaptive approach for ERD-based BMI to decode foot motion imagination from highly non-stationary electroencephalogram (EEG) signals during asynchronous processing.
The electroencephalogram component for detecting the movement imagination is not limited to ERD, and may be event-related synchronization (ERS) related to movement, or another brain wave component. There may be.

Asynchronous systems continuously analyze EEG signals during both rest and exercise imagination periods, allowing users to make exercise decisions at their own pace.
Such a design is complicated by the fact that asynchronous systems have to deal with a very wide range of uncontrolled states (ie resting states) that do not have motion imagination.
The following known documents 3 and 4 disclose a technique in which a classifier using adaptive linear discriminant analysis (LDA) and an adaptive spatial filter are combined.

Known Documents 3) W. Wojcikiewicz, C. Vidaurre, and M. Kawanabe, “Improving classification performance of bcis by using stationary common spatial patterns and unsupervised bias adaptation,” in Hybrid Artificial Intelligent Systems, ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2011, vol. 6679, pp. 34-41.
Known documents 4) R. Tomioka, J. Hill, B. Blankertz, and K. Aihara, “Adapting spatial filtering methods for nonstationary bcis,” transformation, vol. 10, p. 1, 2006.
However, these prior arts only disclose methods for adaptation in "synchronous" BMI, which, as they are, are insufficient for asynchronous BMI.

Therefore, in the following embodiment, the following changes are mainly made to these synchronous BMIs.

First, the fixed adaptive parameters (forgetting factors) used in synchronous BMI conform to asynchronous BMI, which must be adapted frame by frame using a sliding window approach. Absent. Therefore, in the present embodiment, a method of determining the adaptive parameter so as to conform to the asynchronous BMI is used.

Second, in the conventional method, the bias term of the classifier of the linear discriminant analysis (LDA) is adapted based on the new input data, whereas in the algorithm of the present embodiment, the bias of the classifier is adjusted. The average of the input features is directly adapted by subtracting the moving average of the features from the input features without change. Such adaptation helps to extract consistent features, regardless of the non-stationarity of the EEG, or to provide the user directly with adaptive features as feedback.
(Overview of the system)
FIG. 5 is a functional block diagram illustrating a configuration for executing an electroencephalogram pattern classification process.

In the electroencephalograph 300, the EEG signal has 19 channels (F7, Fp1, Fp2, F8, F3, Fz, F4, C3, Cz, P8, P7, Pz, P4, T3, P3, O1, O2, C4, Collected by a dry wireless headset with T4) as well as a reference signal and ground potential.
The measurement data from the electroencephalograph 300 is transmitted by Bluetooth (registered trademark), transmitted to the interface 202, and stored in a buffer in the interface 202.

As will be described later, the EEG signal is processed by the electroencephalogram pattern analysis unit 40 for each signal in the sliding window, and is analyzed asynchronously and adaptively based on a model trained in advance to detect foot movement imagination. To.
In the online operation of the electroencephalogram pattern analysis unit 40, when the decoder output reaches the threshold value, a trigger signal is transmitted to the exoskeleton type robot 1 to start the squat movement.

(Adaptive feature extraction and classification for classification of EEG patterns)
The electroencephalogram pattern analysis unit 40 is used in an asynchronous mode to detect foot movement imagination, and the two classes to be classified are the uncontrolled state and the movement imagination state.

In order to find a spatial filter that maximizes the difference in variance between these two classes, this embodiment is further improved based on the conventional robust Common Spatial Patterns (CSP) algorithm. Use the algorithm that was used.

Such conventional CSP algorithms are disclosed in the following documents.

Known Documents 5) X. Yong, RK Ward, and GE Birch, “Robust common spatial patterns for eeg signal preprocessing.” Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference, vol. 2008, pp. 2087-2090, 2008.
Here, as described below, by performing adaptive weighting processing on the covariance matrix of the electroencephalogram signals acquired from electrodes arranged in a plurality of parts on the user's scalp, the feature amount Since the extraction is performed, such a weighting process is called a "spatial filter".

Hereinafter, first, a conventional robust CSP algorithm will be briefly described based on the contents of the publicly known document 5.

(Robust CSP algorithm)
The CSP algorithm is often used as a method for extracting features in the classification problem of two classes (C ₁ or C ₂ ) of BMI based on rhythmic modulation by motion images.

For example, the motion imaginary state is class C ₁ and the uncontrolled state is class C ₂ .

Here, it is assumed that the number of channels for EEG measurement is p, N is the number of samples included in one trial, and M is the number of training sets. The set S of signals for each trial is defined by the following equation.

At this time, the optimization problem in CSP is expressed by the following equation.

Where C ₁ represents all class C ₁ EEG trials and ω is an unknown weight vector or matrix of spatial filters.

The var function can be calculated as follows.

Here, Λ ₁ and Λ ₂ are covariance matrices of EEG signals belonging to class C ₁ and class C ₂ , respectively.

The CSP algorithm can be summarized as follows;
1) Perform whitening conversion on the matrix Λ = (Λ ₁ + Λ ₂ ).

That is, by eigenvalue decomposition of the matrix Λ = (Λ ₁ + Λ ₂ ), the matrix P (whitening matrix) satisfying the following conditions is found, and the matrix Λ is whitened.

Here, the following relationship holds.

2) The orthogonal matrix R and the diagonal matrix D are calculated by eigenvalue decomposition so as to satisfy the following conditions.

3) for the class C ₁ and Class Class C ₂ is the line of projection matrix W = R ^T P, ω is as follows.

Then, the covariance matrix is multiplied by such a spatial filter (Common Spatial Pattern) ω and projected, and the value of the squared average of z _p (t) (or its log scale value) is calculated. , Can be used as a feature quantity. In the following, the log scale will be used as the feature quantity.

Therefore, for example, the following values can be used as this feature amount.

As the feature quantity, it is also possible to use the variance of z _p (t), which is the result of projection, or the standardized value (or its log scale value) of the variance of z _p (t).

Then, by performing linear discriminant analysis on such features, a classifier can be generated and class classification can be executed.

Further, in order to improve the robustness of such a classifier, as a method for obtaining a robust evaluation value of a covariance matrix, the M-estimater method, the MVE (Minimum Volume Ellipsoid) method, the MCD (Minimum Covariance Determinant) method, etc. The S-estimater method or the like can be used.

Among these methods, for example, the MCD method is disclosed in the following documents.

Known Documents 6) PJ Rousseeuw, “Least medians of squares regression,” Journal of American Statistical Association, vol. 79, pp. 851-857, 1984.
Known Documents 7) PJ Rousseeuw and KV Driessen, “A fast algorithm for the minimum covariance determinant estimator,” Technicals, vol. 41, no. 3, pp. 212-223, 1999.
(Asynchronous and adaptive CSP processing for classification of motion-imagined EEG signals)
Based on the above assumptions, the "adaptive CSP algorithm" in the present embodiment will be described.

(Spatial filter learning process)
First, the learning process of the CSP algorithm of the present embodiment by the training set will be described with reference to FIG. The data of such a training set is stored in the memory 204, and the arithmetic unit 210 executes the following learning process.

During the learning period, under the control of the CSP learning processing unit 422, the filtering unit 402 bandpass filters (7-30Hz) the EEG signal of the training set for signal processing in the asynchronous and adaptive CSP. For the foot movement imagination class, for example, for the start of the movement imagination, an event for 3 seconds from 0.5 seconds to 3.5 seconds is cut out.
On the other hand, under the control of the CSP learning processing unit 422, the filtering unit 402 cuts out an event from the end of the motion imagination, for example, from 2 seconds to 7 seconds later, for a longer non-control class.

The EEG signal is processed in 0.5 second steps, for example, by a 0.5 second long sliding window. Under the control of the CSP learning processing unit 422, the corresponding projection matrix W is calculated for the training set.

Further, under the control of the CSP learning processing unit 422, the adaptive whitening processing unit 404 is not particularly limited, but the first two rows and the last two rows of the projection matrix W representing the most identifiable spatial filter are selected. ..

In the online processing that actually classifies the brain activity pattern of the user and controls the robot in real time, in the asynchronous and adaptive CSP processing, as explained below, the spatial filter is a feature selection algorithm. May be further reduced by.

That is, for the training set, the sliding window from 2s to 6.5s from the start of non-control is considered to be the non-control class, while the sliding window from 0s to 4.5s from the exercise imagination is the foot. It is considered to be an exercise imagination class.
To remove the filter-related features of the CSP that overfit the training set, run the forests of trees algorithm on the training set to calculate the importance of the features.
After that, when N is the total number of features, the features whose importance is greater than 1 / N are selected.
Such forests of trees algorithm is disclosed in the following documents.

Known Documents 8) P. Geurts, D. Ernst, and L. Wehenkel, “Extremely randomized trees,” Machine learning, vol. 63, no. 1, pp. 3-42, 2006.
Then, using the feature amount before making the above selection or the feature amount after making the above selection depending on the situation, the classifier learning processing unit 424 uses logistic regression classification for class classification. Train the vessel.

The logistic function generates a continuous probability output between 0 and 1. For example, a value closer to 0 corresponds to a non-control class, and a value closer to 1 corresponds to a foot movement imagination class.

Training sets may be used according to the all-or-nothing approach to adjust the best threshold for classification.

The all-or-nothing approach is disclosed in the following literature.

Known Documents 9) A. Muralidharan, J. Chae, and DM Taylor, “Extracting attempted hand movements from eegs in people with complete hand paralysis following stroke.” Frontiers in neuroscience, vol. 5, p. 39, Mar 2011.
Known documents 10) G. Lisi and J. Morimoto, “EEG single-trial detection of gait speed changes during treadmill walk,” PLoS ONE, vol. 10, no. 5, p. E0125479, 05 2015.
In the all-or-nothing approach, specifically, the entire trial (ie, a set of consecutive sliding windows belonging to the same class) is represented by the most probable sliding window. Here, a probability output is assigned to all trials, and as will be described later, the threshold value adjustment unit 426 selects the best threshold value by calculating the receiver operating characteristic (ROC) and its best operating point. To do.

(Online processing)
FIG. 6 is a flow chart for executing an asynchronous and adaptive CSP algorithm online, corresponding to FIG.

The data set input online is also stored in the memory 204, and the arithmetic unit 210 executes the following online processing.

With reference to FIGS. 5 and 6, once the spatial filter has been calculated during the learning period, even in online processing, the raw EEG signal is 0.5 seconds by a sliding window with a length of 0.5 seconds. Processed in steps.
The signal in the sliding window is bandpass filtered between 7 and 30 Hz by the filtering unit 402 for a predetermined period as in the learning period (S100).

However, for online processing, spatial filter adaptation processing as described below is executed in order to reduce the effects of unsteady state.

That is, the adaptive whitening processing unit 404 adapts the projection matrix as follows (S102).

Here, Σ _new ^{-1 / 2} is the whitening matrix for the EEG signal input in the current sliding window, and Σ _old ^1/2 is the inverse matrix of the whitening matrix calculated for the training set. Is.

In order to calculate Σ _new ^{-1 / 2} , it is necessary to evaluate the covariance matrix for the input EEG signal online.

To do this, we calculate the exponential moving average of the covariance matrix as follows.

Here, t is the number of trials, Σ is the online evaluation of the covariance matrix, and ν is the forgetting coefficient.

Also, C (t), when assumed to be EEG signal in the current attempts to S, a C (t) = SS ^T.

Since the analysis by W. Wojcikiewicz et al. Disclosed in Known Document 3 had a synchronous property, a constant value forgetting coefficient (for example, ν = 0.005) was used.
In the present embodiment, in order to handle asynchronous processing, the forgetting coefficient is determined as follows based on the time constant of the exponential moving average. ::

Here, ΔT = 0.05 is the step width of the sliding window, and TC is the time constant of the moving average.
Here, in the learning period, depending on the subject, in the exponential moving average described above, for example, n-fold cross-validation is used to select the TC value from among two possible options, 5 seconds or 10 seconds. It is assumed that validation (n is a natural number of 2 or more and is determined using, for example, n = 10). That is, for the training set, the TC value that gives a better classification result is selected by n-fold cross-validation for the output of class discrimination from the classification processing unit 410 described later. However, the moving average time constant TC may be selected from a larger number.

Here, cross-validation is based on LOO (Leave-One-Run-Out) cross-validation, and in n-fold cross-validation, (n-1) trials are used as a training set and one trial is used as a test set. The process to be performed is executed n times while sequentially changing the test set.

In such a setting, t in the equation (2) represents the current frame of the sliding window.
Subsequently, the eigenvalue decomposition of Σ (t) is calculated, and the whitening matrix Σ _new- ^{1 / 2} is obtained as follows.

Here, E and D are an eigenvector matrix and an eigenvalue matrix of Σ (t), respectively.
Subsequently, the feature amount extraction unit 406 multiplies the projection matrix W _adapt in the equation (1) by the sliding window signal of the EEG signal input online, and _projects the feature amount onto the CSP projective space. Is extracted (S104).
(Characteristic adaptation processing)
The adaptation feature processing unit 408 _projects the current frame of the sliding window using the adapted projection matrix W _adapt, and calculates the log intensity of each projection z _p as log (E [z _p ² ]). ..
Subsequently, the adaptive feature processing unit 408 directly adapts the log intensity of the projection by subtracting the moving average (that is, adaptive baseline subtraction) so as to obtain a zero average data set (S106). ..
The adaptive feature processing unit 408 evaluates the adaptive average μ of the projection log intensity by the exponential moving average as follows.

Here, x (t) is the log intensity of the current frame, α is the forgetting coefficient, and for this forgetting coefficient, TC is optimized by cross-validation as in ν in the equation (3). Is converted.
After calculating the moving average, the adaptation feature processing unit 408 calculates the zero average evaluation of the feature at the current time by the following formula.

Using the zero-average feature amount x _zm (t) obtained in the processes up to this point, the classification process described later can be executed.

However, in order to reduce the influence of the artifact, the adaptive feature processing unit 408 adds low-frequency filtering to the zero average feature (x _zm ) by calculating the exponential moving average as follows. May be good.

Here, η is calculated by the equation (3) with TC = 1 second, and xa (t) is an adapted feature.

However, other methods may be used as the low frequency filter processing.

The reason for choosing such a constant time constant is based on the fact that when ERD is used to discriminate motion imagination, ERD is a phenomenon that occurs within a few seconds, and with a smaller TC, attenuation is too fast, so ERD This is because it contains a transient phenomenon that is too early to be regarded as.
(Classification processing and threshold judgment)
In the online processing, the classification processing unit 410 executes the classification processing for the adaptive feature processing unit 408 using the classifier trained in the learning period, and determines whether or not the state belongs to the motion imagination class. Determine (S108).
After that, the threshold value determination unit 412 generates a trigger signal for driving the exoskeleton type robot 1 according to the output of the classifier exceeding a predetermined threshold value, and generates an active joint control command. Output to unit 30 (S110).

If the process is not completed, the process returns to step S100.
(Robot control)
As mentioned above, the exoskeleton robot 1 has active joints that flex and extend the hip, knee and ankle joints driven by PAM (pneumatically driven artificial muscle) and electric motors.

Since the control of such an exoskeleton type robot 1 is described in the above-mentioned document of Ugurlu et al. Of Known Document 2, the outline will be described below.

Assuming that the exoskeleton robot 1 generates the gravity compensation torque required to support the weight of the subject and the robot, the following equation holds:

Here, θ is a joint angle vector, m is a parameter of total pseudo-weight, m _human is the weight of a human, and m _XoR is the weight of an exoskeleton type robot.
The squat motion is realized by modulating the pseudo-weight parameter as follows.

Where t is time, f = 0.5 Hz is the squat frequency, and K = 7 is the manually adjusted squat amplitude.
(Experimental result)
Hereinafter, the experimental results when the above-mentioned system is operated will be described.

First, collection of foot movement imaginary data was performed on three healthy subjects aged 24-29 years.

The past experience of subject's motion imagination is evaluated as several years for subject S1, several weeks for subject S2, and no experience for subject S3.
However, none of the subjects had previously imagined foot movements.
It is noteworthy that he was able to achieve high performance from the very first experiment of foot movement imagination, even if subject S1 had a long-term experience. Therefore, such long experience is not a requirement of the proposed BMI.

During one trial, subjects performed 10 4-second foot movement imaginations with an 8-second interval. For each subject, four doses of data were collected with a few minute break. The EEG signal was collected at a sampling rate of 500 Hz.

FIG. 7 is a diagram for explaining the time variation of the adapted feature amount.

In FIG. 7, by the method described in Embodiment 1, how the adapted features can maintain a constant zero average and reproduce the ERD associated with motion imagination. Is shown.

In FIG. 7, the “basic adaptation” means the forgetting coefficient ν = 0.005 and α = 0.05 throughout the equations (1) to (6) excluding the equation (3). Corresponds to the case where a fixed value is selected. This is essentially equivalent to the method by W. Wojcikiewicz et al. In Known Document 3, and is described as a comparison target for adaptation.

In the proposed adaptation at the bottom of FIG. 7, the peak of the log intensity in the negative direction corresponds to the motion imagination (that is, the gray region portion).
The contrast between segment 1 and segment 2 in FIG. 7 shows that in the proposed adaptation, the ERD of segment 2 is emphasized with respect to segment 1.

Segment 3 shows that unwanted negative peaks (ie, artifacts) are reduced when performing low pass filtering.
In segment 4, the strong non-stationarity results in a decrease in the raw feature average. However, in this case, although the proposed adaptation cannot reproduce the ERD, it shows that the feature mean is close to zero without being erroneously positive.
FIG. 8 is a diagram showing the relationship between the output of the classifier and the threshold value.

As shown in FIG. 8, all trials are assigned a stochastic output from the classifier.

At the time of learning, the threshold value adjusting unit 426 can select the best threshold value by calculating the receiver operating characteristic (ROC) and its best operating point (upper part in FIG. 8). Graph).
To evaluate the pseudo-online performance of the decoder, the thresholds derived by the all-or-nothing approach are used to classify the signals coming in online (bottom in Figure 8). Graph).

Here, "pseudo" means a state in which only signal processing related to the classification of brain waves is performed without moving the robot.

Thus, it can be seen that only one sliding window per trial is sufficient to consider the entire trial as a false or true trigger.

FIG. 9 is a diagram illustrating a state of robot control.

FIG. 9 shows a state in which the robot squats the subject S1 by imagining the foot movement.
The entire uncontrolled period (ie 0-8s) was correctly classified.
The time required to trigger the trigger was 16 seconds after the start of motion imagination (the shaded bar in the figure was displayed).

In this case, the subject reported that his first attempt at exercise imagination failed and therefore he had to relax once and then imagine the foot exercise again.
Therefore, the response time in FIG. 9 indicates two cycles of squat movement.
FIG. 10 is a diagram showing the result of cross-validation when pseudo-online processing is performed.

Quantitative pseudo-online performance of the BMI system was evaluated for each subject by calibration data collected separately.
The pseudo-online performance of the cross-validated all-or-nothing decoder is shown in the table of FIG. 10 as the ROC and Area Under the Curve (AUC).
Based on the AUC, the class distinctiveness in the test set gave good results for subjects S1 and S2 (0.8 ≤ AUC ≤ 0.89), while it was acceptable for subjects S3 (8 ≤ AUC ≤ 0.89). 0.7 ≤ AUC ≤ 0.79).
In addition, the first two subjects have an accuracy of 70% or higher during online processing, which is a value normally considered necessary for BMI communication. FIG. 11 is a diagram showing for each subject an automatically selected spatial pattern over each segmentation process of cross-validation.
For subject S1, the spatial filter constitutes the activity in the cortex of the motor cortex, suggesting that the foot movement imagination of the muscular motor sensation was accurately performed.
The scalp map of subject S2 corresponds to activity in the cortex of the occipital region, indicating that the subject visually imagined movement rather than muscular motor sensation.
For subject S3, the extracted spatial filters were more unstable for each round of cross-validation, with activity in the parietal and occipital cortex in the first two, or in the sensorimotor cortex in the other two. Is shown.
This is considered to indicate the reason why the pseudo online performance is generally poor for the subject S3.
FIG. 12 is a diagram showing a sliding window output of one trial of subject S1.

As expected, the probabilistic output of the classifier's logistic function based on adaptive features corresponds to the higher probability of motion imagination, as the negative peak of log intensity (ie ERD) in the upper part of FIG. 12 corresponds to ERD. It is a good representation of detection.
When the probability output reaches the threshold value, it is detected that the foot movement imagination is being performed.

FIG. 13 compares the pseudo-online performance of cross-validated all-or-nothing with the three adaptation methods.

The three adaptation methods are the proposed adaptation (proposed in the figure), the basic adaptation (Basic in the figure), and no adaptation process (No Adapt in the figure). ).

FIG. 13 shows that the average pseudo-online performance of the proposed adaptation is highest across multiple subjects.
This indicates that even in-session analysis needs to implement an adaptive BMI algorithm to address EEG unsteadiness.

As described above, in the electroencephalogram pattern classification device, the electroencephalogram pattern classification method, and the electroencephalogram pattern classification program of the present embodiment, by using an adaptive algorithm, the feature amount from the electroencephalogram signal is unsteady in an asynchronous state. It is possible to cope with the steadyness and improve the classification performance of the classifier.

Further, in the neurofeedback system of the present embodiment, the user imagines the movement of a part of his / her body, for example, the foot, and the sensory movement is in the form of proprioceptive afferent input from the movement of the exoskeleton robot. Rhythm feedback is given. Here, "proprioceptor" means acting on sensory receptors that convey information on body position and movement, and "afferent" means transmitting stimuli and excitement from the periphery to the central nervous system. .. Thus, for example, stroke patients can gradually learn motor imagination during training and at the same time induce neuronal plasticity, which can lead to restoration of motor control.

Moreover, such a brain activity measurement system is a plug-and-play system, which is a co-adapted BRI system that can be set up quickly.

That is, in the neurofeedback system of the present embodiment, by using the above-mentioned electroencephalogram pattern classification method, the user's sensory movement imagination classification is performed with high reliability in a short setup time, and the movement by the assist robot is performed. Supportive rehabilitation can be carried out.
[Embodiment 2]
FIG. 14 is a conceptual diagram showing learning and online processing of the electroencephalogram pattern analysis unit 40 according to the first embodiment.

FIG. 15 is a conceptual diagram showing learning and online processing of the electroencephalogram pattern analysis unit 40 according to the second embodiment.

First, as shown in FIG. 14, in the first embodiment, a sufficient amount of a certain number of data (datasets Data1, Data2, Data3 in the figure) for learning were acquired and accumulated from the electroencephalograph 300. Later, the learning process of the electroencephalogram pattern analysis unit 40 is executed using all of these accumulated data.

Then, using the electroencephalogram pattern analysis unit 40 that has completed such learning processing, analysis of the electroencephalogram pattern is a matter of online processing.

On the other hand, in the second embodiment shown in FIG. 15, before a sufficient amount of data for learning is accumulated from the electroencephalograph 300 (in the figure, when the acquisition of the data set Data1 is completed). , The learning process of the electroencephalogram pattern analysis unit 40 is started based on the accumulated data.

Regarding the electroencephalogram pattern analysis unit 40 for which the learning process for the data set Data1 has been completed, the data obtained by combining the next data set Data2 acquired in parallel with the learning process and the acquired data set Data1 is further described as follows. The parameters are updated by executing the learning process in parallel with the acquisition process of the data set of.

Hereinafter, in the same manner, the learning process of the electroencephalogram pattern analysis unit 40 is sequentially executed in parallel with the data acquisition of the next data set.

With the above configuration, it is possible to start the electroencephalogram pattern classification process in a shorter setup time as compared with the first embodiment.

Therefore, by using such an electroencephalogram pattern classification process, it is possible to start rehabilitation by the neurofeedback system in a shorter setup time.

The embodiments disclosed this time are examples of configurations for concretely implementing the present invention, and do not limit the technical scope of the present invention. The technical scope of the present invention is indicated by the scope of claims, not the description of the embodiment, and includes modifications within the scope of the wording of the claims and the scope of the equivalent meaning. Is intended.

1 exoskeleton type robot, 10 internal control device, 11 I / F part, 12 exoskeleton, 20 external control device, 30 active joint control command generation part, 40 brain wave pattern analysis part, 121 base, 122 lower body, 123 active joint, 124 Detection mechanism, 131 Recording processing unit, 132 Storage device, 133 Measuring device, 134 Control unit, 135 Output device, 402 Filter processing unit, 404 Adaptive whitening processing unit, 406 Feature extraction unit, 408 Adaptive feature processing unit, 410 Classification processing unit, 412 threshold value judgment unit, 420 learning processing unit, 422 CSP learning processing unit, 424 classifier learning processing unit, 426 threshold value adjustment unit.

Claims (8)

It is an electroencephalogram pattern classification device for asynchronously classifying electroencephalogram patterns based on the subject's motion imagination.
An electroencephalogram measuring means for acquiring a plurality of channels of electroencephalogram signals for detecting the activity of the subject's brain at a plurality of locations on the subject's scalp.
A filter means for extracting a component of a predetermined frequency band induced by motion imagination by a subject from each of the signals of the plurality of channels.
From the components of the frequency band of the signals of the plurality of channels extracted by the filter means, a feature amount calculation means for adaptively calculating the feature amount by the spatial filter by processing for each sliding window along the time. Prepare,
The feature amount calculation means is
Spatial filter updating means for adaptively updating the spatial filter derived from the covariance matrix of the multi-channel electroencephalogram signals up to the present time by calculating the exponential moving average of the covariance matrix.
The oblivion coefficient determining means for determining the oblivion coefficient in the exponential moving average is included based on the ratio of the step width of the sliding window to the time constant of the exponential moving average.
An electroencephalogram pattern classification device including a classification processing means for outputting a signal indicating that the subject has performed the motion imagination according to the feature amount by a classification process based on the feature amount from the feature amount calculation means. The derivation of the spatial filter is based on the CSP (Common Spatial Patterns) method.
The spatial filter updating means is
Adaptive obtained by multiplying the projection matrix in the CSP method derived in the training set by the whitening matrix for the current EEG signal and the inverse matrix of the whitening matrix calculated for the training set during the learning period. The electroencephalogram pattern classification device according to claim 1, further comprising an adaptive whitening processing means for calculating the updated spatial filter from a projection matrix. The oblivion coefficient determining means determines the time constant from among a plurality of candidates for the time constant of the exponential moving average by n-fold cross-validation (n: a natural number of 2 or more) with respect to the output of the classification processing means. The electroencephalogram pattern classification device according to claim 1 or 2. Any one of claims 1 to 3, wherein the feature amount calculating means further includes an adaptive feature processing means for converting the feature amount derived by the updated spatial filter into a feature amount having a zero average. The electroencephalogram pattern classification device described in the section. The electroencephalogram pattern classification device according to any one of claims 1 to 4, wherein the electroencephalogram measuring means includes a wireless headset using a dry electrode. Based on the signal from the electroencephalogram measuring device for acquiring the electroencephalogram signal of multiple channels for detecting the activity of the brain of the subject at a plurality of places on the scalp of the subject, the subject is subjected to the computer having a computing device and a storage device. It is a brain wave pattern classification method for executing the process of asynchronously classifying the brain wave patterns by the motion imagination of.
Each of the signals of the plurality of channels is stored in the storage device, and the arithmetic unit extracts components of a predetermined frequency band induced by motion imagination by the subject by filtering from the stored signals of the plurality of channels. Steps and
The arithmetic unit adaptively calculates the feature amount by the spatial filter from the components of the frequency band of the signals of the plurality of channels extracted in the step of extracting by the filtering by the processing for each sliding window along the time. With steps
The step of calculating the feature amount is
A step of adaptively updating the spatial filter derived from the covariance matrix of the multi-channel EEG signals up to the present time by calculating the exponential moving average of the covariance matrix.
Including a step of determining the forgetting coefficient in the exponential moving average based on the ratio of the step width of the sliding window to the time constant of the exponential moving average.
The calculation device outputs a signal indicating that the subject has performed the motion imagination according to the feature amount by the classification process based on the feature amount calculated in the step of calculating the feature amount. EEG pattern classification method to prepare. Based on the signal from the electroencephalogram measuring device for acquiring the electroencephalogram signal of multiple channels for detecting the activity of the brain of the subject at a plurality of places on the scalp of the subject, the subject is subjected to the computer having a calculation device and a storage device. It is a brain wave pattern classification program for executing the process of asynchronously classifying the brain wave patterns by the motion imagination of.
The program
Each of the signals of the plurality of channels is stored in the storage device, and the arithmetic unit extracts components of a predetermined frequency band induced by motion imagination by the subject by filtering from the stored signals of the plurality of channels. Steps and
The arithmetic unit adaptively calculates the feature amount by the spatial filter from the components of the frequency band of the signals of the plurality of channels extracted in the step of extracting by the filtering by the processing for each sliding window along the time. With steps
The step of calculating the feature amount is
A step of adaptively updating the spatial filter derived from the covariance matrix of the multi-channel EEG signals up to the present time by calculating the exponential moving average of the covariance matrix.
Including a step of determining the forgetting coefficient in the exponential moving average based on the ratio of the step width of the sliding window to the time constant of the exponential moving average.
The calculation device outputs a signal indicating that the subject has performed the motion imagination according to the feature amount by the classification process based on the feature amount calculated in the step of calculating the feature amount. A brain wave pattern classification program that causes a computer to perform the processing to be prepared. It is a neurofeedback system that supports the movement of the subject in response to the detection of the movement imagination in the brain of the subject.
The electroencephalogram pattern analyzer is provided with an electroencephalogram pattern analyzer for asynchronously classifying the electroencephalogram patterns based on the motion imagination in the subject's brain.
An electroencephalogram measuring means for acquiring a plurality of channels of electroencephalogram signals for detecting the activity of the subject's brain at a plurality of locations on the subject's scalp.
A filter means for extracting a component of a predetermined frequency band induced by motion imagination by a subject from each of the signals of the plurality of channels.
From the components of the frequency band of the signals of the plurality of channels extracted by the filter means, a feature amount calculation means for adaptively calculating the feature amount by the spatial filter by processing for each sliding window along the time. Prepare,
The feature amount calculation means is
Spatial filter updating means for adaptively updating the spatial filter derived from the covariance matrix of the multi-channel EEG signals up to the present time by calculating the exponential moving average of the covariance matrix.
It has a forgetting coefficient determining means for determining the forgetting coefficient in the exponential moving average based on the ratio of the step width of the sliding window to the time constant of the exponential moving average.
A classification processing means that outputs a signal indicating that the subject has performed the motion imagination according to the feature amount by the classification process based on the feature amount from the feature amount calculation means.
Includes a threshold value determining means that generates a trigger signal when the output of the classification processing means exceeds a predetermined threshold value.
A neurofeedback system including an assist robot that supports the movement of the target person in response to the trigger signal.