KR20220098533A - Data acquisition method for brain-computer interface based on speech imagery using ear-eeg and the system thereof - Google Patents

Abstract

The present invention relates to a data acquisition method for a brain-computer interface based on a speech image using ear-EEG and a system using the same. The method comprises the steps of: using an audio image to measure ear-EEG data (ear-EEG) using an ear-peripheral electrode and measuring scalp-EEG data (scalp-EEG) using a scalp-peripheral electrode; extracting an ear-EEG feature matrix and a scalp-EEG feature matrix by decomposing the ear-EEG data and the scalp-EEG data into EEG frequency bands; and acquiring result data of the scalp-EEG, the ear-EEG, and the ear-scalp EEG for voice image monitoring in a brain-computer interface by mapping the ear-EEG feature matrix and the scalp-EEG feature matrix using an extreme learning machine (ELM) model. The present invention is to acquire the result data for the voice image monitoring in the brain-computer interface.

Description

DATA ACQUISITION METHOD FOR BRAIN-COMPUTER INTERFACE BASED ON SPEECH IMAGERY USING EAR-EEG AND THE SYSTEM THEREOF

The present invention relates to a data acquisition method and system for a brain-computer interface based on a voice image using an ear-EEG, and more particularly, to an ear-EEG acquisition method for image monitoring as an alternative to the scalp-EEG acquisition method. will be.

Brain-Computer Interface (BCI) has been extensively studied as an alternative method of communication and control by converting a user's brain activity into computer commands. Numerous studies have demonstrated that BCI can successfully help patients with neurological disorders regain the ability to lead a normal life. However, BCI is not yet suitable for everyday life, especially in normal healthy people. However, apart from its effect, BCI for daily life should be convenient, easy, out of fashion, and in harmony with daily life activities. The BCI paradigm reported in many studies is limited by the mode of BCI. For example, steady-state BCIs, such as reactive BCI and Steady State Visually Evoked Potential (SSVEP), require external stimuli, rendering them inappropriate in daily life. BCI using motor imagery (MI) does not require stimulation, but is limited by degrees of freedom when used for control and is therefore not suitable for communication. In addition, the MI-based BCI may be difficult to associate with a user's desired task (eg, turning on a TV) and thus may not be intuitive depending on the situation.

To overcome these limitations, speech images have been studied and proposed as an alternative mode of BCI. Speech Imagery (SI) is a type of mental task in which the user imagines speaking aloud without actually moving the connector or speaking. Speech image-based BCI can be intuitive compared to other types of BCI in that the user can simply think of a word related to an output command that the system will detect. Also, working with speech images requires less training time as most people are already naturally accustomed to it, and theoretically supports as many commands as there are words.

Much of the early speech image research was based on imagining vowels. For example, existing studies suggest the idea of a speech image by examining the potential elicited from the imagination of vowel /a/, or by using Common Spatial Patterns (CSP) in the collected EEG signal, subjects A/, /u/ and electroencephalogram (EEG) classifications obtained when imagining resting states were performed. Similarly, previous studies used a Support Vector Machine (SVM) and a Relevance Vector Machine (RVM) to classify EEGs from the imagination of Japanese vowels. In addition, existing studies used Hilbert spectral analysis to classify syllables /ba/ and /ku/ into different rhythms, or to classify syllables including imagined syllables (/ba/) in different rhythms.

In addition to the above, recently, voice images using words with semantic meanings are also being studied. For example, existing studies have utilized the Riemannian manifold characteristics used for classification of short, long, and vowels that are periodically imagined in a fixed rhythm, or five words (go, back, left, right, stop). has been used to perform multi-class classification and achieves an accuracy of up to 40.30%. In addition, existing studies have focused on data classification from imagining five Spanish words (up, down, left, right, select) by clustering on four words and classifying all five words using the learned cluster. Transfer learning was applied, or both voice and visual image experiments were performed on 13 different words and images.

However, most of the studies as described above were performed using conventional EEG acquisition methods to acquire EEG from the user's scalp using an electrode with an electrolyte gel or electrically conductive paste (ie, a wet electrode) in the cap. This method provides high-quality EEG signals with a wide range of EEG channels covering all parts of the human brain that makes this EEG acquisition technique the best when it comes to the accuracy of the BCI system. Nevertheless, scalp-based EEG has three major concerns: (1) it takes time to prepare the equipment and requires additional assistance from others, (2) caps and wet electrodes make them uncomfortable, and (3) they are not fashionable and can be socially awkward. It is not suitable for BCI intended for use in everyday life for several main reasons. To address this problem, the researchers attempted to change the design of the EEG acquisition tool so that the wet electrode could be replaced and worn with a different type of electrode. Examples of commercially available wearable EEG acquisition devices include NeuroSky, which uses a single active dry electrode to acquire EEG from a user's forehead, and Emotiv, which uses a saline solution and disposable sponge electrodes.

Ear-EEG is an alternative EEG acquisition method that is gaining popularity in the field of BCI research due to its convenience, mobility, and prudence. This EEG acquisition method measures the EEG around the user's ear. Ear-EEG does not require complicated equipment preparation, and since the sensor does not come into contact with the user's hair, it is easier and more convenient for the user to use than the conventional scalp-EEG method. In addition, electrode placement in the ear-EEG method also makes them invisible to others and makes this method very careful and does not cause unwanted attention in daily life.

Accordingly, the present invention proposes a voice image-based BCI system using an ear-EEG to promote the development of BCI in daily life.

Looney D, Kidmose P, Park C, Ungstrup M, Rank ML, Rosenkranz K, Mandic DP. The in-the-ear recording concept: User-centered and wearable brain monitoring. IEEE pulse. 2012 Dec 12;3(6):32-42.

An object of the present invention is to obtain an ear-EEG through a voice image-based BCI system by making it possible to wear a low-cost ear-EEG acquisition device (eg, a wearable device), and scalp-EEG using a scalp-peripheral electrode We want to acquire the data and obtain the result data for voice image monitoring in the brain-computer interface.

In the data acquisition method for a brain-computer interface (BCI) based on speech-imagery using the ear-EEG according to an embodiment of the present invention, by the voice image, the ear-peripheral electrode Measuring ear-EEG data (ear-EEG) using, scalp-EEG data (scalp-EEG) using peripheral electrodes, the ear-EEG data and the scalp-EEG data to EEG Extracting an ear-EEG characteristic matrix and a scalp-EEG characteristic matrix by decomposing it into frequency bands, and mapping the ear-EEG characteristic matrix and the scalp-EEG characteristic matrix using an ELM (Extreme Learning Machine) model to a brain-computer and acquiring result data of scalp-EEG, ear-EEG, and ear-scalp EEG for voice image monitoring in the interface.

In the data acquisition system for the brain-computer interface (BCI) based on speech-imagery using the ear-EEG according to an embodiment of the present invention, by the voice image, the ear-peripheral electrode Measuring ear-EEG data (ear-EEG) using a measurement unit that measures scalp-EEG data (scalp-EEG) using scalp-peripheral electrodes, the ear-EEG data and the scalp-EEG data The ear-EEG characteristic matrix and the scalp-EEG characteristic matrix are mapped using an extractor that extracts the ear-EEG characteristic matrix and the scalp-EEG characteristic matrix by decomposing it into EEG frequency bands and an ELM (Extreme Learning Machine) model. - Includes an acquisition unit for acquiring result data of scalp-EEG, ear-EEG, and ear-scalp EEG for voice image monitoring in the computer interface.

According to an embodiment of the present invention, a low-cost ear-EEG acquisition device (eg, a wearable device) can be worn to acquire ear-EEG through a voice image-based BCI system, and scalp-peripheral electrodes are used to By acquiring the scalp-EEG data, it is possible to acquire result data for voice image monitoring in the brain-computer interface.

In addition, according to an embodiment of the present invention, it can be confirmed that the ear-EEG has great potential as an alternative to the scalp-EEG acquisition method for monitoring voice images.

1 is a flowchart illustrating a data acquisition method for voice image-based BCI using an ear-EEG according to an embodiment of the present invention.
2 shows an example of electrode arrangement of ear-EEG and scalp-EEG according to an embodiment of the present invention.
3 illustrates an example of a wearable device for measuring ear-EEG according to an embodiment of the present invention.
4 shows a model structure according to an embodiment of the present invention.
5 is a diagram illustrating an experimental procedure of a voice image task and a rest task according to an embodiment of the present invention.
6 illustrates a procedure of a data acquisition system processing EEG data collection, feature extraction, and classification according to an embodiment of the present invention.
7 is a block diagram illustrating a detailed configuration of a data acquisition system for voice image-based BCI using ear-EEG according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be embodied in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments, and is not intended to limit the present invention. In this specification, the singular also includes the plural, unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly specifically defined.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

The present invention directly compares the performance with the existing 32-channel scalp-EEG setting in a multi-class voice image classification task using a wearable ear-EEG acquisition device to provide a scalp-EEG for voice image monitoring in a brain-computer interface (BCI), Its gist is to acquire an ear-EEG and an ear-scalp EEG.

Accordingly, the present invention measures the EEG in the scalp and the ear simultaneously in a multi-class voice image experiment, directly compares the classification results between the two EEG acquisition methods, and provides ear-EEG characteristics to improve the accuracy of the ear-EEG-based system. is trained to map to the scalp-EEG functional space. At this time, the feature extraction method of the present invention is based on the EEG covariance matrix of the Riemannian framework. In addition, the present invention uses a Multi-Layer Extreme Learning Machine (MLELM) as a classifier.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a data acquisition method for voice image-based BCI using ear-EEG according to an embodiment of the present invention, and FIG. 2 is an ear-EEG and scalp-EEG according to an embodiment of the present invention. An example of the electrode arrangement is shown. Also, FIG. 3 shows an example of a wearable device for measuring ear-EEG according to an embodiment of the present invention, and FIG. 4 shows a model structure according to an embodiment of the present invention.

The method of FIG. 1 is performed by the data acquisition system for voice image-based BCI using the ear-EEG according to the embodiment of the present invention shown in FIG. 7 .

Referring to FIG. 1 , in step S110, ear-EEG data (ear-EEG) is measured using an ear-peripheral electrode by a voice image, and scalp-EEG data (scalp-EEG) is measured using a scalp-peripheral electrode. ) is measured.

In step S110, a wearable device of a C-shaped earphone that wraps around the back of the user's ear is used, and ear-EEG data for a voice image may be measured using an ear-peripheral electrode of a snap wet electrode attached to the wearable device. Also, in step S110, scalp-EEG data for the voice image may be measured using a plurality of scalp-peripheral electrodes attached to the periphery of the user's brain hemisphere.

Referring to FIG. 2 , the present invention measures ear-EEG data through 6 channels, 3 each in both ears of the test subject (or user), as shown in FIG. 2( a ). At this time, the peripheral electrodes of the left ear are L1, L2, and L3, and the peripheral electrodes of the right ear are R1, R2, and R3. The ear-peripheral electrode is grounded with respect to the electrode of the lower right (REF) and left ear (GND), respectively. Moreover, it is characterized in that the length between L1 and R1 and the length between the GND and REF electrodes is 55 mm, and the channels L3 and R3 are perpendicular to the L1-GND and R1-REF lines with a distance of 20 mm.

Ear-EEG data is obtained using a wearable device as shown in FIG. 3 . The wearable device includes a C-shaped earphone made of flexible silicone that wraps around the ear in the form of a horizontal headband that wraps around the back of the user's ear and the back of the user's head. The present invention uses a foam-type snap wet electrode (3M Red Dot) cut to a diameter of 14 mm for a wearable device, and the electrode can be easily attached and detached from a socket built into a silicone earphone. Silicone earphones and foam-type snap electrodes provide a soft touch to the user's skin, making the device comfortable to wear. Moreover, the electrode achieves an impedance value of less than 15 KΩ and maintains the same impedance level for at least 6 hours.

Moreover, the silicone earphones are attached to a 3D printed frame made of ABS material. Wires connect to an EEG sensing board included in a 3D-printed case that can be attached to the user's clothing, which includes a portable battery and charger. In addition, the present invention uses the Cyton Biosensing Board of OpenBCI as an EEG sensing board, and the EEG acquisition sampling rate is 250 Hz.

Referring to Fig. 2, the present invention is scalp-EEG at a sampling rate of 500 Hz using BrainVision actiCHAMP with an EEG cap composed of 32 Ag/AgCl electrodes around the left hemisphere as shown in Fig. 2(b). Acquire data. In this case, Fpz and FCz are selected as the ground channel and the reference channel, respectively.

The present invention attaches scalp-peripheral electrodes only to the left hemisphere of the brain for two main reasons. The first reason is to reduce user fatigue by reducing the number of electrodes to shorten equipment preparation time. The second reason is that the left brain contains Broca's areas (F5, FT7, FC5, FC3) and Wernicke's areas (TP7, CP5, CP3, P5) involved in speech production. Furthermore, the scalp-peripheral electrodes are not placed in channels T9, TP9 and P9 because they are close to the ear-peripheral electrodes. In addition, an electrolyte gel is inserted to ensure the connection between the electrode and the scalp and keep the impedance level below 10 KΩ.

Referring back to FIG. 1 , in step S120 , the ear-EEG data and the scalp-EEG data are decomposed into EEG frequency bands to extract an ear-EEG characteristic matrix and a scalp-EEG characteristic matrix.

In step S120, a notch filter is applied to the ear-EEG data and the scalp-EEG data to remove noise (Data Pre-Processing), divided into EEG epochs, and decomposed into EEG frequency bands, By calculating the tangent vector, a covariance matrix of the ear-EEG feature matrix and the scalp-EEG feature matrix of the feature matrix dimension may be extracted. In this case, in step S120, the covariance matrix is projected onto the corresponding tangent space (Tangent Space Projection) and the tangent vector is constructed so that it can be effectively used as a feature of the classification algorithm.

More specifically, step S120 applies a notch filter having a 60Hz cutoff frequency to the ear-EEG data and the scalp-EEG data to remove noise from the power line. The ear-EEG data and scalp-EEG data are then split into multiple 2-second EEG epochs for each trial from the beginning of the visual cue, and labeled with the corresponding classes. Finally, the EEG epochs are delta (0.5 Hz - 4 Hz), Theta (4 Hz - 7 Hz), alpha (7 Hz - 14 Hz), beta (14 Hz - 30 Hz), gamma (30 Hz - 100 Hz) and 'all' (0.5 Hz - 100 Hz). ) into 5 EEG frequency bands including The first five frequency bands are common EEG frequency bands classified according to their unique characteristics and functions, and the purpose of the ‘all’ bands is to capture and process EEG data as a whole. Accordingly, in step S120, by decomposing the EEG data into different bands and extracting features from the different bands, the relationship between the EEG and the cognitive mechanism of the voice image task can be extracted by analyzing the experimental results.

을 나타낼 수 있으며, 여기서 n은 채널의 수를 나타내고, T는 한 epoch의 데이터 포인트를 나타낸다. 공분산 행렬

은 다음과 같이 정의된다. To explain the process of extracting the ear-EEG characteristic matrix and the scalp-EEG characteristic matrix from the EEG data, the EEG data epoch of each frequency band in the i-th trial is a matrix

, where n denotes the number of channels and T denotes a data point of one epoch. covariance matrix

is defined as

[Formula 1]

In this case, the resulting covariance matrix Pi represents a symmetric positive-definite (SPD) matrix.

에서 접선 공간 벡터

는 하기의 [수식 2]와 같이 정의된다. Since the space of the SPD matrix is in the Riemannian manifold, the present invention cannot effectively use the covariance matrix directly as a feature of the classification algorithm based on projection to the hyperplane. Accordingly, the present invention projects the covariance matrix into the corresponding tangent space and constructs a tangent vector so that it can be effectively used as a feature of a classification algorithm. For each covariance matrix Pi

tangent space vector in

is defined as in [Equation 2] below.

[Equation 2]

가중치를 적용함으로써, 이를 벡터화하는 연산자를 나타낸다. PR은 N 공분산 행렬의 Riemannian 평균을 나타내고, Logc(P)는 [수식 3]과 같이 정의된 기준점 C를 사용한 행렬 P의 로그 매핑을 나타낸다.Here, upper(X) maintains only the upper triangular part of the matrix X and applies a single weight to the diagonal elements to the rest.

By applying a weight, we represent an operator that vectorizes it. PR represents the Riemannian mean of the N covariance matrix, and Logc(P) represents the log mapping of the matrix P using the reference point C defined as in [Equation 3].

[Equation 3]

Then, the present invention calculates the tangent vector separately for each frequency band. Then the characteristic matrix is constructed by connecting all the tangent vectors of each frequency band. The dimension of the feature matrix is (Ns×126) for ear-EEG data and (Ns×2976) for scalp-EEG data, where Ns represents the number of samples. Finally, the ANONA F-value is computed and used to select the k-features that are best suited for classification, concluding that the dimension of the final feature matrix is (Ns×k). At this time, the present invention is k = [1, 10, 20, ... in the case of ear-EEG. , 110], and for scalp-EEG, k = [1, 100, 200, … , 2500] to run the algorithm. The feature selection method can help to reduce the computational cost of the system, and can improve the accuracy of the system.

Referring back to FIG. 1 , in step S130, an ear-EEG feature matrix and a scalp-EEG feature matrix are mapped using an ELM (Extreme Learning Machine) model in the brain-computer interface. Acquire result data of scalp-EEG, ear-EEG, and ear-scalp EEG for voice image monitoring.

Step S130 may be obtained by improving the ear-EEG by mapping the ear-EEG feature matrix to the scalp-EEG feature space using the ELM model. At this time, in step S130, voice image monitoring is performed at the brain-computer interface through an Ear-to-Scalp mapped feature matrix obtained using the scalp-EEG feature matrix of the same sample as the output layer in the ELM model. For scalp-EEG (Scalp-EEG results), ear-EEG (Ear-EEG Results), and ear-scalp EEG (Ear-to-Scalp Results) result data can be obtained.

More specifically, an ELM (Extreme Learning Machine) model is a single layer feed-forward neural network composed of an input layer, a single hidden layer, and an output layer. The ELM model has a very fast training speed due to random initialization of the input weights and bias values, and is suitable for BCI in everyday life applications where the classification model needs to be updated regularly due to the non-static nature of EEG. In addition, the ELM model shows better performance in the speech image-based BCI compared to the general classification method.

및

에서 은닉 계층 출력은 하기의 [수식 4]와 같이 정의된다.i=1, … , N, where j denotes the number of input nodes, and m denotes the number of output nodes, N distinct samples.

and

The hidden layer output is defined as [Equation 4] below.

[Equation 4]

는 입력 가중치를 나타내며,

는 편향을 나타낸다. 출력 계층은 하기의 [수식 5]와 같이 정의된다.where g(x) represents the activation function,

is the input weight,

represents the bias. The output layer is defined as [Equation 5] below.

[Equation 5]

은 출력 계층의 행렬을 나타낸다. N 훈련 샘플을 고려해보면, n개의 은닉 노드를 가지는 ELM 모델은 다음과 같이 구성될 수 있다.here,

denotes the matrix of the output layer. Considering N training samples, an ELM model with n hidden nodes can be constructed as follows.

[Equation 6]

로 계산할 수 있다. 이때, Ht는 행렬 H의 Moore-Penrose 일반화 역을 나타낸다.There are three steps to training an ELM model. The first step is to assign random values (0 and 1) to the input weights a and bias b, the second step is to compute the matrix H, and finally the output weights are

can be calculated as In this case, Ht represents the Moore-Penrose generalization inverse of the matrix H.

One variant of the ELM model is the automatic encoding ELM (ELM-AE). The model structure of the ELM-AE is shown in Fig. 4(a), which shows an unsupervised learning ELM in which the output of the network is the same as the input of the network. At this time, ELM-AE is trained in the same way as general ELM.

Furthermore, the present invention applies the concept of deep learning to ELM as interest in deep learning increases. A Multi-Layer Extreme Learning Machine (MLELM) was constructed using multiple ELM-AEs. MLELM uses multiple ELM-AEs to train the input for each hidden layer, which uses the output weights V learned from the ELM-AE to transfer the input data of each layer to the feature space. The l-th hidden layer of the MLELM model can be expressed as follows.

[Equation 7]

Note that in the first hidden layer (l=1), H0 is the input layer x. The output weight connecting the last hidden layer and the output layer is learned in the same way as the original ELM, and the structure of the MLELM model with k hidden layers is shown in Fig. 4(b).

Accordingly, in FIG. 1 , step S130 may be obtained by improving the ear-EEG by mapping the ear-EEG feature matrix to the scalp-EEG feature space using the ELM model. Specifically, the mapping process is performed using an ELM model. The ear-scalp mapping ELM model is trained in the same way as the classification task, but instead of setting the sample label y as the output layer, it uses the scalp-EEG characteristics of the same sample as the output layer. At this time, the Ear-to-Scalp feature matrix is further processed and classified in the same manner as the ear-EEG feature matrix and the scalp-EEG feature matrix.

5 is a diagram illustrating an experimental procedure of a voice image task and a rest task according to an embodiment of the present invention.

The experimental procedure is described using visual cues at the start of the experiment, and the subjects' ear-EEG and scalp-EEG are simultaneously recorded during the experiment. Each session of the experiment had 10 task blocks, each with four voice image tasks on the words "Right," "Left," "Foward," and "Go back," and asked subjects to relax with their eyes open. control tasks are included in random order.

The experimental steps are different between the voice image task and the control task (the resting state task). The voice image task (Fig. 5(a)) started with an audio signal in which a female voice reads the word with an American accent. Two seconds later, a crosshair signal appeared for one second while instructing the subjects to relax. Then, circle marks were given for 2 seconds, and they were expected to say the previously given word. Then, a cross for 1 second was shown again for relaxation. Because the actual connected voice usually took less than a second, the subjects actually rested for more than a second for the next step. The loading bar was then shown for 2 seconds, and the subject was instructed to imagine the word in an expanded fashion according to the progress of the loading bar. It was shown 5 times in a row, with a crosshair of 1 second in between. Subjects rested for 2.5 seconds before starting the next task.

The control task (Fig. 5(b)) was performed in a similar manner to the voice image task, but instead of an auditory word, a beep sounded and there was no next step in the connected voice. In this task, the subject is instructed to look at the loading bar without imagining a word.

Classification results over 10 subjects through the above-described experimental procedure were 38.2% and 43.1%, respectively, which is a maximum of 43.8%, and 55.0% for scalp-EEG. Analysis of variance (ANOVA) showed that 6 out of 10 subjects showed no significant difference between the performance of ear-EEG and scalp-EEG. Through this, it can be confirmed that the ear-EEG has great potential as an alternative to the scalp-EEG acquisition method for monitoring voice images.

6 shows a procedure of a data acquisition system processing EEG data collection, feature extraction and classification according to an embodiment of the present invention, and FIG. It is a block diagram showing the detailed configuration of a data acquisition system for BCI.

6 and 7 , the data acquisition system 700 according to an embodiment of the present invention may acquire ear-EEG for image monitoring as an alternative to the scalp-EEG acquisition method.

To this end, the data acquisition system 700 according to the embodiment of the present invention includes a measuring unit 710 , an extracting unit 720 , and an acquiring unit 730 .

The measurement unit 710 measures the ear-EEG data (ear-EEG) using the 6 Channels Ear-EEG by the voice image, and the scalp-peripheral electrode (32 Channels Scalp-EEG) to measure scalp-EEG data (scalp-EEG).

The measurement unit 710 uses a wearable device of a C-shaped earphone that wraps around the back of the user's ear, and can measure the ear-EEG data for the voice image using the ear-peripheral electrode of the snap wet electrode attached to the wearable device. have. In addition, the measurement unit 710 may measure the scalp-EEG data for the voice image by using a plurality of scalp-peripheral electrodes attached to the periphery of the user's brain hemisphere.

The extraction unit 720 decomposes the ear-EEG data and the scalp-EEG data into EEG frequency bands to extract an ear-EEG characteristic matrix and a scalp-EEG characteristic matrix.

The extraction unit 720 applies a notch filter to the ear-EEG data and the scalp-EEG data to remove noise (Data Pre-Processing), divides it into EEG epochs, and decomposes it into EEG frequency bands, each frequency By calculating a tangent vector with respect to a band, a covariance matrix of the ear-EEG feature matrix and the scalp-EEG feature matrix of the feature matrix dimension may be extracted. In this case, the extraction unit 720 projects the covariance matrix onto the corresponding tangent space (Tangent Space Projection) and constructs a tangent vector so that it can be effectively used as a feature of the classification algorithm.

The acquisition unit 730 maps an ear-EEG feature matrix and a scalp-EEG feature matrix using an ELM (Extreme Learning Machine) model to monitor voice images in a brain-computer interface. Acquire result data of scalp-EEG, ear-EEG and ear-scalp EEG.

The acquirer 730 may improve and acquire the ear-EEG by mapping the ear-EEG characteristic matrix to the scalp-EEG characteristic space using an ELM model (Classification Model). At this time, the acquisition unit 730 receives the voice from the brain-computer interface through an Ear-to-Scalp mapped feature matrix obtained by using the scalp-EEG characteristic matrix of the same sample as the output layer in the ELM model. Result data of scalp-EEG (Scalp-EEG Results), ear-EEG (Ear-EEG Results), and ear-scalp EEG (Ear-to-Scalp EEG Results) for image monitoring may be acquired.

A data acquisition system 700 according to an embodiment of the present invention is evaluated using 10-fold cross-validation for each session of the experiment. For each iteration of the cross-validation, 225 training samples and 25 test samples were provided, and samples from the same block stayed in the same fold. Tangent space projections and feature selectors are computed using only data from the training sample. In addition, using the grid search method, [50, 60, … , 200], and since the ELM model and its variants use randomized values as their weights and bias values, random seeds are specified such that each run of model training gives the same output.

Although the description is omitted in the system of FIG. 7 of the present invention, the constituent means constituting the system of the present invention may include all the contents described in FIGS. 1 to 6 , which will be appreciated by those skilled in the art. it is self-evident

The system or apparatus described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, the devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (12)

In a data acquisition method for a brain-computer interface (BCI) based on speech-imagery using an ear-EEG,
Measuring the ear-EEG data (ear-EEG) using the ear-peripheral electrode and measuring the scalp-EEG data (scalp-EEG) using the scalp-peripheral electrode by the voice image;
extracting an ear-EEG characteristic matrix and a scalp-EEG characteristic matrix by decomposing the ear-EEG data and the scalp-EEG data into EEG frequency bands; and
Results of scalp-EEG, ear-EEG, and ear-scalp EEG for voice image monitoring in a brain-computer interface by mapping the ear-EEG feature matrix and the scalp-EEG feature matrix using an Extreme Learning Machine (ELM) model Steps to Acquire Data
A data acquisition method comprising According to claim 1,
The measuring step
Using a wearable device of a C-shaped earphone surrounding the back of the user's ear, and measuring the ear-EEG data for the voice image using the ear-peripheral electrode of a snap wet electrode attached to the wearable device, data acquisition Way. According to claim 1,
The measuring step
A data acquisition method of measuring the scalp-EEG data for the voice image using a plurality of the scalp-peripheral electrodes attached to the periphery of the user's brain hemisphere. According to claim 1,
The extraction step
A notch filter is applied to the ear-EEG data and the scalp-EEG data to remove noise, divided into EEG epochs and decomposed into EEG frequency bands, and a characteristic matrix by calculating a tangent vector for each frequency band and extracting the ear-EEG feature matrix and the scalp-EEG feature matrix of dimensions. According to claim 1,
The obtaining step is
The method for acquiring data, characterized in that the ear-EEG is improved by mapping the ear-EEG characteristic matrix to the scalp-EEG characteristic space using the ELM model. 6. The method of claim 5,
The obtaining step is
Scalp-EEG, ear for voice image monitoring in the brain-computer interface through the ear-scalp feature matrix (Ear-to-Scalp) obtained using the scalp-EEG feature matrix of the same sample as the output layer in the ELM model - A method of acquiring data, characterized in that it acquires result data of EEG and ear-scalp EEG. In a data acquisition system for a brain-computer interface (BCI) based on speech-imagery using an ear-EEG,
a measurement unit for measuring ear-EEG data (ear-EEG) using an ear-peripheral electrode, and measuring scalp-EEG data (scalp-EEG) using a scalp-peripheral electrode;
an extraction unit for decomposing the ear-EEG data and the scalp-EEG data into EEG frequency bands to extract an ear-EEG characteristic matrix and a scalp-EEG characteristic matrix; and
Results of scalp-EEG, ear-EEG, and ear-scalp EEG for voice image monitoring in a brain-computer interface by mapping the ear-EEG feature matrix and the scalp-EEG feature matrix using an Extreme Learning Machine (ELM) model Acquisition unit that acquires data
A data acquisition system comprising a. 8. The method of claim 7,
The measurement unit
Using a wearable device of a C-shaped earphone surrounding the back of the user's ear, and measuring the ear-EEG data for the voice image using the ear-peripheral electrode of a snap wet electrode attached to the wearable device, data acquisition system. 8. The method of claim 7,
The measurement unit
A data acquisition system for measuring the scalp-EEG data for the voice image by using a plurality of the scalp-peripheral electrodes attached to the periphery of the user's brain hemisphere. 8. The method of claim 7,
The extraction unit
A notch filter is applied to the ear-EEG data and the scalp-EEG data to remove noise, divided into EEG epochs and decomposed into EEG frequency bands, and a characteristic matrix by calculating a tangent vector for each frequency band extracting the ear-EEG feature matrix and the scalp-EEG feature matrix of dimensions. 8. The method of claim 7,
the acquisition unit
and improving the ear-EEG by mapping the ear-EEG feature matrix to the scalp-EEG feature space using the ELM model. 12. The method of claim 11,
the acquisition unit
Scalp-EEG, ear for voice image monitoring in the brain-computer interface through the ear-scalp feature matrix (Ear-to-Scalp) obtained using the scalp-EEG feature matrix of the same sample as the output layer in the ELM model - A data acquisition system, characterized in that it acquires result data of EEG and ear-scalp EEG.

Images