KR102296838B1 - Motion Intention Discrimination Method and Prosthesis Control Method by Linking EEG Signal and Peripheral Nerve Signal - Google Patents

Abstract

In accordance with an embodiment of the present invention, there is provided a method for determining an operation intention in conjunction with an EEG signal and a peripheral nerve signal, the method comprising: obtaining a peripheral nerve signal and an EEG signal of the user when a user performs a preset operation; learning the EEG signal based on the peripheral nerve signal by a machine learning model; acquiring the EEG signal of the user in real time; and determining the user's intention to operate by using the EEG signal acquired in real time by the machine learning model.

Description

{Motion Intention Discrimination Method and Prosthesis Control Method by Linking EEG Signal and Peripheral Nerve Signal}

The present application relates to a method for determining a motion intention in conjunction with an EEG signal and a peripheral nerve signal and a method for controlling the will.

A prosthesis including a prosthetic arm, a prosthetic leg, and the like is widely used for patients whose body parts have been amputated.

Such a prosthesis can be divided into Myoelectric prosthesis, Brain-machine interface, Targeted muscle reinnervation, Neural control prosthesis, etc. depending on the control method. have.

However, such existing prosthetic control techniques have limitations in that natural control of the prosthesis is difficult, signal processing is complicated, or it is difficult to use non-permanently.

Publication No. 10-2018-0109451 (published on Oct. 8, 2018) Japanese Patent Publication No. 2014-528800 (published on October 30, 2014)

Accordingly, there is a need in the art for a method for overcoming the limitations of the existing will control technology.

In order to solve the above problems, an embodiment of the present invention provides a method for determining the intention of operation in conjunction with the EEG signal and the peripheral nerve signal.

The method for determining an operation intention in conjunction with the EEG signal and the peripheral nerve signal includes: acquiring the user's peripheral nerve signal and the EEG signal when the user performs a preset operation; learning the EEG signal based on the peripheral nerve signal by a machine learning model; acquiring the EEG signal of the user in real time; and determining the user's intention to operate by using the EEG signal acquired in real time by the machine learning model.

In addition, another embodiment of the present invention provides a will control method in conjunction with an EEG signal and a peripheral nerve signal.

The will control method interlocking the EEG signal and the peripheral nerve signal includes: acquiring the user's peripheral nerve signal and the EEG signal when the user performs a preset operation; learning the EEG signal based on the peripheral nerve signal by a machine learning model; acquiring the EEG signal of the user in real time; determining the user's intention to operate by using the EEG signal acquired in real time by the machine learning model; and generating a will control signal based on the determined operation intention of the user.

Incidentally, the means for solving the above problems do not enumerate all the features of the present invention. Various features of the present invention and its advantages and effects may be understood in more detail with reference to the following specific embodiments.

According to an embodiment of the present invention, by learning an EEG signal based on a peripheral nerve signal simultaneously acquired when a user performs a specific operation, it is possible to accurately determine the user's intention of operation thereafter using only the EEG signal. Based on the user's intention, the will can be controlled to operate naturally.

1 is a flowchart of a method for determining an operation intention in conjunction with an EEG signal and a peripheral nerve signal according to an embodiment of the present invention.
FIG. 2 is a detailed flowchart of step S200 shown in FIG. 1 .
3 is a diagram illustrating a process of extracting a first characteristic factor from a peripheral nerve signal according to an embodiment of the present invention.
4 is a diagram illustrating a process of extracting a second characteristic factor from an EEG signal according to an embodiment of the present invention.
5 is a diagram illustrating an example of generating a time-frequency two-dimensional image from an EEG signal.
6 is a flowchart of a will control method in which an EEG signal and a peripheral nerve signal are interlocked according to another embodiment of the present invention.

Hereinafter, preferred embodiments will be described in detail so that those of ordinary skill in the art can easily practice the present invention with reference to the accompanying drawings. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for parts having similar functions and functions.

In addition, throughout the specification, when a part is 'connected' with another part, it is not only 'directly connected' but also 'indirectly connected' with another element interposed therebetween. include In addition, 'including' a certain component means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a flowchart of a method for determining an operation intention in conjunction with an EEG signal and a peripheral nerve signal according to an embodiment of the present invention, and FIG. 2 is a detailed flowchart of step S200 shown in FIG.

Referring to FIG. 1 , first, when the user performs a preset operation, the user's peripheral nerve signal and EEG signal may be acquired ( S100 ). According to an embodiment, a peripheral nerve signal may be obtained by a precipitation electrode invasively fixed to the user's peripheral nerve, and an EEG signal may be obtained by an electrode provided non-invasively on the user's upper head. However, the method for acquiring the peripheral nerve signal and the EEG signal is not limited thereto, and the user's peripheral nerve signal and the EEG signal may be obtained by various methods known to those skilled in the art.

Here, although the peripheral nerve signal has relatively low noise and high resolution, there is a limitation in that it is difficult to permanently use the invasive acquisition of the peripheral nerve signal by the precipitation electrode. On the other hand, EEG signals are non-invasive and permanently obtainable, but have a disadvantage in that they contain a lot of noise.

Therefore, the present invention proposes a technique for analyzing a correlation between a peripheral nerve signal and an EEG signal by linking the two signals by a machine learning model, and then determining an operation intention using only an EEG signal that can be permanently obtained. In other words, the accuracy of EEG signal learning can be further improved by allowing the machine learning model to learn noisy EEG signals based on relatively accurate peripheral nerve signals.

Thereafter, the EEG signal may be learned based on the peripheral nerve signal by the machine learning model (S200).

According to an embodiment, as shown in FIG. 2 , the first and second characteristic factors are extracted from the peripheral nerve signal and the EEG signal ( S210 ), and the corresponding operation is performed using the extracted first and second characteristic factors, respectively. It is possible to generate first and second probabilistic models for (S220), and to analyze the correlation between peripheral nerve signals and EEG signals when the corresponding operation is performed by a machine learning model based on the first and second probabilistic models. can be (S230). This will be described in more detail with reference to FIGS. 3 to 5 .

3 is a diagram illustrating a process of extracting a first characteristic factor from a peripheral nerve signal according to an embodiment of the present invention.

Referring to FIG. 3 , in order to extract the first characteristic factor from the peripheral nerve signal, the DC component may be removed by first performing preprocessing on the peripheral nerve signal ( S211 ).

Thereafter, entropy for each electrode may be extracted from the preprocessed peripheral nerve signal (S212). Specifically, the amount of information is the amount of surprise obtained after an event occurs, and the more unpredictable (that is, the more rarely it appears), the greater the information amount. Entropy is an expected value of this amount of information and can be calculated according to Equation (1). Here, m is a set of factors, M is the number of factors, p represents the probability of occurrence, and x represents a signal.

[Equation 1]

Thereafter, an entropy histogram may be generated based on the entropy of each electrode ( S213 ).

In other words, according to an embodiment, an entropy histogram may be generated from the peripheral nerve signal as the first characteristic factor.

4 is a diagram illustrating a process of extracting a second characteristic factor from an EEG signal according to an embodiment of the present invention.

Referring to FIG. 4 , in order to extract the second characteristic factor from the EEG signal, preprocessing may be performed on the EEG signal ( S214 ). For example, noise may be removed by performing preprocessing by wavelet transform or the like.

Thereafter, a time-frequency two-dimensional image may be generated based on the preprocessed EEG signal (S215). Specifically, short-time Fourier transform (STFT) is applied to the EEG signal acquired for each channel, and the frequency band of the channel related to the kinesthetic cortex on the frequency axis, that is, the mu band (8~13Hz) ), it is possible to generate a time-frequency two-dimensional image by extracting only the bata band (17-30Hz).

5 is a diagram showing an example of generating a time-frequency two-dimensional image from an EEG signal, (a) of FIG. 5 shows a time-frequency analysis result image for an EEG signal obtained from one channel , (b) shows an example in which two-dimensional images obtained from three channels related to the kinesthetic cortex are combined to form one two-dimensional image. In FIG. 5 , the color represents the relative intensity of the amplitude obtained after the Fourier transform in the frequency domain with respect to the reference electrode.

In other words, according to an embodiment, a time-frequency two-dimensional image may be generated from the EEG signal as the second characteristic factor.

Meanwhile, in step S220, in order to generate first and second probabilistic models for a corresponding operation using the extracted first and second feature factors, respectively, for the first feature factor, for example, a Gaussian mixture model is applied to A first probabilistic model may be generated, and a second probabilistic model may be generated by applying, for example, a convolutional neural network (CNN) technique to the second feature factor.

Thereafter, in step S230, the correlation between the peripheral nerve signal and the EEG signal may be learned by the machine learning model based on the first and second probability models generated as described above.

Steps S100 and S200 described above may be repeatedly performed for a plurality of operations for which operation intention is to be determined according to the present invention.

Thereafter, the user's brain wave signal is acquired in real time (S300), and the user's operation intention can be determined using the brain wave signal acquired in real time by the above-described machine learning model (S400).

6 is a flowchart of a will control method in which an EEG signal and a peripheral nerve signal are interlocked according to another embodiment of the present invention.

Referring to FIG. 6 , since steps S100 to S400 are the same as those described above with reference to FIGS. 1 to 5 , a redundant description thereof will be omitted.

Thereafter, a will control signal may be generated based on the user's operation intention determined through steps S100 to S400 ( S500 ).

The method for determining the operation intention in conjunction with the EEG signal and peripheral nerve signal described above with reference to FIG. 1 and the method for controlling the will by interworking the EEG signal and the peripheral nerve signal described above with reference to FIG. 6 are learning by signal processing and machine learning model and a processing device capable of analysis.

The present invention is not limited by the above embodiments and the accompanying drawings. For those of ordinary skill in the art to which the present invention pertains, it will be apparent that the components according to the present invention can be substituted, modified and changed without departing from the technical spirit of the present invention.

Claims (5)

acquiring a peripheral nerve signal and an EEG signal of the user when the user performs a preset operation;
analyzing the correlation between the peripheral nerve signal and the EEG signal by learning the EEG signal based on the peripheral nerve signal by a machine learning model;
acquiring the EEG signal of the user in real time; and
A method of determining an operation intention in conjunction with an EEG signal and a peripheral nerve signal, comprising the step of determining the user's operation intention by using the EEG signal obtained in real time by the machine learning model.
According to claim 1, wherein the step of learning the EEG signal based on the peripheral nerve signal,
extracting first and second characteristic factors from the peripheral nerve signal and the EEG signal, respectively;
generating first and second probabilistic models for the operation using the first and second characteristic factors, respectively; and
EEG signal and peripheral nerve signal comprising the step of analyzing the correlation between the peripheral nerve signal and the EEG signal when the operation is performed by the machine learning model based on the first and second probabilistic models A method for determining the intention of an operation in conjunction with
The method of claim 2, wherein the extracting of the first and second characteristic factors comprises:
performing pre-processing on the peripheral nerve signal;
extracting entropy for each electrode from the preprocessed peripheral nerve signal; and
An operation intention determination method in conjunction with an EEG signal and a peripheral nerve signal, comprising the step of generating an entropy histogram based on the entropy for each electrode.
The method of claim 2, wherein the extracting of the first and second characteristic factors comprises:
performing pre-processing on the EEG signal; and
Based on the pre-processed EEG signal, the method for determining the operation intention in conjunction with the EEG signal and the peripheral nerve signal, characterized in that it comprises the step of generating a time-frequency two-dimensional image.
acquiring a peripheral nerve signal and an EEG signal of the user when the user performs a preset operation;
analyzing the correlation between the peripheral nerve signal and the EEG signal by learning the EEG signal based on the peripheral nerve signal by a machine learning model;
acquiring the EEG signal of the user in real time;
determining the user's intention to operate by using the EEG signal acquired in real time by the machine learning model; and
A will control method in conjunction with an EEG signal and a peripheral nerve signal, comprising the step of generating a will control signal based on the determined operation intention of the user.

Images