KR100546770B1 - Method for discriminating yes/no decision of human being based on spatio-temporal relationship of brain waves - Google Patents

Abstract

The present invention relates to a method of determining the intention of human positive or negative by analyzing the temporal and spatial correlation of brain waves for a computer interface, the method comprising the steps of (a) measuring the brain waves of the subject from at least two places; (b) removing the honhap mixed in the brain waves; (c) determining whether the subject has made a cognitive affirmative / negative decision; (d) when the subject makes a cognitive affirmative / negative decision, a synchronization rate, a polarity mean, a deflection fluctuation width, and cross-correlation representing the spatiotemporal characteristics of the brain waves. Computing a normalized cross-correlation variable; And (e) inputting the variables calculated in step (d) to an artificial neural network that has already been learned to discriminate the cognitive affirmation / negative intention of the user.

Description

Method for discriminating yes / no decision of human being based on spatio-temporal relationship of brain waves}

1 is a view showing an example of the EEG measured at two points for the discrimination of the cognitive affirmation / denial of the user according to the present invention.

2 is a view showing an example of the EEG measured at three or more points for the discrimination of the cognitive affirmation / denial of the user according to the present invention.

3 is a flowchart illustrating a method of discriminating a cognitive affirmative / negative doctor according to a preferred embodiment of the present invention.

FIG. 4 is a detailed flowchart of the calculation and feature detection step of the spatiotemporal correlation of the brain waves shown in FIG. 3.

FIG. 5 is a diagram illustrating an artificial neural network used in the affirmative / negative pseudo-discrimination step of the user illustrated in FIG. 3.

6 is a flowchart illustrating an example of applying a user's cognitive affirmation / denial pseudo discrimination method in a binary tree format to a computer interface.

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to device manipulation and computer interface (HCI) using biosignals, and more particularly, to a computer interface (BCI) using brain waves.

Device manipulation technology using bio-signals or bio-signal-based computer interface technologies have been actively studied as interface technologies for the handicapped. In addition, such a technology is likely to evolve into a new computer interface technology that follows keyboard, mouse, and voice recognition in the future. In particular, computer interface technology using brain waves is expected to have a great ripple effect in terms of reading human thoughts.

Biosignals being studied for computer interfaces include EEG, EEG, and EMG. The interface technology using biosignals, which has been researched and developed to date, mainly uses a method of intentionally generating a biosignal and delivering a signal according to a predetermined promise. Intentional behavior that is not directly related to a person's wishes, for example, to keep an eye on the icon representing a function to activate a function or to blink or close the eye when the cursor is over the desired icon. By doing so, a method of performing a desired function is used.

However, since the method of operating a biosignal-related device uses a biosignal artificially generated by an appointment that is not directly related to the human intention, the user should always keep in mind the artificial behavior and also There is a problem that requires a certain level of training for use. Therefore, such a method has a problem that is difficult for the general public to use.

Other EEG-related computer interface technologies include a device for detecting and using EEG in a special situation such as sleep and anesthesia, and determining and responding to a user's emotional state as in the method disclosed in Korean Patent Registration No. 10-0291596. EEG potential of the EEG obtained by repeating a specific situation, such as the method described in US Patent No. 5,649,061, entitled "DEVICE AND METHOD FOR ESTIMATING A MENTAL DECISION," issued July 15, 1997, obtained by Smyth. Event Related Potential (ERP).

However, devices for detecting brain waves in special situations, such as sleep and anesthesia, cannot handle various types of commands performed in a situation in which a person is active, which makes it difficult to apply them directly to a computer interface. In addition, since the method disclosed in Korean Patent Registration No. 10-0291596 determines a user's emotional positive / negative state such as like or dislike, it is recognized as yes or wrong. There is a drawback that it is difficult to apply variously to various situations that require judgment of enemy positive / negative status. In addition, the above-mentioned techniques are difficult to process in real time because they are performed through complex tasks. In particular, the method using the induced potential, such as U.S. Patent No. 5,649,061, requires the generation of induced potential through eye tracking and EEG measurement by measuring head movement and angle. There is a problem that cannot be handled.

An object of the present invention is to provide a method for real-time discrimination of human cognitive affirmation / negative intention without generating intentional biosignal.

In order to achieve the above object, the cognitive affirmative / negative doctor's classification method using the spatiotemporal correlation of brain waves according to the present invention comprises the steps of: (a) measuring the brain waves of the subject from at least two places; (b) removing the honhap mixed in the brain waves; (c) determining whether the subject has made a cognitive affirmative / negative decision; (d) when the subject makes a cognitive affirmative / negative decision, a synchronization rate, a polarity mean, a deflection fluctuation width, and cross-correlation representing the spatiotemporal characteristics of the brain waves. Computing a normalized cross-correlation variable; And (e) inputting the variables calculated in step (d) to an artificial neural network that has already been learned to discriminate the cognitive affirmation / negative intention of the user.

In order to achieve the above object, the present invention provides a method for interfacing with a computer, the method comprising: (a) asking a user a question requiring cognitive affirmation / denial; (b) calculating at least two or more variables indicative of the spatiotemporal interrelationships of the brain waves acquired from the user, and inputting the calculated variables into an already learned artificial neural network to discern the cognitive affirmation / negative intention of the user Doing; (c) repeating steps (a) and (b) until a predetermined function is selected based on the classified result; And (d) executing the selected function when the predetermined function is selected as a result of performing the step (b) or (c).

Hereinafter, with reference to the accompanying drawings will be described in detail a preferred embodiment of the present invention.

1 is a view showing an example of EEG measured at two points for the discrimination of the cognitive affirmation / denial of the user according to the present invention. Referring to FIG. 1, the measurement of the EEG according to the present invention is mainly performed at least two places including the left and right front portions of the head 1 (for example, Fp1 and Fp2 or F3 and F4 points in the international electrode placement method). do. EEG measuring sensors 11, 12 and 13 (e.g., cup electrodes) are placed at the measurement positions, and the EEG signals input through the sensors 11, 12 and 13 are predetermined through the amplifiers 18 and 19. Amplified at the ratio of and recorded. The measurement site of the EEG and the number of measurement electrodes can be adjusted according to the use of the apparatus to be applied.

As will be described in detail below, the EEG, electrocardiogram (ECG), electrocardiogram (ECG), and other biological signals unrelated to human intention, and mechanical and electrical noise are mixed with EEG. As it comes in, such miscellaneous waves are removed through a predetermined process. After the removal of such crosstalks, variables such as synchronization rate, polarity mean, deflection fluctuation width, cross-correlation, and the like that may represent the spatiotemporal characteristics of the two EEG signals s1 and s2, Normalized cross-correlation is calculated for each, and the user's cognitive affirmation / negative intention is discriminated based on variables representing the temporal and temporal characteristics of the EEG.

2 is a view showing an example of the EEG (s1, s2, s3, ..., sz) measured more than three points for the discrimination of affirmative / negative doctors of the user according to the present invention. In the positive / negative pseudo-classification method of the user according to the present invention, for example, when three brain wave signals s1, s2, s3 are acquired, three signals s1, s2, s3 can be configured. The synchronization rate, bias average, deflection fluctuation width, cross-correlation, and normalized cross-correlation for three data pairs s1, s2), (s1, s3), and (s2, s3) are calculated, respectively. As a result, the spatiotemporal characteristics of the brain waves of the three signals s1, s2, and s3 are extracted, and the user's cognitive affirmation / negative intention is discriminated.

3 is a flowchart illustrating a method of discriminating a cognitive affirmative / negative doctor according to a preferred embodiment of the present invention. Referring to FIG. 3, in the cognitive affirmation / negative doctor's classification method, first, the human brain wave is measured as shown in FIGS. 1 and 2 (10). When the EEG is measured, the harmonics included in the measured EEG signal are removed (step 20). Here, the noise removal is a process of removing biological signals, mechanical, and electrical noise that are not related to human intention, such as safety (EOG), electromyography (EMG), and electrocardiogram (ECG), which are mixed when measuring EEG. As a result, all unnecessary signals other than the EEG signal are removed from the measured signals, leaving only the pure EEG signal. After the harmonics are removed in this manner, it is determined whether the user has made a decision about cognitive affirmation / denial (step 30). As a result of the determination, if the user has not made a decision about cognitive affirmation / negation, the measurement of the EEG continues. If the user determines a decision regarding cognitive affirmation / negation, the brain-space time-space correlation calculation and feature extraction are performed to discriminate whether the doctor is positive or negative (step 40). As will be described in detail below, the spatiotemporal characteristics of EEG are extracted by calculation of the EEG signal's synchronization rate, deflection mean, deflection fluctuation width, cross-correlation, and normalized cross-correlation, and are calculated according to the number of acquired EEG. The number of variables can vary. When the spatiotemporal feature of the EEG is extracted by the calculation of the variables as described above, it is discriminated based on the extracted feature whether the user has determined the positive intention or the negative intention (step 50). A classification method such as a linear decision function, a neural network, or the like is used as a method of discriminating a user's cognitive affirmation / negative intention using the spatiotemporal characteristics of the extracted brain waves.

FIG. 4 is a detailed flowchart of the calculation and feature detection step (step 40) of the temporal and spatial correlation of the brain waves shown in FIG. 3. Referring to FIG. 4, in step 40, it is first determined how many brain waves are measured (step 41). As a result of the determination, when EEG is measured at two points as shown in FIG. 1, the two EEGs are calculated by calculating synchronization rate, deflection mean, deflection fluctuation width, cross-correlation, and normalized cross-correlation for two EEG signals, respectively. The spatial and temporal correlation of the signal is calculated and the features of the EEG signal are extracted (step 42). If the EEG is measured at three or more points as shown in FIG. 2, the measurement signals are classified into a plurality of non-overlapping data pairs (step 43). For example, when three EEG signals s1, s2, s3 are acquired, these EEG signals s1, s2, s3 are data of (s1, s2), (s1, s3), (s2, s3). Are classified into pairs (step 43). Then, synchronization rate, deflection mean, deflection fluctuation width, cross-correlation, and normalization for the two EEG signals constituting each data pair ((s1, s2), (s1, s3), (s2, s3)). By calculating the cross-correlation for each data pair, the temporal and spatial correlation of the brain waves of each data pair is calculated, and the characteristics of the EEG signal are extracted (step 44).

The calculation of the temporal and spatial correlation of EEG consists of the calculation of variables that quantify the temporal and spatial correlation of EEG. Variables calculated to extract features of EEG in the present invention include a synchronization rate, a polarity average, a deflection fluctuation width, a cross-correlation, and a normalized cross. There are five variables of -correlation.

Synchronization rate is a variable indicating the degree to which the EEG measured at two places (see s1 and s2 in FIG. 1) increases or decreases together, and the bias is a phase space composed of the EEG components (s1 and s2) measured at two places. The direction of rotation in. This is used to indicate which of the two places where the measurement is made is activated first. This motivation and bias are as follows.

[Equation 1]

[Equation 2]

[Equation 1] represents the synchronization rate, and [Equation 2] represents the deflection. Here, s1 and s2 are brain wave signals measured at two positions, and vector s is a two-dimensional vector composed of s1 and s2. H (x) used in the definition of synchronization rate is a step function that has a value of 0 when x is negative or 0, and a value of 1 when positive, and w is the interval size for calculating the synchronization rate. . The unit vector θ used to define the deflection is a unit vector that rotates counterclockwise around the origin in the phase space consisting of s1 and s2. The deflection mean may mean an average value of deflections in a certain section, and the deflection fluctuation width may be expressed as a standard deviation of deflection in a certain section.

Cross-correlation is a variable that indicates the degree to which two EEG signals correlate with relative EEG signals, and follows the general definition used in the field of signal processing. Normalized cross-correlation also follows the common definition used in the field of signal processing, which is defined as the square root of the product of the auto-correlation values of each of the two signals (s1, s2) in the cross-correlation diagram. do.

The five variables (i.e. synchronism rate, bias average, deflection fluctuation width, cross-correlation, and normalized cross-correlation) obtained by this calculation method are used to classify the characteristics of EEG into classification methods such as linear decision function and neural network. It is applied as a variable to indicate the user's cognitive affirmation / negative intention.

Features used to discern the user's cognitive affirmation / negative intentions include the use of one of the variables mentioned above (e.g., motivational rate) alone, or several variables (e.g., motivational rate and bias). Average), and for one type of variable, the parameters used to calculate the variable, for example, calculated values with different section sizes (for example, 0.5 second intervals) Using a time series of features that show the variation over time of each variable (e.g., computed at intervals of 0.1 seconds before 0.5 seconds). You can also use a synchronization time series. In addition, when there are three or more measurement sites as shown in FIG. 2, variables calculated for each data pair may be used. At this time, the features used to discern the cognitive affirmation / denial of the user can be adjusted in consideration of the efficiency of the application device.

FIG. 5 is a diagram illustrating an artificial neural network used in the affirmative / negative pseudo-discrimination step of the user illustrated in FIG. 3. As described above, when variables representing the temporal and temporal characteristics of EEG such as synchronization rate, deflection mean, deflection fluctuation width, cross-correlation, and normalized cross-correlation are calculated (see step 40 of FIG. 3), the user according to the present invention In the positive / negative pseudo classification method of, these variables are applied to classification methods such as linear decision function and neural network, respectively, to discriminate the cognitive positive / negative intention of the user (see step 50 of FIG. 3). Referring to FIG. 5, the artificial neural network includes a multilayer perceptron including an input layer 51, a hidden layer 52, and a node level output layer 53. When the variables (eg, synchronism rate, deflection mean, deflection fluctuation width, cross-correlation degree, normalized cross-correlation degree) representing the spatiotemporal characteristics of EEG are calculated by the method illustrated in FIGS. 3 and 4, the variable They are input to the input layer 51 of the artificial neural network and the learning of the artificial neural network begins. Here, the learning of the artificial neural network is performed through a known error backpropagation algorithm, and the like, in the case of the artificial neural network that has already been learned, in response to variables input through the input layer 51. A cognitive affirmation / denial intention of the user is determined, and the determined data is output through the output layer 53.

6 is a flowchart illustrating an example of applying a user's cognitive affirmation / denial pseudo discrimination method in a binary tree format to a computer interface. Referring to FIG. 6, the computer first asks the user a question 1 that requires a cognitive affirmation / denial (step 100). Subsequently, when the user determines the cognitive affirmation / negative intention in response to the question 1, the user's cognitive affirmation / negative intention to the question 1 is discerned (step 101). As a result of the discrimination, if it is determined that the user has made a positive decision, the computer raises the question 2-1 to the user (step 111). If it is determined that the user has decided to cheat, the computer raises Question 2-2 to the user (step 112).

When question 2-1 is raised in step 111, the user responds to the question to determine a cognitive affirmation / negative decision, and a classification of the user's cognitive affirmation / negative decision is performed (step 121). As a result of the discrimination, if it is determined that the user has made a positive decision, the computer continues to raise the next question to the user until the predetermined function is selected. When following the binary tree thus constructed and reaching a predetermined function (step 181, step 182, ..., or step 189), the function of the computer is activated to perform the function. When the predetermined function is activated in the computer as described above, the selected function is performed (step 191, 192, ..., or 199).

For example, if the computer raises the question "Do you want to edit the document?" As Question 1 (see step 100), then if the user's affirmative / negative decision is affirmative, the computer uses "Document Editor A." Would you like to do it? "As question 2-1 (see step 111). In this case, if it is determined that the user has indicated affirmative intention, the computer activates and executes the text editor A as the final result derived by question 2-1 (see steps 181 and 191). And, if it is determined that the user indicated negative intent to question 2-1, "Would you like to use text editor B?" Raises another question, and executes the text editor B or raises another question according to the user's affirmative / negative decision result.

As described above, when the user's cognitive affirmation / decision making method according to the present invention is applied to a computer interface, unlike a conventional computer interface technology using a biosignal, a human decision can be made without a predetermined intentional action. Can be captured. In addition, compared to the methods disclosed in US Patent No. 5,649,061 and Korean Patent Registration No. 10-0291596, it has an advantage of a faster speed of discrimination of false positives / false negatives. Therefore, by improving the convenience of the biosignal-based computer interface technology, in particular EEG-based computer interface technology, it is possible to implement an EEG-based computer interface that can be easily used by the general public as well as the disabled.

In particular, although the cognitive affirmative / negative doctor has been specifically illustrated for the computer interface using the same in terms of being the most basic human doctor, the interface can be easily applied to other systems that are currently used, and it is also possible to control other devices. The present invention can be applied.

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, and may also be implemented in the form of a carrier wave (for example, transmission over the Internet). Include. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

As described above, according to the positive / negative pseudo-discrimination method using the spatiotemporal correlation of brain waves according to the present invention, the present invention analyzes the characteristics of brain waves that occur naturally when a person thinks, that is, the spatiotemporal correlation of brain waves. Discern cognitive affirmations, one of the most basic human doctors. Since it is not a form of generating an artificial bio-signal according to a predetermined promise for communication, the circuit configuration is simple and the positive / negative pseudo classification can be realized in real time. Therefore, it can be easily applied to a computer interface or the like.

Claims (11)

delete delete delete delete (a) measuring the brain waves of the subject from at least two places; (b) removing the honhap mixed in the brain waves; (c) determining whether the subject has made a cognitive affirmative / negative decision; (d) when the subject makes a cognitive affirmative / negative decision, a synchronization rate, a polarity mean, a deflection fluctuation width, and cross-correlation representing the spatiotemporal characteristics of the brain waves. Computing a normalized cross-correlation variable; And (e) An artificial neural network, which is composed of a multilayer perceptron composed of an input layer, a hidden layer, and a node level output layer and is already trained, in step (d). Enter the calculated variables, The neural network, which has been learned, discriminates the cognitive affirmation / negativeness of the user in response to the variables input through the input layer, and outputs the fractionation result through the output layer. Cognitive Positive / Negative Pseudo Discrimination Using Spatio-temporal Interrelationship. The method of claim 5, In the step (e), the artificial neural network discriminates the cognitive affirmation / negative intention of the subject by using any one of the variables alone. Positive / Negative Pseudo Discrimination. The method of claim 5, In step (e), the artificial neural network discriminates the cognitive affirmation / negative intention of the subject by using at least two or more variables among the variables. Positive / Negative Pseudo Discrimination. The method according to claim 6 or 7, In the step (e), the artificial neural network distinguishes the subject's cognitive affirmation / negative intention by using a numerical value calculated by varying a parameter with respect to the variable. Cognitive Positive / Negative Pseudo Discrimination. The method according to claim 6 or 7, In the step (e), the artificial neural network discriminates the cognitive affirmation / negative intention of the subject by using a numerical value calculated by varying the time series for the variable. Positive / Negative Pseudo Discrimination. In a computer system with a processor: (a) asking a user a question requiring cognitive affirmation / negative will; (b) calculating at least two or more variables indicative of the spatiotemporal interrelationships of the brain waves acquired from the user, and inputting the calculated variables into an already learned artificial neural network to discern the cognitive affirmation / negative intention of the user Doing; (c) repeating steps (a) and (b) until a predetermined function is selected based on the classified result; And and (d) executing the selected function if the predetermined function is selected as a result of performing step (b) or (c). (a) measuring the brain waves of the subject from at least two places; (b) removing the honhap mixed in the brain waves; (c) determining whether the subject has made a cognitive affirmative / negative decision; (d) when the subject makes a cognitive affirmative / negative decision, a synchronization rate, a polarity mean, a deflection fluctuation width, and cross-correlation representing the spatiotemporal characteristics of the brain waves. Computing a normalized cross-correlation variable; And (e) inputting the variables calculated in step (d) to an artificial neural network that has already been learned, and using a computer to record a program for executing a step of discriminating a user's cognitive affirmation / negativeness on a computer. Recording media.

Images