CZ304005B6 - Brain-machine interface with automatic identification of user - Google Patents

Abstract

The present invention relates to a brain-machine interface with automatic identification of user consisting of an EEG apparatus (1) for scanning human brain activity, provided with an electrode array, a first module (2) for parameterization and pre-processing the EEG signal, a first module (3) for recognizing patterns of metal activities intended for controlling device in the EEG signal interconnected with a knowledge base (5) for recognizing metal activity, and an action member (4) for carrying out the corresponding action. A second module (2) for parameterization and pre-processing of the EEG signal is connected to EEG apparatus (1). The second parameterization and pre-processing module (6) consists of a multiplier (11) for modulating the signal by a complex exponential function, which is connected via a decimation unit (12) with a segmentation unit (13), wherein said decimation unit (12) is formed by a low pass filter followed up with a sample omission circuit. The segmentation unit (13) output is connected to the input of a self-correlation analysis unit (14). The second parameterization and EEG signal pre-processing module (6) is connected to the input of a second module (7) for recognizing a user wherein said second module (7) for recognizing a user consists of a plurality of calculation units (16) of Mahalanobis distance wherein comparison inputs of said Mahalanobis distance calculation units (16) are connected to the outputs of a set (9) of knowledge bases for recognizing metal activity of the individual users. The number of the bases in the set is given by the number of the system users and their output are connected to a unit (17) for determining minimum distance wherein said minimum distance determining unit (17) output is connected to module (8) of knowledge base selection. Said knowledge base selection module (8) is formed at its input by a time filtration unit (18) having its output connected via a detection unit (19) of a user change to a knowledge base change-over switch (20) having its inputs connected to outputs of a knowledge base from the set (9) of the knowledge bases for recognizing metal activity of the individual users, while the output thereof is connected to said first pattern recognition module (3).

Description

Brain-machine interface with automatic user identification

Technical field

The present invention relates to a re-creation of a brain-machine interface that allows automatic identification of its current user.

BACKGROUND OF THE INVENTION

At present, it is not possible for the brain-machine interface to recognize perfectly the EEG signal patterns of all system users. This is due to the large variability between the EEGs of different persons. For example, a brain-machine interface will be used to control computer games and be used by multiple users. Then, each time a user changes, for example, a game player, the system must be manually switched to recognizing the characteristic types of brain activity of the new user in order to maximize the recognition success of each brain activity. There is an analogy with the user login to the operating system.

Currently, the brain-machine EEG interface is implemented by a human brain activity sensing device equipped with an electrode field, an EEG signal parameterization and preprocessing module, a mental activity pattern recognition module for controlling devices in an EEG signal coupled to a knowledge base for mental activity recognition and an actuator for the implementation of the action.

SUMMARY OF THE INVENTION

The above drawbacks are partially overcome by the proposed solution using a combination of a biometric system with a brain-machine interface, referred to as BCI, a Brain-Computer Interface.

The brain-machine interface with automatic user identification consists of an EEG device for sensing human brain activity equipped with an electrode array, a first module for parameterization and preprocessing of the EEG signal, a first module for pattern recognition of mental activities designed to operate the device using an EEG signal coupled to a knowledge base for mental activity recognition and an action member for the implementation of the action. The essence of the new solution is that a second module of EEG signal parameterization and preprocessing is connected to the EEG device for sensing human brain activity. This second EEG signal parameterization and preprocessing module consists of a multiplexer for modulating the signal with a complex exponential that is connected through a low pass decimation block and a sample skip circuit with a segmentation block. The segmentation block output is linked to the autocorrelation analysis block input. A second user recognition module is connected to this second EEG signal parameterization and preprocessing module. The second user recognition module consists of a set of blocks of Mahalanobis distance calculation, whose comparative inputs are connected to the outputs of a set of knowledge bases for recognizing the mental activity of individual users and whose outputs are connected to the minimum distance determination block. The number of bases in the set is given by the number of users of the system, each user is assigned one knowledge base. The output of the minimum distance block is linked to the knowledge base selection module. The knowledge base selection module is formed at the input by a time filtering block, the second input of which can be connected to the EEG output of the human brain activity sensing device indicating a user disconnection from the system and whose output is connected via the user change detection block to the knowledge base switch. The knowledge base switch inputs are coupled to the outputs of the knowledge base set to recognize the mental activity of each user. The output of the knowledge base switch is connected to the first pattern recognition module.

- 1 GB 304005 B6

In one possible connection, a surface filter is provided at the input of the second EEG signal preprocessing module, which may advantageously be implemented as a eight-neighbor laplace filter.

In another embodiment, it is possible to output to the output of the second parameterization and preprocessing module of the EEG signal a block position calculation modeling filter.

The low pass of the decimation block may be a Chebyshev filter.

In another possible embodiment, a sliding window block is used as the segmentation block.

The time filtering block may be a majority decoder. The time filter block can be connected to the EG device if it provides an output indicating a user disconnect. The user disconnect information will then be used to refine the detection of the system user change.

The brain-machine interface system thus created is used as a conventional brain-machine interface system enhanced with automatic user identification. The advantage of this solution is that the interface is simplified, the biometric system itself identifies the person working with the device and selects the data base, ie types and parameters of the mental activity of the respective user, based on the characteristic EEG signal parameters identifying the user of the system. Therefore, there is no need to explicitly change interface settings when changing users, such as when exchanging players for a game. Compared to the manual user login to the system, which is the current solution, the proposed approach brings savings of time spent manually adjusting the system and increased user comfort.

Furthermore, the proposed brain-machine interface design provides continuous user recognition.

This ensures that the system parameters are updated immediately when the user is replaced.

To identify a person, the experimental implementation of the biometric system uses the frequency distribution of harmonic components of the rhythms of the EEG signal on the scalp of the experimental person. The choice of these parameters brings the advantage of the independence of the measurement method from the character of the used EEG instrument and its parameters and at the same time it is very resistant to artifacts in the measured EEG.

Clarification of drawings

An exemplary embodiment of the proposed brain-machine interface is documented in the accompanying drawings. Figure 1 shows the state of the art. Giant. 2 shows a block diagram of a new solution. Fig. 3 shows a detailed diagram of the parameterization block and EEG signal preprocessing. Giant. 4 shows a block diagram of a user recognition block creation, and FIG. 5 shows a knowledge base switch block.

DETAILED DESCRIPTION OF THE INVENTION

The current form of brain-machine interface is shown in Figure 1. The interface consists of a standard EEG apparatus 1 for sensing the human brain activity of the system user, shown in the header, through an electrode array. The sensed potentials are then passed to a control system, such as a personal computer, implementing the first EEG signal parameterization and preprocessing module 2, which is connected to the knowledge recognition base 5 via the first pattern recognition module 3 for controlling the devices in the EEG signal. mental activity and, secondly, an actuator 4 for carrying out the respective action, which further provides feedback to system users. In the first module 2 of parameterization and pre-processing of the EEG signal, the scanned EEG signals are processed and necessary adjustments are made before its further use, such as surface filter calculation, parameterization, artifact detection, etc. for recognizing mental activities. First Mental Pattern Recognition Module 3. 9.

The recognition is performed on the basis of information from the knowledge base 5, which includes the types of brain activity of the user of the system used to control the BCI interface and their characteristic parameters. The actuator 4 performs an appropriate action, such as moving the cursor on the computer screen, based on the detected brain activity types of the test person.

Proposed adjustments can then be found in Figure 2. Against the previous figure, a biometric system is added which is connected to the EEG apparatus I for sensing human brain activity. This added system is formed by the second module 6 of EEG signal parameterization and preprocessing. This module processes the sensed EEG signals and performs preprocessing and extraction of the person recognition parameters and may have some common blocks with the first EEG signal parameterization and preprocessing module 2 if the actual brain-machine interface system already uses some of the blocks described herein. A specific embodiment of the second parameterization module will be described below, see Fig. 3. The extracted parameters are further passed to the second user recognition module 2, the detailed implementation of which will be described further below with reference to Fig. 4. The second user recognition module 7 performs user recognition based on EEG parameters signal. Here, based on the calculated EEG signal parameters, the most likely identity of the test person is determined and passed on to the knowledge base selection module 8. This knowledge base selection module 8 based on the identity of the experimenter selects one of the prepared knowledge bases in a set of 9 knowledge bases for recognizing the mental activity of individual users, replacing the original one knowledge base 5 for the recognition algorithm in the first module of pattern recognition of mental activity in the EEG signal brain-machine interface.

The second EEG signal parameterization and preprocessing module 6, FIG. 3, consists of a multiplier 11. for modulating the signal with a complex exponential which is connected through a low pass decimation block 12, for example a Chebyshev filter, followed by a sample skip circuit with a segmentation block 13. The output of the segmentation block 13 is coupled to the input of the autocorrelation analysis block 14. In the present example, a surface filter 10, for example a eight-neighbor Lplace filter, is upstream of the inlet. At the output of the second EEG signal parameterization and preprocessing module 6, in the example, the pole position calculation block 15 of the modeling filter is connected.

The second user recognition module 7, Fig. 4, consists of a set of blocks 16 for calculating the Mahalanobis distance, to whose comparative inputs the knowledge base outputs from the set of 9 knowledge base sets are connected for recognizing the mental activity of individual users. Their outputs are coupled to the minimum distance block 17 whose output is coupled to the knowledge base selection module 8. This knowledge base selection module 8, FIG. 5, is formed at the input by a time filtering block 18, for example formed by a major decoder, the second input of which may be connected to the EEG output of the human brain activity sensing device 1. The output of the time filtering block 18 is coupled via the user change detection block 19 to the knowledge base switch 20 interconnected by its inputs to the outputs of the knowledge base set 9 for recognizing the mental activity of individual users and its output coupled to the first pattern recognition module 3.

EEG signal preprocessing consists of two steps: surface filtration by surface filter JO and parameterization of the EEG signal.

The sensed EEG signal X2J1, n1 at the electrode j, j = 1 ... E, where? Is the number of electrodes, at the time nT & T is the sampling period of the EEG signal, is filtered by a surface filter JO. However, this step is not necessary and can be omitted. The resulting filtered EEG x _[beta ], n] is passed to multiplier 11 to modulate the signal by the complex exponential.

In order to maximize the immunity of the recognition to interference, ie to technical and biological artifacts, and at the same time to maximize the independence of the recognition quality from the actual EEG settings for human brain activity sensing, frequency-magnifying autoregressive modeling is used to parameterize the EEG signal. so-called Frequency-Zooming AR modeling FZAR. FZAR analysis allows to analyze EEG parameters

Only in the selected narrow frequency band. This approach is not commonly used for EEG signal processing, it was originally developed for the analysis of audio signals, therefore the description of the individual steps used in the experimental implementation of the biometric system follows.

Parameterisation of the EEG signal is then performed in the following steps. The first is modulation. The EEG signal is modulated so that its spectrum is centered around the center of the frequency band to be analyzed. Modulation is performed by multiplying the complex exponential by the multiplier 11. The modulated EEG is obtained by the relation x _m [j, n] = e ^iQn x _pf [j, n \, where Q = 2xnxf _with xT (Ω is the angular frequency corresponding to /) is the mean frequency of the analyzed band.

The next step is decimation. The EEG signal x _m [j, n] is decimated with a decimation factor D <B _with -xT, where B 1 is the bandwidth to be analyzed; the result is a decimated signal xffn] with a new sampling period Tj = DxT.

Segmentation follows. The EEG signal is segmented using a sliding window technique of length N _with samples shifted with the TV step. * The segmentation results in EEG segments s [j, n] of the EEG from the electrode j, for a time point nxN _to xT.

Autocorrelation analysis is then applied. Using standard autocorrelation analysis, AR coefficients of modeling filter of decimated signal are determined. In the experimental system the order of the modeling filter was chosen p = 1. The output of this step is the signal parameter vectors f [j, n] calculated for the electrode j, and the signal segment s [j, n] ·, each vector containing p values.

Finally, the position of the poles is determined. Optionally, the pole positions of the modeling filter can be calculated based on the coefficients, i.e., the parameter vectors) f [j, n]. This step can be omitted, but its use increases the success of the classification using a small filter order p.

The second user recognition module 7 analyzes the calculated vectors of the parameters f [j, n] and determines the user's identity of the system. The experimental system uses a classifier based on regularized Mahalanobis distance to identify people; a description of the procedure follows, see also Figure 4.

Let f [o] be the Exp vector of the mean of the parameters f [j, n] of the test person o aS [o] is similarly the estimate of the covariance matrix of these parameters (the matrix is of the dimensions (Exp) x (Exp) of the elements); both variables are stored in the knowledge base of the system pertaining to the person o. The test person is then recognized according to the following procedure.

First, the distance is calculated for each person for whom there is a knowledge base in the system. The distance M [fo] between the calculated vector of parameters f [j, n] and the mean value of the parameters f [o] of the person concerned, the index of the person o = E..O, where O is the total number of users working with the BCI system, will be calculated. The minimum distance is found. This means that between the calculated values of M [fo] for all persons, a minimum M [fo _mm ] is found and the person is identified as a person of _min .

In the Knowledge Base Selection Module 8, these steps then take place. The time filtering block 18 deliberately ignores the change of the system user unless indicated for a sufficient period of time to limit the effect of accidental misidentification of the user identity by the second user recognition module 7. The time filtering block 18 monitors the identified user and, in the event of a change, activates, via the user change detection block 19, switching the input of the mental activity recognition module 3 to the corresponding knowledge base from the knowledge base set 9 by the knowledge base switch 20. The knowledge base switch 20 automatically selects one or more of the knowledge base from a set of 9 knowledge base based on the person &apos; s identity and eventually provides awareness of the BCI system itself when the recognition process or other support processes need to be restarted.

The following is a description of advantageous implementations of individual device blocks.

Surface filtration with surface filter 10 can be implemented using a common reference calculation technique such as a discrete laplace filter, splines a laplace filter, and the like. The experimental system used a laplace filter with eight neighbors.

In modulation by multiplier H, in the experimental system was calculated with parameters μ band - 8 to 13 Hz - hence f _s = 10.5Hz ·, and was defined bandwidth as B _s = 5Hz. After decimation, a Chebyshev filter was used as the low pass filter in the decimator. During the segmentation the experimental system used N _s = 15 sec / T and step Nk = Nf4. When applying autocorrelation analysis, the order of the modeling filter was chosen p = 1 in the experimental system. First, the autocorrelation coefficients of the processed signal were determined = [[s [y, My My, + + dL ^τ = ° ··· Ρ.

and then the AR coefficients f [j, n] were calculated by solving the Yule-Walker equations

* _# [0] * _# [θ] · • T? _# [p-2] '/ ₀ [y * # [2] Z [y. «] = • Λ _# [0]. fp-x

Other solutions are possible, such as Levinson-Durbin recursion and the like.

Time filtering can be implemented, for example, by a majority decoder - it identifies the user as the most frequent person in the last L time points.

If it is convenient to determine the position of the filter poles, the parameter vector f [j, n] is first rewritten to form a polynomial function // (z) = 1 + f ₀ \ j, n] z ~ '+ / [/, n] z “ ² + / ₂ [/,«] ζ ' ³ + [/', n} z ~ ^p and its roots are calculated. The calculated roots will then be used as parameters f [j, n] for further processing.

In calculating the distance, the experimental system works with a regularized Mahalanobis distance, a set of blocks 16 calculating the Mahalanobis distance in Figure 4 = (/ - / [o]) ((l - 4) (S [o] + εΕΥ '+ AE) (/ - f [o \) ^T where λ is a regularization coefficient which controls the ratio between the hyperspheric and hypereliptic components of the distance and ε stabilizes the classification process if the matrix S [o] has a determinant close to zero, the person's index o = l ... O, where O is Total number of users working with BCI system.

-5GB 304005 B6

Industrial applicability

The following applications are currently foreseen. The proposed solution can be used wherever the brain-machine interface is used for eg entertainment (computer games) and others and the device is used by multiple users. In these applications, the proposed modification will increase operator comfort. The proposed system can be further integrated with the operating system to complement it with another level of authentication - the commonly used password authentication can be extended with authentication using characteristic brain activity.

Claims (5)

PATENT CLAIMS A brain-machine interface with an automatic user identification, comprising an EEG device (1) for sensing human brain activity, provided with an electrode array, a first module (2) for EEG signal parameterization and preprocessing, a first module (3) for pattern recognition of mental activities to be controlled a device by means of an EEG signal coupled to a knowledge base (5) for recognizing mental activity and a sake member (4) for performing the respective action, characterized in that a second module (6) is connected to the human brain activity sensing device (1) parameterizing and preprocessing the EEG signal consisting of a multiplexer (11) for modulating the signal with a complex exponential which is coupled through a low pass decimation block (12) followed by a sample skip circuit with a segmentation block (13) whose output is coupled to the block input (14) autocorrelation analysis, to this second module (6) pa Rametrization and pre-processing of the EEG signal is connected by its input to the second user recognition module (7), which consists of a set of blocks (16) of calculation of Mahalanobis distance, on whose comparative inputs the knowledge base outputs from the set (9) wherein the number of bases in the set is given by the number of system users and their outputs are coupled to a minimum distance block (17) whose output is coupled to a knowledge base selection module (8) formed at the input by a time filtering block (18) the output is connected via a user change detection block (19) to a knowledge base switch (20) interconnected by its inputs with the knowledge base outputs from a set of (9) knowledge base for recognizing the mental activity of individual users and its output coupled to the first recognition module (3) of patterns. Brain-machine interface according to claim 1, characterized in that a surface filter (10) is provided at the input of the second EEG preprocessing module (6). Brain-machine interface according to claim 2, characterized in that the surface filter (10) is a eight-neighbor Lplace filter. Brain-machine interface according to one of Claims 1 to 3, characterized in that the output of the second module (6) of the parameterization and preprocessing of the EEG signal is assigned a pole (15) for calculating the positions of the modeling filter. Brain-machine interface according to any one of claims 1 to 4, characterized in that the low pass filter of the decimation block (12) is a Cebyshev filter. Brain-machine interface according to any one of claims 1 to 5, characterized in that the segmentation block (13) is a sliding window technique. Brain-machine interface according to any one of claims 1 to 6, characterized in that the time filtration block (18) is constituted by a major decoder. -fiEN 304005 B6 Brain-machine interface according to any one of claims 1 to 7, characterized in that the time filtering block (18) has an additional input which is connected to the EEG output of the human brain activity sensing device (1) indicating a user disconnection from the system. The brain-machine interface of any one of claims 1 to 8, wherein: The EEG apparatus (1) for sensing human brain activity is provided with an output for indicating a user's disconnection and a time filtering block (18) is connected to this output.