WO2021019776A1 - "Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" - Google Patents
"Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" Download PDFInfo
- Publication number
- WO2021019776A1 WO2021019776A1 PCT/JP2019/030295 JP2019030295W WO2021019776A1 WO 2021019776 A1 WO2021019776 A1 WO 2021019776A1 JP 2019030295 W JP2019030295 W JP 2019030295W WO 2021019776 A1 WO2021019776 A1 WO 2021019776A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- nonlinear
- network
- signals
- nonlinear dynamical
- systems
- Prior art date
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
Definitions
- the features extracted by the aforementioned extractor are entered into a numerical classifier (usually a neural network) previously trained to recognize a brain state comprising a specific "imaginary" action, i.e., an action conceived by the person to whom the system is applied. If the classifier recognizes a brain state comprising the aforementioned imaginary action, it issues a command to execute a "physical" action corresponding to (i.e., "concretizing") said imaginary action. The command is then sent to a suitable actuation device in order to produce, as an effect, the fulfillment of the aforementioned physical action (which may also consist in the communication of information only).
- a numerical classifier usually a neural network
- the expression "imaginary action” does not mean only an action conceived by the person to whom the means of acquisition are applied, but also an action performed by the mind of this person and consisting, for example, in a sudden transition from the waking state to the state of sleep, classifiable as sudden drowsiness.
- the physical action of which the numerical classifier issues an execution command is not a concrete action of the imaginary action, but consists, for example, in issuing an alarm sound signal suitable to awaken the said person (bringing them back to the wakeful state).
- the nonlinear dynamical systems are substantially known. More precisely, the concept of synchronization of nonlinear dynamical systems by an external system is well established (e.g., S. Boccaletti, The synchronization of chaotic systems, Phys. Rep. 366 (2002), 1-101). Similarly, the nonlinear, possibly chaotic nature of physiological signals, such as the electroencephalogram, is well known (e.g., C.J. Stam, Nonlinear dynamical analysis of EEG and MEG: review of an emerging field, Clin Neurophysiol. 116(2005), 2266-301). However, to the best of our knowledge, the notion of synchronizing an artificial nonlinear dynamical system or network to a physiological signal, particularly in real-time, is entirely new and not found in the academic or patent literature.
- a brain-computer interface system based on a numerical classifier in the form of a neural network is known in order to control a robotic arm starting from an analysis in time real of signals recorded from brain activity (L. Minati, A. Nigri, C. Rosazza, M.G. Bruzzone, Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals, Med Eng Phys. 34(2012), 650-8).
- a nonlinear dynamical system can have a so-called chaotic behavior.
- a nonlinear dynamical system has a chaotic behavior when small variations of parameters associated with the differential equations included in said system can significantly alter the output signals of the same without however that said output signals are classifiable as noise.
- a nonlinear dynamical system has a chaotic behavior it is able to exhibit phase transitions: that is, the system has access to a very wide repertoire of behaviors.
- the value of said first coefficient, the values of said second coefficients and, if present, the values of said third coefficients are such that said nonlinear dynamical system, or said at least one of the nonlinear dynamical systems of said network comprising at least three of said differential equations if these linear dynamical systems are more than one, have a chaotic behavior.
- a nonlinear dynamical system includes at least three differential equations, it might be advantageous to choose the parameters of the same in such a way that the nonlinear dynamical system has a chaotic behavior.
- brain activity in addition to being, in certain aspects, similar to the behavior of a nonlinear dynamical system, can have chaotic elements. According to this aspect of the invention, therefore, the synchronization between the nonlinear dynamical system or the network of nonlinear dynamical systems is even better, and the brain activity of the person to whom the acquisition means are applied.
- said brain-computer interface system comprises feedback means that can be operated by a user (which could be the person to whom said acquisition means are applied) for communication, to said supervision unit, of an own evaluation of the correspondence between said imaginary action and said physical action (performed by said machine), said supervision unit being suitable for: ⁇ receiving as input: - said filtered signal, possibly transformed if said nonlinear transform is present, but not processed by said nonlinear dynamical system or, if said linear dynamical systems are more than one, - said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network, nor evaluated by said second evaluation means, if present; ⁇ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the assessment communicated by said user.
- the brain-computer interface system includes: ⁇ further acquisition means applicable to said person for the acquisition of at least one further physiological signal not directly associated with the brain activity of said person (such as for example the heartbeat); ⁇ means for monitoring said machine for detecting an execution of said physical action by said machine, said supervision unit being suitable for: ⁇ evaluating, from an examination of said further physiological signal, a correspondence between said imaginary action and said physical action, and ⁇ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the evaluation carried out by said supervision unit.
- the assessment made by the supervisory unit is indirect and is based on the recognition of emotional reactions by the user (i.e., the person to whom the means of acquisition is applied), in particular, frustration.
- emotional reactions i.e., the person to whom the means of acquisition is applied
- frustration i.e., the person to whom the means of acquisition is applied
- a transient acceleration of the heartbeat and an increase in skin conductance can be expected.
- said supervision unit is suitable for: ⁇ receiving as input also: - said signal processed by said nonlinear dynamical system or - said signals processed by the nonlinear dynamical systems of said network if said linear dynamical systems are more than one; ⁇ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients also as a function of said signal processed by said nonlinear dynamical system or, if said linear dynamical systems are more than one, of said signals processed by the nonlinear dynamical systems of said network.
- Figure 1 shows a brain-computer interface system 1 applied to a person 2 in order to recognize an imaginary action conceived by the latter or performed by the latter's mind and to command, upon recognition of said imaginary action, the execution of a physical action (possibly realizing said imaginary action) to a machine 3.
- the system 1, object of invention includes a device suitable for acquiring a physiological signal associated with the brain activity of the person 2.
- This device can comprise, by way of example, an electrode for electroencephalography or an optode for near-infrared spectroscopy.
- an electrode 4 for electroencephalography shown, in figure 1, applied to the scalp of person 2
- said signal corresponds to an electroencephalographic signal.
- the signal acquired by the electrode 4 is sent to an amplifier 5 which, in addition to amplifying said signal, can convert it to the digital domain.
- the signal thus amplified is transmitted to a filter 6, preferably of the band-pass type, in order to attenuate the components of the signal corresponding to non-relevant activities of the person 2. More precisely, the filter 6 preferably attenuates both the frequencies which are too low (because they are artifacts of the movements of the person 2), and the frequencies which are too high (because they are artifacts of the muscular activity of the person 2).
- the filtered signal is sent to a nonlinear transform 7 (for example, a sigmoidal function) to reshape its distribution conveniently.
- the filtered signal is transformed so as to emphasize certain ranges of the dynamics of said signal, that is to say in order to highlight the signal components most relevant in brain-computer interfacing for the purpose of recognizing states cerebral including imaginary actions (for example, zero-crossing events).
- the signal thus transformed is fed into a nonlinear dynamical system 8 comprising at least two, and preferably at least three, differential equations 9 (shown in figure 1, by way of example, in a number equal to three: "d1", "d2" and "d3").
- Each equation 9 specifies a time course of a variable as a function of said variable and/or of at least another variable also included in another equation 9, so that each equation 9 is interdependent with at least another equation.
- the transformed signal (received as input by the system 8) corresponds to at least one variable of the equations 9, for example identifiable with the letter "s".
- At least one of the equations 9 comprises said variable "s" multiplied by a first coefficient (of coupling between the brain activity of the person 2 and the system 8), for example identifiable with the letter "k".
- Each of the equations 9 further comprises at least one variable that does not correspond to the transformed signal (that is, that is not "s") and multiplied by a second coefficient (of mutual coupling between the variables of the equations 9, i.e., between the variables of the system 8).
- the second coefficients are for example three and are respectively identifiable with the letters "a", "b” and "c".
- the transformed signal "s" can be entered in one or more of the equations 9 of system 8, as well as in all equations 9.
- an equation 9 can be interdependent with one or more of the other equations 9 of system 8, as with all equations 9.
- the system 1 comprises a first memory 10 where at least one value assumed by the coefficient "k” can be stored, and a second memory 11 where, for each of the coefficients "a”, “b” and “c", at least one value assumed by said coefficient "a", "b” or "c" can be stored.
- the signal thus processed (by the system 8) is sent to a feature extractor 13 (previously identified with the expression “extraction means") suitable to extract from the processed signal characteristics of the latter relevant to the classification of one brain state of the person 2.
- the signal that is sent to feature extractor 13 corresponds to one or more of the variables of the equations 9.
- the feature extractor 13 generates a feature vector that is entered in a numerical classifier 14 (preferably, but not necessarily, a neural network) previously trained for the recognition, from said features extracted from the feature extractor 13, of a brain state of the person 2 including an imaginary action.
- the classifier 14 is suitable for examining said characteristics so as to establish whether the brain state of the person 2 (corresponding to said characteristics) comprises said imaginary action, so as to recognize or not the presence of the latter in the brain activity which is associated with the physiological signal from which said characteristics have been extracted.
- the feature extractor 13 in addition to receiving the signal processed by the system 8, also receives the signal transformed by the nonlinear transform 7 but not processed by the system 8.
- the feature extractor 13 is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 not only from the signal transformed by the nonlinear transform 7 after having been processed by the system 8, but also by the signal transformed by the nonlinear transform 7 and not processed by the system 8.
- the supervision unit 16 also receives the signal as processed by the system 8 at the input and is suitable to update the value of the coefficient "k” and the value of each of the coefficients "a” , "b” and “c” also as a function of the aforementioned signal as processed by the system 8.
- Figure 3 shows the frequency spectrum 22 of the signal 20.
- the system 8 has intrinsic dynamics
- the values assigned to the parameters "a", "b” and “c” could be such that the system 8 has no intrinsic dynamics, or, although not having intrinsic dynamics, it is very close to having them.
- the system 8 is in this last condition, it is said that it is close to a so-called critical point, i.e., it is maximally responsive to external stimuli.
- each of the filtered signals is transformed so as to emphasize certain ranges of the dynamics of the said signal, that is, so as to highlight the signal components most relevant for brain-computer interfacing, such as zero-crossing events, in order to recognize brain states including imaginary actions.
- Each system 8 of the network 38 comprises at least one equation 9 specifying a time course of a variable as function not only of said variable and/or of at least another variable also included in another equation 9 of said system 8 (as in system 1), but preferably also of at least another variable also included in another equation 9 of another system 8 of the network 38, so that not only each equation 9 of a system 8 of the network 38 is interdependent with at least another equation 9 of said system 8, but each system 8 of the network 38 is also interdependent with at least another system 8 of the network 38.
- Each of the transformed signals received as input by a system 8 of the network 38 corresponds to at least one variable of the equations 9 of the said system 8.
- the variable of the i-th transformed signal can for example be identified with "s i ".
- the signals processed in this way are sent to a unit 40 (denoted, in Figure 10, by "Syn1" suitable for evaluating how much each of the received input signals (thus processed by the systems 8 of the network 38) is synchronized with each of the other input signals received.
- the signal that is sent to the unit 40 corresponds to one or more of the variables of the equations 9.
- the unit 40 previously identified with the expression "first means of evaluation", generates a synchronization (or, more generally, statistical interdependence) matrix concerning the signals received at the input.
- Said synchronization matrix is sent to the feature extractor 13 which is suitable for extracting from the signals processed by the systems 8 of the network 38 as inserted in said synchronization matrix (i.e., after having been evaluated by the unit 40), characteristics relevant to the classification of a brain state of the person 2.
- the latter does not comprise the unit 40 and/or the unit 41.
- the feature extractor 13 receives as input (and is therefore preferably suitable for extracting features relevant to the classification of a brain state of the person 2) the signals processed by the systems 8 of the network 38 and preferably also the signals transformed by the nonlinear transforms 37 and not processed by the systems 8 of the network 38.
- the supervision unit 16 also receives the signals as processed by the systems 8 of the network 38 as input and is suitable to update, in each of the systems 8 of the network 38, the value of the coefficient "k i ", the value of each of the coefficients "a i ", "b i “ e “c i " and the value of each of the coefficients "g i,j " also as a function of the aforesaid signals as processed by the systems 8 of the network 38.
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Veterinary Medicine (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Theoretical Computer Science (AREA)
- Public Health (AREA)
- Psychiatry (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Fuzzy Systems (AREA)
- Neurosurgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Dermatology (AREA)
- Neurology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Psychology (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
Brain-computer interface systems aim to extract purposeful commands from physiological signals, such as the electroencephalogram, usually for assisting a disabled user in performing useful actions. The present invention pertains to such a system with increased accuracy and flexibility. Rather than being processed directly, the recorded signals are fed to one or more nonlinear dynamical systems driven by them and realized physically or simulated numerically. Commands are then extracted from the temporal activity of these systems rather than directly from the recorded signals. By means of insightful choice of the system type, parameters and coupling scheme, this approach can reveal and enhance the features of physiological activity that are relevant to command decoding, and attenuate all the others, beyond what is possible based on the state of the art.
Description
The present invention finds application in the field of the so-called "brain-computer interface systems" or "brain-machine interface systems", i.e., systems suitable for receiving as input physiological signals associated with a person's brain activity (such as electroencephalographic or near-infrared spectroscopy signals) in order to process said signals and extract from them one or more commands for other actuation devices (for example, a robotic arm).
The systems of this type are typically used to help people with motor and/or communication disabilities interact with the outside world. In particular, the brain-computer interface systems, when combined with appropriate devices, allow these people to overcome, at least in part, their motor and/or communication limits by performing one or more actions that, otherwise, would be precluded to them (for example, due to damage to the brain or spinal cord).
The systems of this type can also be used to determine the level of alertness and reactivity of a person driving a vehicle, in order to detect the onset of sudden drowsiness.
Current brain-computer interface systems include, first of all, one or more devices suitable for acquiring physiological signals associated with a person's brain activity, such as an electrode of an electroencephalograph. The electroencephalographic signals are amplified, appropriately filtered and sent to a suitable extractor to extract from the same characteristics relevant to the classification of the brain state of the person from whom the physiological signals were acquired. Said characteristics may consist, by way of example, of univariate parameters such as the amplitude in a certain frequency band and/or in bivariate parameters such as the synchronization between two signals. The features extracted by the aforementioned extractor are entered into a numerical classifier (usually a neural network) previously trained to recognize a brain state comprising a specific "imaginary" action, i.e., an action conceived by the person to whom the system is applied. If the classifier recognizes a brain state comprising the aforementioned imaginary action, it issues a command to execute a "physical" action corresponding to (i.e., "concretizing") said imaginary action. The command is then sent to a suitable actuation device in order to produce, as an effect, the fulfillment of the aforementioned physical action (which may also consist in the communication of information only).
The known brain-computer interface systems have relatively long response times: that is, a person must think for a long time about the (imaginary) action for which the classifier has been trained so that the latter is able to recognize it (in order to then issue a command to execute a corresponding physical action). In addition to this, the accuracy of the recognition remains too low.
The object of the present invention is to overcome the aforementioned drawbacks by indicating a faster and more accurate brain-computer interface system than the known systems of the same type.
The object of the present invention is a brain-computer interface system comprising:
・ means applicable to a person for the acquisition of at least one physiological signal associated with the brain activity of said person. These means will be indicated later in this description with the expression "means of acquisition" and include, by way of example, at least one electrode for electroencephalography or an optode for near-infrared spectroscopy (better known as "NIRS");
・ amplifying means of said physiological signal (acquired from said acquisition means);
・ filtering means of said amplified signal (from said amplification means). Said filtering means preferably comprise a band-pass filter;
・ extraction means, from said filtered signal (from said filtering means), of characteristics relevant to the classification of a brain state of said person (from whom the physiological signal was acquired);
・ at least one numerical classifier (preferably a neural network) of said features extracted from said extraction means,
said numerical classifier being trained to recognize, from said features extracted from said extraction means, a brain state of said person comprising an imaginary action (i.e., being suitable for examining said characteristics so as to establish whether the brain state of said person corresponding to said features includes said imaginary action, so as to recognize or not the presence of the latter in the brain activity to which said physiological signal is associated from which said characteristics have been extracted),
said numerical classifier being suitable, upon the recognition of a brain state comprising said imaginary action, to emit at least one execution command of a physical action, for example corresponding to (i.e., "concretizing") said imaginary action.
・ means applicable to a person for the acquisition of at least one physiological signal associated with the brain activity of said person. These means will be indicated later in this description with the expression "means of acquisition" and include, by way of example, at least one electrode for electroencephalography or an optode for near-infrared spectroscopy (better known as "NIRS");
・ amplifying means of said physiological signal (acquired from said acquisition means);
・ filtering means of said amplified signal (from said amplification means). Said filtering means preferably comprise a band-pass filter;
・ extraction means, from said filtered signal (from said filtering means), of characteristics relevant to the classification of a brain state of said person (from whom the physiological signal was acquired);
・ at least one numerical classifier (preferably a neural network) of said features extracted from said extraction means,
said numerical classifier being trained to recognize, from said features extracted from said extraction means, a brain state of said person comprising an imaginary action (i.e., being suitable for examining said characteristics so as to establish whether the brain state of said person corresponding to said features includes said imaginary action, so as to recognize or not the presence of the latter in the brain activity to which said physiological signal is associated from which said characteristics have been extracted),
said numerical classifier being suitable, upon the recognition of a brain state comprising said imaginary action, to emit at least one execution command of a physical action, for example corresponding to (i.e., "concretizing") said imaginary action.
For the sake of convenience, here and in the following of the present invention, the expression "imaginary action" does not mean only an action conceived by the person to whom the means of acquisition are applied, but also an action performed by the mind of this person and consisting, for example, in a sudden transition from the waking state to the state of sleep, classifiable as sudden drowsiness. In this case, the physical action of which the numerical classifier issues an execution command is not a concrete action of the imaginary action, but consists, for example, in issuing an alarm sound signal suitable to awaken the said person (bringing them back to the wakeful state).
According to the invention, said brain-computer interface system also includes:
・ at least one nonlinear dynamical system for processing said filtered signal and sending the processed signal to said extracting means,
said extraction means being means for extracting said features from said filtered signal after having been processed by said nonlinear dynamical system,
said nonlinear dynamical system comprising at least two differential equations,
each of said differential equations specifying a time course of a variable as a function of said variable and/or of at least another variable also included in another of said differential equations, so that each of said differential equations is interdependent with at least one of said differential equations,
said filtered signal (received as input by said nonlinear dynamical system) corresponding to at least one of said variables of said differential equations,
at least one of said differential equations comprising said variable corresponding to said filtered signal, multiplied by a first coefficient (of coupling between the brain activity and the nonlinear dynamical system),
each of said differential equations comprising at least one of said variables not corresponding to said filtered signal, multiplied by a second coefficient (of coupling between the variables of said nonlinear dynamical system),
said signal processed and sent to said extraction means corresponding to one or more of said variables of said equations,
・ a first memory for storing at least one value assumed by said first coefficient;
・ a second memory for storing, for each of said second coefficients, at least one value assumed by said second coefficient,
the value of said first coefficient and the value of each of said second coefficients being such that said nonlinear dynamical system:
・ preferably has intrinsic dynamics (i.e., generates a spontaneous activity)
and
・ processes said filtered signal in such a way that, in said signal sent to said extraction means, one or more of said characteristics relevant to the classification of a brain state of said person are reveled and/or enhanced.
・ at least one nonlinear dynamical system for processing said filtered signal and sending the processed signal to said extracting means,
said extraction means being means for extracting said features from said filtered signal after having been processed by said nonlinear dynamical system,
said nonlinear dynamical system comprising at least two differential equations,
each of said differential equations specifying a time course of a variable as a function of said variable and/or of at least another variable also included in another of said differential equations, so that each of said differential equations is interdependent with at least one of said differential equations,
said filtered signal (received as input by said nonlinear dynamical system) corresponding to at least one of said variables of said differential equations,
at least one of said differential equations comprising said variable corresponding to said filtered signal, multiplied by a first coefficient (of coupling between the brain activity and the nonlinear dynamical system),
each of said differential equations comprising at least one of said variables not corresponding to said filtered signal, multiplied by a second coefficient (of coupling between the variables of said nonlinear dynamical system),
said signal processed and sent to said extraction means corresponding to one or more of said variables of said equations,
・ a first memory for storing at least one value assumed by said first coefficient;
・ a second memory for storing, for each of said second coefficients, at least one value assumed by said second coefficient,
the value of said first coefficient and the value of each of said second coefficients being such that said nonlinear dynamical system:
・ preferably has intrinsic dynamics (i.e., generates a spontaneous activity)
and
・ processes said filtered signal in such a way that, in said signal sent to said extraction means, one or more of said characteristics relevant to the classification of a brain state of said person are reveled and/or enhanced.
Nonlinear dynamical systems, if they communicate with each other, tend to synchronize with each other. If two nonlinear dynamical systems are synchronized with each other, they generate mutually related output signals. Since brain activity is, in some respects, similar to the behavior of a nonlinear dynamical system, the nonlinear dynamical system of the brain-computer interface system object of invention tends to synchronize with the brain activity of the person to whose brain the acquisition means are applied: in this way, the signals outgoing from the nonlinear dynamical system of the brain-computer interface system of the invention are correlated with the physiological signals acquired by the acquisition means.
The suitably filtered and amplified physiological signal is injected into the nonlinear dynamical system in a sufficiently strong manner as to influence the behavior of said system without, however, overcoming its intrinsic dynamics. The value of said first coefficient and the value of each of said second coefficients are chosen in such a way that the output signal from the nonlinear dynamical system entering the extraction means "helps" the latter to distinguish certain characteristics of the input signal, so as to facilitate the task of the numerical classifier.
Incidentally, the nonlinear dynamical system must include at least two differential equations since if it included only one, it would not be able to oscillate, i.e., it would not have intrinsic dynamics. In other words, this system would not have a spontaneous activity.
Incidentally, a nonlinear dynamical system is realizable numerically or electronically. In this second case the variables of the differential equations of the system correspond to circuit quantities (i.e., voltages and currents).
By way of example, it is known that it is possible to construct a nonlinear, and potentially chaotic, dynamical system in the form of an integrated circuit by combining multiple inverter rings. Advantageously, for the present application, this allows an easy control of the dynamical properties of the oscillations through the variation of the currents in the same, obtaining the generation of signals qualitatively very similar to the electroencephalographic traces (L. Minati, M. Frasca, N. Yoshimura, L. Ricci, P. O?wiecimka, Y. Koike, K. Masu, H. Ito, Current-Starved Cross-Coupled CMOS Inverter Rings as Versatile Generators of Chaotic and Neural-Like Dynamics Over Multiple Frequency Decades, IEEE Access 7(2019), 54638-54657).
The nonlinear dynamical systems are substantially known. More precisely, the concept of synchronization of nonlinear dynamical systems by an external system is well established (e.g., S. Boccaletti, The synchronization of chaotic systems, Phys. Rep. 366 (2002), 1-101). Similarly, the nonlinear, possibly chaotic nature of physiological signals, such as the electroencephalogram, is well known (e.g., C.J. Stam, Nonlinear dynamical analysis of EEG and MEG: review of an emerging field, Clin Neurophysiol. 116(2005), 2266-301). However, to the best of our knowledge, the notion of synchronizing an artificial nonlinear dynamical system or network to a physiological signal, particularly in real-time, is entirely new and not found in the academic or patent literature.
For example, US 2007 0213786 A1 describes a closed-loop system for preventing epileptic seizures based on real-time analysis of the electroencephalogram. Even though notions of nonlinear dynamics and synchronization are included, the fundamental notion of synchronizing the measured signal to an artificial nonlinear dynamical system or network is entirely missing. On the contrary, and in contrast to the central notion in the present invention, linear and nonlinear features are explicitly extracted from the measured signals.
Analogous arguments apply to US 2010 0198098 A1, which describes a similar system. In this case, too, nonlinear dynamics are only mentioned with regards to ways of measuring features of the measured signals. There is not any reference to the idea of synchronizing the measured signal to an artificial nonlinear dynamical system or network.
In WO 2011 123059 A1 a brain-computer interface based on the electroencephalogram and imaginary movement is described. In this case, too, nonlinearity is only mentioned in the context of nonlinear regression. There is not any reference to the idea of synchronizing the measured signal to an artificial nonlinear dynamical system or network.
Finally, in US 2010 0292752 A1, a system and a method for analyzing and generating neural signals is presented. The invention is based on the notion of separating the phase and amplitude of a signal and performing a decomposition into so-called dynamical modes. Again, there is not any reference to the idea of synchronizing the measured signal to an artificial nonlinear dynamical system or network.
Other innovative features of the present invention are illustrated in the following description and referred to in the dependent claims.
According to one aspect of the invention, the brain-computer interface system includes:
・ a plurality of said acquisition means, respectively for acquiring a plurality of physiological signals associated with the brain activity of said person;
・ a plurality of said amplification means, respectively for the amplification of said plurality of physiological signals (respectively acquired by said plurality of acquisition means);
・ a plurality of said filtering means, respectively for filtering said amplified signals (respectively from said plurality of amplification means);
・ a nonlinear dynamical network comprising a plurality of said nonlinear dynamical systems, respectively for processing said filtered signals and sending the processed signals to said extracting means (i.e., each nonlinear dynamical system of said network is suitable to receive as input a respective filtered signal, to process it and to send the processed signal to the extracting means. In other words, each signal filtered by one of the filtering means of said plurality is sent as input to a respective nonlinear dynamical system of said network),
said extraction means being means for extracting said features from said filtered signals after having been processed by the nonlinear dynamical systems of said network,
each nonlinear dynamical system of said network comprising at least one differential equation specifying a time course of a variable as a function of:
- said variable
and/or of
- at least one other variable also included in another of said differential equations of said system
and/or of
- at least one other variable also included in one of said differential equations of another nonlinear dynamical system of said network
in such a way that each linear dynamical system of said network is interdependent with at least one other nonlinear dynamical system of said network,
each of said filtered signals (received as input by said plurality of nonlinear dynamical systems) corresponding to at least one of said variables of said differential equations of a nonlinear dynamical system of said network,
in each nonlinear dynamical system of said network, at least one of said differential equations comprising said variable corresponding to one of said filtered signals, multiplied by said first coefficient,
in each nonlinear dynamical system of said network, each of said differential equations comprising at least one of said variables not corresponding to one of said filtered signals and not included in one of said differential equations of another nonlinear dynamical system of said network, multiplied by said second coefficient,
in each nonlinear dynamical system of said network, at least one of said differential equations comprising at least one of said variables not corresponding to one of said filtered signals and also comprised in one of said differential equations of another nonlinear dynamical system of said network, multiplied by a third coefficient (of mutual coupling between the nonlinear dynamical systems of said network),
said signals sent to said extraction means corresponding to said variables of said linear equations,
for each nonlinear dynamical system of said network:
- said first memory being suitable for storing at least one value assumed by said first coefficient,
- said second memory being suitable for storing, for each of said second coefficients, at least one value assumed by said second coefficient,
・ a third memory for storing, for each of said third coefficients, at least one value assumed by said third coefficient,
for each nonlinear dynamical system of said network, the value of said first coefficient, the value of each of said second coefficients and the value of each of said third coefficients being such that said nonlinear dynamical system:
・ preferably has intrinsic dynamics
and
・ processes said filtered signal in such a way that, in said signal sent to said extraction means, one or more of said characteristics relevant to the classification of a brain state of said person are revealed or enhanced.
・ a plurality of said acquisition means, respectively for acquiring a plurality of physiological signals associated with the brain activity of said person;
・ a plurality of said amplification means, respectively for the amplification of said plurality of physiological signals (respectively acquired by said plurality of acquisition means);
・ a plurality of said filtering means, respectively for filtering said amplified signals (respectively from said plurality of amplification means);
・ a nonlinear dynamical network comprising a plurality of said nonlinear dynamical systems, respectively for processing said filtered signals and sending the processed signals to said extracting means (i.e., each nonlinear dynamical system of said network is suitable to receive as input a respective filtered signal, to process it and to send the processed signal to the extracting means. In other words, each signal filtered by one of the filtering means of said plurality is sent as input to a respective nonlinear dynamical system of said network),
said extraction means being means for extracting said features from said filtered signals after having been processed by the nonlinear dynamical systems of said network,
each nonlinear dynamical system of said network comprising at least one differential equation specifying a time course of a variable as a function of:
- said variable
and/or of
- at least one other variable also included in another of said differential equations of said system
and/or of
- at least one other variable also included in one of said differential equations of another nonlinear dynamical system of said network
in such a way that each linear dynamical system of said network is interdependent with at least one other nonlinear dynamical system of said network,
each of said filtered signals (received as input by said plurality of nonlinear dynamical systems) corresponding to at least one of said variables of said differential equations of a nonlinear dynamical system of said network,
in each nonlinear dynamical system of said network, at least one of said differential equations comprising said variable corresponding to one of said filtered signals, multiplied by said first coefficient,
in each nonlinear dynamical system of said network, each of said differential equations comprising at least one of said variables not corresponding to one of said filtered signals and not included in one of said differential equations of another nonlinear dynamical system of said network, multiplied by said second coefficient,
in each nonlinear dynamical system of said network, at least one of said differential equations comprising at least one of said variables not corresponding to one of said filtered signals and also comprised in one of said differential equations of another nonlinear dynamical system of said network, multiplied by a third coefficient (of mutual coupling between the nonlinear dynamical systems of said network),
said signals sent to said extraction means corresponding to said variables of said linear equations,
for each nonlinear dynamical system of said network:
- said first memory being suitable for storing at least one value assumed by said first coefficient,
- said second memory being suitable for storing, for each of said second coefficients, at least one value assumed by said second coefficient,
・ a third memory for storing, for each of said third coefficients, at least one value assumed by said third coefficient,
for each nonlinear dynamical system of said network, the value of said first coefficient, the value of each of said second coefficients and the value of each of said third coefficients being such that said nonlinear dynamical system:
・ preferably has intrinsic dynamics
and
・ processes said filtered signal in such a way that, in said signal sent to said extraction means, one or more of said characteristics relevant to the classification of a brain state of said person are revealed or enhanced.
According to this aspect of the invention, the brain-computer interface system comprises a plurality of nonlinear dynamical systems, each of which is advantageously coupled with at least another of said nonlinear dynamical systems. In other words, nonlinear dynamical systems interact with each other, that is, they have a level of mutual synchronization. As a result of this, the signals coming out of the nonlinear dynamical systems and entering the extraction means help the latter even more to distinguish certain characteristics of the input signals, so as to facilitate the task of the numerical classifier.
Incidentally, as the intensity of the coupling increases, the aforementioned synchronization is initially manifested in the form of coherence of phase fluctuations, and finally also in the form of overlapping amplitude fluctuations. As an example, the onset of this phenomenon can be quantified in a particularly advantageous way on biological signals and coming from nonlinear systems through a measure known as "warped phase synchronization" (L. Minati, N. Yoshimura, M. Frasca, S. Dro?d?, Y. Koike, Warped phase coherence: An empirical synchronization measure combining phase and amplitude information, Chaos 29(2019), 021102).
As an example, the application of a brain-computer interface system based on a numerical classifier in the form of a neural network (multi-layer perceptron) is known in order to control a robotic arm starting from an analysis in time real of signals recorded from brain activity (L. Minati, A. Nigri, C. Rosazza, M.G. Bruzzone, Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals, Med Eng Phys. 34(2012), 650-8).
Likewise, a similar application of a brain-computer interface system based on a classifier consisting not of a neural network but of a hybrid controller containing a series of relations and operating rules based on the extraction of specific features from brain activity is known (L. Minati, N. Yoshimura, Y. Koike, Hybrid Control of a Vision-Guided Robot Arm by EOG, EMG, EEG Biosignals and Head Movement Acquired via a Consumer-Grade Wearable Device, IEEE Access 4(2016), 9528-9541).
According to an aspect of the invention, said extraction means are means for extracting said features
not only:
・ from said filtered signal after having been processed by said nonlinear dynamical system
or
・ from said filtered signals after having been processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one,
but also:
・ from said signal filtered but not processed by said nonlinear dynamical system
or
・ from said signals filtered but not processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one.
not only:
・ from said filtered signal after having been processed by said nonlinear dynamical system
or
・ from said filtered signals after having been processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one,
but also:
・ from said signal filtered but not processed by said nonlinear dynamical system
or
・ from said signals filtered but not processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one.
According to another aspect of the invention, the brain-computer interface system includes:
・ a nonlinear transform (for example, a sigmoidal function) for:
- the transformation of said filtered signal so as to reveal and/or enhance certain intervals of the dynamics of said signal (so as to highlight the signal components most relevant for the purpose of brain-computer interfacing, for example, zero-crossing events), and
- sending the transformed signal to said nonlinear dynamical system,
said nonlinear dynamical system being suitable for processing said filtered signal after having been transformed by said nonlinear transform,
or, if said linear dynamical systems are more than one:
・ a plurality of nonlinear transforms (for example, sigmoid functions), respectively for:
- the transformation of said filtered signals so as to reveal and/or enhance certain intervals of the dynamics of said signals (so as to highlight the signal components most relevant for the purpose of brain-computer interfacing, for example, zero-crossing events), and
- sending the transformed signals to the nonlinear dynamical systems of said network,
the nonlinear dynamical systems of said network being respectively suitable for processing said filtered signals after having been transformed by said nonlinear transforms.
・ a nonlinear transform (for example, a sigmoidal function) for:
- the transformation of said filtered signal so as to reveal and/or enhance certain intervals of the dynamics of said signal (so as to highlight the signal components most relevant for the purpose of brain-computer interfacing, for example, zero-crossing events), and
- sending the transformed signal to said nonlinear dynamical system,
said nonlinear dynamical system being suitable for processing said filtered signal after having been transformed by said nonlinear transform,
or, if said linear dynamical systems are more than one:
・ a plurality of nonlinear transforms (for example, sigmoid functions), respectively for:
- the transformation of said filtered signals so as to reveal and/or enhance certain intervals of the dynamics of said signals (so as to highlight the signal components most relevant for the purpose of brain-computer interfacing, for example, zero-crossing events), and
- sending the transformed signals to the nonlinear dynamical systems of said network,
the nonlinear dynamical systems of said network being respectively suitable for processing said filtered signals after having been transformed by said nonlinear transforms.
According to another aspect of the invention, said extraction means are means of extracting said features not only:
・ from said transformed signal after having been processed by said nonlinear dynamical system
or
・ from said transformed signals after having been processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one,
but also:
・ from said signal transformed but not processed by said nonlinear dynamical system
or
・ from said signals transformed but not processed from the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one.
・ from said transformed signal after having been processed by said nonlinear dynamical system
or
・ from said transformed signals after having been processed by the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one,
but also:
・ from said signal transformed but not processed by said nonlinear dynamical system
or
・ from said signals transformed but not processed from the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one.
According to another aspect of the invention, said brain-computer interface system includes first evaluation means for:
・ the incoming reception of said signals processed by the nonlinear dynamical systems of said network,
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a first synchronization matrix (or, more generally, statistical interdependence) concerning said signals received at the input,
said extraction means being means for extracting said features from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix (i.e., after having been evaluated by said first evaluation means).
・ the incoming reception of said signals processed by the nonlinear dynamical systems of said network,
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a first synchronization matrix (or, more generally, statistical interdependence) concerning said signals received at the input,
said extraction means being means for extracting said features from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix (i.e., after having been evaluated by said first evaluation means).
According to another aspect of the invention, said brain-computer interface system includes second means of evaluation for:
・ the reception as input of said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network,
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a second synchronization matrix concerning said received input signals,
said extraction means being means for extracting said features not only from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix, but also by said signals filtered and possibly transformed if said nonlinear transforms are present, but not processed by nonlinear dynamical systems of said network as inserted in said second synchronization matrix.
・ the reception as input of said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network,
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a second synchronization matrix concerning said received input signals,
said extraction means being means for extracting said features not only from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix, but also by said signals filtered and possibly transformed if said nonlinear transforms are present, but not processed by nonlinear dynamical systems of said network as inserted in said second synchronization matrix.
According to another aspect of the invention, said extraction means are not only means of extracting said features:
・ from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix,
and
・ from said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network as inserted in said second synchronization matrix,
but also:
・ from said signals processed by the nonlinear dynamical systems of said network but not evaluated by said first evaluation means,
and/or
・ from said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network, nor evaluated by said second evaluation means.
・ from said signals processed by the nonlinear dynamical systems of said network as inserted in said first synchronization matrix,
and
・ from said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network as inserted in said second synchronization matrix,
but also:
・ from said signals processed by the nonlinear dynamical systems of said network but not evaluated by said first evaluation means,
and/or
・ from said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network, nor evaluated by said second evaluation means.
According to another aspect of the invention, in the case in which said linear dynamical systems are more than one, said plurality of acquisition means comprises a plurality of electrodes for electroencephalography,
said brain-computer interface system comprising further filtering means including at least one spatial filter suitable for reconstructing, from said filtered signals, a plurality of physiological signals as would be acquired if said electrodes were placed on the cerebral cortex of said person,
the nonlinear dynamical systems of said network being respectively suitable for processing said filtered signals after having been processed by said spatial filter, and possibly transformed if said nonlinear transforms are present.
said brain-computer interface system comprising further filtering means including at least one spatial filter suitable for reconstructing, from said filtered signals, a plurality of physiological signals as would be acquired if said electrodes were placed on the cerebral cortex of said person,
the nonlinear dynamical systems of said network being respectively suitable for processing said filtered signals after having been processed by said spatial filter, and possibly transformed if said nonlinear transforms are present.
According to another aspect of the invention, said nonlinear dynamical system, or at least one of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, comprises at least three of said differential equations.
Incidentally, if a nonlinear dynamical system includes at least three differential equations, it can have a so-called chaotic behavior. A nonlinear dynamical system has a chaotic behavior when small variations of parameters associated with the differential equations included in said system can significantly alter the output signals of the same without however that said output signals are classifiable as noise. When a nonlinear dynamical system has a chaotic behavior it is able to exhibit phase transitions: that is, the system has access to a very wide repertoire of behaviors.
The fact that a nonlinear dynamical system can have a chaotic behavior is known. We therefore do not dwell on providing further details.
According to another aspect of the invention, in said nonlinear dynamical system or in at least one of the nonlinear dynamical systems of said network comprising at least three of said differential equations if said linear dynamical systems are more than one, the value of said first coefficient, the values of said second coefficients and, if present, the values of said third coefficients, are such that said nonlinear dynamical system, or said at least one of the nonlinear dynamical systems of said network comprising at least three of said differential equations if these linear dynamical systems are more than one, have a chaotic behavior.
If a nonlinear dynamical system includes at least three differential equations, it might be advantageous to choose the parameters of the same in such a way that the nonlinear dynamical system has a chaotic behavior. As is known, brain activity, in addition to being, in certain aspects, similar to the behavior of a nonlinear dynamical system, can have chaotic elements. According to this aspect of the invention, therefore, the synchronization between the nonlinear dynamical system or the network of nonlinear dynamical systems is even better, and the brain activity of the person to whom the acquisition means are applied.
According to another aspect of the invention, in said nonlinear dynamical system or in at least one of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the values of said second coefficients and, if present, the values of said third coefficients are such that in said nonlinear dynamical system, or in said at least one of the nonlinear dynamical systems of said network, if said linear dynamical systems are more than one, emergent phenomena manifest themselves.
According to another aspect of the invention, said brain-computer interface system includes:
・ a machine for receiving said command from said digital classifier and the execution of said physical action (comprised in a brain state of said person upon recognition of which said numerical classifier is trained) for example corresponding to said imaginary action;
・ a supervision unit for updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, of the value of said first coefficient, of the value of each of said second coefficients and, if present, the value of each of said third coefficients.
・ a machine for receiving said command from said digital classifier and the execution of said physical action (comprised in a brain state of said person upon recognition of which said numerical classifier is trained) for example corresponding to said imaginary action;
・ a supervision unit for updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, of the value of said first coefficient, of the value of each of said second coefficients and, if present, the value of each of said third coefficients.
According to another aspect of the invention, said brain-computer interface system comprises feedback means that can be operated by a user (which could be the person to whom said acquisition means are applied) for communication, to said supervision unit, of an own evaluation of the correspondence between said imaginary action and said physical action (performed by said machine),
said supervision unit being suitable for:
・ receiving as input:
- said filtered signal, possibly transformed if said nonlinear transform is present, but not processed by said nonlinear dynamical system
or, if said linear dynamical systems are more than one,
- said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network, nor evaluated by said second evaluation means, if present;
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the assessment communicated by said user.
said supervision unit being suitable for:
・ receiving as input:
- said filtered signal, possibly transformed if said nonlinear transform is present, but not processed by said nonlinear dynamical system
or, if said linear dynamical systems are more than one,
- said filtered and possibly transformed signals if said nonlinear transforms are present, but not processed by the nonlinear dynamical systems of said network, nor evaluated by said second evaluation means, if present;
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the assessment communicated by said user.
According to another aspect of the invention, the brain-computer interface system includes:
・ further acquisition means applicable to said person for the acquisition of at least one further physiological signal not directly associated with the brain activity of said person (such as for example the heartbeat);
・ means for monitoring said machine for detecting an execution of said physical action by said machine,
said supervision unit being suitable for:
・ evaluating, from an examination of said further physiological signal, a correspondence between said imaginary action and said physical action, and
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the evaluation carried out by said supervision unit.
・ further acquisition means applicable to said person for the acquisition of at least one further physiological signal not directly associated with the brain activity of said person (such as for example the heartbeat);
・ means for monitoring said machine for detecting an execution of said physical action by said machine,
said supervision unit being suitable for:
・ evaluating, from an examination of said further physiological signal, a correspondence between said imaginary action and said physical action, and
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients according to the evaluation carried out by said supervision unit.
The assessment made by the supervisory unit is indirect and is based on the recognition of emotional reactions by the user (i.e., the person to whom the means of acquisition is applied), in particular, frustration. As an example, in the event of an incorrect operation, a transient acceleration of the heartbeat and an increase in skin conductance can be expected.
According to another aspect of the invention, said supervision unit is suitable for:
・ receiving as input also:
- said signal processed by said nonlinear dynamical system
or
- said signals processed by the nonlinear dynamical systems of said network if said linear dynamical systems are more than one;
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients also as a function of said signal processed by said nonlinear dynamical system or, if said linear dynamical systems are more than one, of said signals processed by the nonlinear dynamical systems of said network.
・ receiving as input also:
- said signal processed by said nonlinear dynamical system
or
- said signals processed by the nonlinear dynamical systems of said network if said linear dynamical systems are more than one;
・ updating, in said nonlinear dynamical system or in each of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, the value of said first coefficient, the value of each of said second coefficients and, if present, the value of each of said third coefficients also as a function of said signal processed by said nonlinear dynamical system or, if said linear dynamical systems are more than one, of said signals processed by the nonlinear dynamical systems of said network.
Further objects and advantages of the present invention will become apparent from the detailed description provided below of example embodiments thereof and from the accompanying drawings merely given by way of a non-limiting example, in which:
- figure 1 shows a schematization of a brain-computer interface system according to the present invention;
- figure 2 shows an electroencephalographic signal acquired by a person to whom the system of figure 1 is applied;
- figure 3 shows the frequency spectrum of the signal in figure 2;
- figure 4 shows the signal in figure 2 as processed by a nonlinear dynamical system forming part of the system in figure 1, in the case in which there is substantially no coupling between the signal in figure 2 and said nonlinear dynamical system;
- figure 5 shows the frequency spectrum of the signal in figure 4;
- figure 6 shows the XY plot between the signal in figure 2 and the signal in figure 4;
- figure 7 shows the signal in figure 2 as processed by the nonlinear dynamical system forming part of the system in figure 1, in the case in which there is a coupling between the signal in figure 2 and said nonlinear dynamical system which is not zero but at the same time not so strong as to overwhelm the spontaneous activity of the said nonlinear dynamical system;
- figure 8 shows the frequency spectrum of the signal in figure 7;
- figure 9 shows the XY plot between the signal in figure 2 and the signal in figure 7;
- figure 10 shows a schematization of a variant of the system in figure 1;
- figure 11 shows a two-dimensional matrix whose elements express an average synchronization value between twenty-one electroencephalographic signals acquired by a person to whom the system in figure 10 is applied;
- figure 12 shows a two-dimensional matrix whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value;
- figure 13 shows a two-dimensional matrix whose elements express a mean synchronization value between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value, as well as respectively processed by twenty-one nonlinear dynamical systems forming parts of a dynamical nonlinear network included in the system in figure 10, in the case in which there is substantially no coupling between said nonlinear dynamical systems of said network;
- figure 14 shows a two-dimensional array whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 13 expresses an average synchronization value;
- figure 15 shows a two-dimensional matrix whose elements express an average synchronization value between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value, as well as respectively processed by the twenty-one nonlinear dynamical systems forming parts of the dynamical network linear included in the system in figure 10, in the case in which there is a non-zero coupling between said nonlinear dynamical systems of said network;
- figure 16 shows a two-dimensional array whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 15 expresses an average synchronization value.
- figure 1 shows a schematization of a brain-computer interface system according to the present invention;
- figure 2 shows an electroencephalographic signal acquired by a person to whom the system of figure 1 is applied;
- figure 3 shows the frequency spectrum of the signal in figure 2;
- figure 4 shows the signal in figure 2 as processed by a nonlinear dynamical system forming part of the system in figure 1, in the case in which there is substantially no coupling between the signal in figure 2 and said nonlinear dynamical system;
- figure 5 shows the frequency spectrum of the signal in figure 4;
- figure 6 shows the XY plot between the signal in figure 2 and the signal in figure 4;
- figure 7 shows the signal in figure 2 as processed by the nonlinear dynamical system forming part of the system in figure 1, in the case in which there is a coupling between the signal in figure 2 and said nonlinear dynamical system which is not zero but at the same time not so strong as to overwhelm the spontaneous activity of the said nonlinear dynamical system;
- figure 8 shows the frequency spectrum of the signal in figure 7;
- figure 9 shows the XY plot between the signal in figure 2 and the signal in figure 7;
- figure 10 shows a schematization of a variant of the system in figure 1;
- figure 11 shows a two-dimensional matrix whose elements express an average synchronization value between twenty-one electroencephalographic signals acquired by a person to whom the system in figure 10 is applied;
- figure 12 shows a two-dimensional matrix whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value;
- figure 13 shows a two-dimensional matrix whose elements express a mean synchronization value between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value, as well as respectively processed by twenty-one nonlinear dynamical systems forming parts of a dynamical nonlinear network included in the system in figure 10, in the case in which there is substantially no coupling between said nonlinear dynamical systems of said network;
- figure 14 shows a two-dimensional array whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 13 expresses an average synchronization value;
- figure 15 shows a two-dimensional matrix whose elements express an average synchronization value between the twenty-one signals of which the matrix in figure 11 expresses an average synchronization value, as well as respectively processed by the twenty-one nonlinear dynamical systems forming parts of the dynamical network linear included in the system in figure 10, in the case in which there is a non-zero coupling between said nonlinear dynamical systems of said network;
- figure 16 shows a two-dimensional array whose elements express a synchronization difference between the twenty-one signals of which the matrix in figure 15 expresses an average synchronization value.
In the continuation of the present description, a figure may also be shown with reference to elements not expressly indicated in that figure but in other figures. The scale and proportions of the different elements depicted do not necessarily correspond to the actual ones.
Figure 1 shows a brain-computer interface system 1 applied to a person 2 in order to recognize an imaginary action conceived by the latter or performed by the latter's mind and to command, upon recognition of said imaginary action, the execution of a physical action (possibly realizing said imaginary action) to a machine 3.
The system 1, object of invention, includes a device suitable for acquiring a physiological signal associated with the brain activity of the person 2. This device, previously identified with the expression "means of acquisition", can comprise, by way of example, an electrode for electroencephalography or an optode for near-infrared spectroscopy. Assuming that the physiological signal is acquired by an electrode 4 for electroencephalography (shown, in figure 1, applied to the scalp of person 2), said signal corresponds to an electroencephalographic signal.
The signal acquired by the electrode 4 is sent to an amplifier 5 which, in addition to amplifying said signal, can convert it to the digital domain. The signal thus amplified is transmitted to a filter 6, preferably of the band-pass type, in order to attenuate the components of the signal corresponding to non-relevant activities of the person 2. More precisely, the filter 6 preferably attenuates both the frequencies which are too low (because they are artifacts of the movements of the person 2), and the frequencies which are too high (because they are artifacts of the muscular activity of the person 2). The filtered signal is sent to a nonlinear transform 7 (for example, a sigmoidal function) to reshape its distribution conveniently. In particular, the filtered signal is transformed so as to emphasize certain ranges of the dynamics of said signal, that is to say in order to highlight the signal components most relevant in brain-computer interfacing for the purpose of recognizing states cerebral including imaginary actions (for example, zero-crossing events).
The signal thus transformed is fed into a nonlinear dynamical system 8 comprising at least two, and preferably at least three, differential equations 9 (shown in figure 1, by way of example, in a number equal to three: "d1", "d2" and "d3"). Each equation 9 specifies a time course of a variable as a function of said variable and/or of at least another variable also included in another equation 9, so that each equation 9 is interdependent with at least another equation. The transformed signal (received as input by the system 8) corresponds to at least one variable of the equations 9, for example identifiable with the letter "s". At least one of the equations 9 comprises said variable "s" multiplied by a first coefficient (of coupling between the brain activity of the person 2 and the system 8), for example identifiable with the letter "k". Each of the equations 9 further comprises at least one variable that does not correspond to the transformed signal (that is, that is not "s") and multiplied by a second coefficient (of mutual coupling between the variables of the equations 9, i.e., between the variables of the system 8). The second coefficients are for example three and are respectively identifiable with the letters "a", "b" and "c".
Incidentally, for the avoidance of doubt, the transformed signal "s" can be entered in one or more of the equations 9 of system 8, as well as in all equations 9. In addition to this, an equation 9 can be interdependent with one or more of the other equations 9 of system 8, as with all equations 9.
As it is possible to note in figure 1, the system 1 comprises a first memory 10 where at least one value assumed by the coefficient "k" can be stored, and a second memory 11 where, for each of the coefficients "a", "b" and "c", at least one value assumed by said coefficient "a", "b" or "c" can be stored.
The signal thus processed (by the system 8) is sent to a feature extractor 13 (previously identified with the expression "extraction means") suitable to extract from the processed signal characteristics of the latter relevant to the classification of one brain state of the person 2. Incidentally, the signal that is sent to feature extractor 13 corresponds to one or more of the variables of the equations 9.
The feature extractor 13 generates a feature vector that is entered in a numerical classifier 14 (preferably, but not necessarily, a neural network) previously trained for the recognition, from said features extracted from the feature extractor 13, of a brain state of the person 2 including an imaginary action. In other words, the classifier 14 is suitable for examining said characteristics so as to establish whether the brain state of the person 2 (corresponding to said characteristics) comprises said imaginary action, so as to recognize or not the presence of the latter in the brain activity which is associated with the physiological signal from which said characteristics have been extracted.
The classifier 14, upon recognition of a brain state comprising said imaginary action, issues a command to execute a physical action, for example "concretizing" said imaginary action. Said command is preferably sent to the machine 3 which is suitable for carrying out said physical action (possibly corresponding to the imaginary action comprised in a brain state of the person 2 to the recognition of which the classifier 14 has been trained). The machine 3, upon receipt of said command from classifier 14, performs said physical action. The machine 3 comprises, by way of example, a robotic arm or a screen showing a message.
By way of example, it is known how robotic arms are made that can receive simple high-level commands, such as "grab the red object", and execute them thanks to the guidance of an artificial vision system (L. Minati, N. Yoshimura, Y. Koike, Hybrid Control of a Vision-Guided Robot Arm by EOG, EMG, EEG Biosignals and Head Movement Acquired via a Consumer-Grade Wearable Device, IEEE Access 4(2016), 9528-9541).
By way of example, it is known how robotic arms are made that can receive simple high-level commands, such as "grab the red object", and execute them thanks to the guidance of an artificial vision system (L. Minati, N. Yoshimura, Y. Koike, Hybrid Control of a Vision-Guided Robot Arm by EOG, EMG, EEG Biosignals and Head Movement Acquired via a Consumer-Grade Wearable Device, IEEE Access 4(2016), 9528-9541).
Likewise, the method is known for making robots capable of walking in multiple speeds and gaits thanks to pattern generators, incidentally based on nonlinear dynamical systems, and placed under the control of variables that can be set on the basis of the output of a numerical classifier (L. Minati, M. Frasca, N. Yoshimura, Y. Koike, Versatile locomotion control of a hexapod robot using a hierarchical network of nonlinear oscillator circuits. IEEE Access 6(2018), 8042-8065).
Equally similarly, it is known how virtual keyboards are displayed on video that can be controlled through brain activity and are designed to allow a paralyzed subject to express words and sentences (L.A. Farwell, E. Donchin, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, Electroencephalogr Clin Neurophysiol 70(1988), 510-23).
The system 8 tends to synchronize with the brain activity of the person 2 so that the signal output by the system 8 (and sent to the feature extractor 13) is correlated with the physiological signal acquired by the electrode 4. The value of the coefficient "k" and the value of each of the coefficients "a", "b" and "c" are such that system 8:
・ have, preferably but not necessarily, intrinsic dynamics (i.e., generate a spontaneous activity)
and
・ processes the signal "s" (received as input by the system 8) in such a way that, in the signal that is sent to thefeature extractor 13, one or more of said characteristics relevant to the classification of a brain state of the person 2 are revealed or enhanced.
・ have, preferably but not necessarily, intrinsic dynamics (i.e., generate a spontaneous activity)
and
・ processes the signal "s" (received as input by the system 8) in such a way that, in the signal that is sent to the
In other words, the value of the coefficient "k" and the value of each of the coefficients "a", "b" and "c" are chosen in such a way that the signal output by the system 8 and sent to the feature extractor 13 "helps" the latter to distinguish certain characteristics of the input signal, so as to facilitate the task of the classifier 14.
By way of example only, a first known nonlinear dynamical system comprising two differential equations is the Stuart-Landau system, which has the following form:
dx(t)/dt = α・x(t) - ω・y(t) - x(t)・(x(t)2 + y(t)2)
dy(t)/dt = ω・x(t) + α・y(t) - y(t)・(x(t)2 + y(t)2)
where "α" represents a bifurcation parameter and "ω" is the natural frequency of the oscillator.
dx(t)/dt = α・x(t) - ω・y(t) - x(t)・(x(t)2 + y(t)2)
dy(t)/dt = ω・x(t) + α・y(t) - y(t)・(x(t)2 + y(t)2)
where "α" represents a bifurcation parameter and "ω" is the natural frequency of the oscillator.
Again by way of example, a second known nonlinear dynamical system comprising two differential equations is the Rossler system, which has the following form:
dx(t)/dt = - y(t) - z(t)
dy(t)/dt = x(t) + a・y(t)
dz(t)/dt = b + z(t)・(x(t) - c)
where "a" is a parameter that controls the chaoticity level of the system, and "b" and "c" are usually set to constant values.
dx(t)/dt = - y(t) - z(t)
dy(t)/dt = x(t) + a・y(t)
dz(t)/dt = b + z(t)・(x(t) - c)
where "a" is a parameter that controls the chaoticity level of the system, and "b" and "c" are usually set to constant values.
As a further example, a Rossler system that receives the signal "s(t)" coupled to its variable "x(t)" as input has the following form:
dx(t)/dt = - y(t) - z(t) + k・(s(t) - x(t))
dy(t)/dt = x(t) + a・y(t)
dz(t)/dt = b + z(t)・(x(t) - c)
dx(t)/dt = - y(t) - z(t) + k・(s(t) - x(t))
dy(t)/dt = x(t) + a・y(t)
dz(t)/dt = b + z(t)・(x(t) - c)
Preferably, but not necessarily, if the system 8 includes at least three differential equations 9, the value of the coefficient "k" and the value of each of the coefficients "a", "b" and "c" are chosen in such a way that the system 8 has a chaotic behavior.
Preferably, but not necessarily, the value of the coefficient "k" and the value of each of the coefficients "a", "b" and "c" are chosen in such a way that in the system 8 emergence phenomena occur, as predominantly observed, for example, near a transition between periodic and chaotic dynamics, and between presence and absence of intrinsic dynamics (understood as sustained oscillation even in the absence of external stimuli), i.e., near a so-called "critical point". The critical points of nonlinear dynamical systems, as well as their intrinsic dynamics, are substantially known. We therefore do not dwell on providing further details.
Preferably, but not necessarily, the feature extractor 13, in addition to receiving the signal processed by the system 8, also receives the signal transformed by the nonlinear transform 7 but not processed by the system 8. The feature extractor 13 is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 not only from the signal transformed by the nonlinear transform 7 after having been processed by the system 8, but also by the signal transformed by the nonlinear transform 7 and not processed by the system 8.
The system 1 preferably, but not necessarily, comprises a device 15 (for example, a micro-switch) operable by a user to express their own evaluation of the correspondence between the imaginary action recognized by the classifier 14 and the physical action performed by the machine 3. Said user could correspond to the person 2. In this case, the device 15 can be activated by the person 2 to express his own evaluation of the correspondence between the imaginary action eventually thought of her and the physical action performed by the machine 3. The device 15, previously identified with the expression "feedback means", communicates the evaluation expressed by the person 2 to a supervision unit 16 that also receives the signal transformed by the nonlinear transform 7 but not processed by the system 8 The unit 16 is suitable for updating the value of the coefficient "k" and the value of each of the coefficients "a", "b" and "c" according to the assessment of such correspondence communicated by the person 2.
According to a variant of system 1, the latter does not include the nonlinear transform 7. According to this variant:
・ the system 8 processes the signal as filtered by the filter 6;
・ thefeature extractor 13, in addition to receiving as input the signal processed by the system 8, preferably (but not necessarily) receives also the signal filtered by the filter 6 but not processed by the system 8. The feature extractor 13, according to this variant of the system 1, is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 not only from the signal filtered by the filter 6 after having been processed by the system 8, but also by the signal filtered by the filter 6 and not processed by the system 8;
・ theunit 16, if present, also receives the signal filtered by the filter 6 but not processed by the system 8.
・ the system 8 processes the signal as filtered by the filter 6;
・ the
・ the
According to a variant of the system 1, the supervision unit 16 also receives the signal as processed by the system 8 at the input and is suitable to update the value of the coefficient "k" and the value of each of the coefficients "a" , "b" and "c" also as a function of the aforementioned signal as processed by the system 8.
Figure 2 shows an example of an electroencephalographic signal 20 acquired by the person 2 by means of the electrode 4. More precisely, the signal shown in figure 2 corresponds to the aforementioned signal as sent to the system 8 after having been amplified by the amplifier 5, filtered by the filter 6 and transformed by the nonlinear transform 7. The dotted line marked by the number 21 corresponds to the instant in which the person 2 started to think of an imaginary action to the recognition of which (that is, to the recognition of a brain state comprising which) the classifier 14 has been previously trained. As it is possible to note in figure 2, the trajectory of the signal 20 is visually different before and after the instant 21. Before the instant 21 the signal 20 is in fact characterized by large amplitude oscillations and relatively periodic. After the instant 21 the signal 20 instead presents a more irregular time course. However, this difference is extremely slight and it is precisely the weak nature of these differences that makes it necessary to develop systems such as the subject of the invention.
Figure 3 shows the frequency spectrum 22 of the signal 20.
Figure 4 shows the time course of a first example of a signal 25 sent by the system 8 to the feature extractor 13 following the input reception of the signal 20. In other words, the signal 25 is the result of the signal processing 20 by the system 8. In the example of figure 4, the parameter "k" is equal to zero. This means that, in this example, there is no coupling between the brain activity of the person 2 and the system 8. The signal 25 therefore has no correlation with the signal 20 and is due exclusively to the spontaneous activity of the system 8.
Figure 5 shows the frequency spectrum 26 of the signal 25.
Figure 6 shows the XY plot 27 between the signals 20 and 25. The fact that the XY plot 27 substantially corresponds to a large rectangle is a consequence of the fact that between the signals 20 and 25 there is no synchronization. Nevertheless, from a comparison between the graphs respectively shown in figures 2 and 4 it is possible to notice that the signal 25, although not correlated to the signal 20, has a frequency content which in large part overlaps with the latter's content. This is confirmed by the partial overlap found by comparing the frequency spectra 22 and 26. The above is due to the appropriate way in which the differential equations 9 that make up system 8 have been chosen, and the values that have been assigned to the parameters "a", "b" and "c" of the same.
Incidentally, if, conversely to the example of figure 4, the parameter "k" tends to infinity, between the brain activity of the person 2 and the system 8 there would be a coupling so strong as to overwhelm the activity spontaneous of the system 8. In this case, the signal leaving the system 8 would be a replica of the signal 20 and the XY plot between said signals would be very flattened along a diagonal segment (since there would be almost complete synchronization between them).
Figure 7 shows the time course of a second example of signal 30 sent by the system 8 to the feature extractor 13 following the input reception of the signal 20. Similarly to what was said for the signal 25 of figure 4, the signal 30 is the result of the processing of the signal 20 by the system 8. In the example of figure 7, the parameter "k" was chosen greater than zero but not so large as to overwhelm the spontaneous activity of the system 8. This example corresponds to the preferred mode of use of the system 1. In other words, the signal 20 is injected into the system 8 sufficiently strongly as to influence the behavior of the latter without, however, overcoming its intrinsic dynamics. The values of the parameters "a", "b" and "c" are also appropriately chosen (preferably through an initial training phase) so that the signal leaving the system 8 helps the feature extractor 13 to distinguish certain characteristics of the signal 20 in order to facilitate the work of the classifier 14.
Figure 8 shows the frequency spectrum 31 of the signal 30.
Figure 9 shows the XY plot 32 between the signals 20 and 30. The fact that the XY plot 32 substantially corresponds to a flattened and inclined ellipsoid resembling a straight line having a positive slope is a consequence of the fact that between the signals 20 and 25 there is a partial synchronization. In addition to this, also the distribution of the amplitudes of the frequency spectra takes a form much more similar to the starting electroencephalographic signal with respect to the case with null coupling, which is characterized by a superimposable but very different shape frequency range.
Incidentally, although it is preferable that the system 8 has intrinsic dynamics, the values assigned to the parameters "a", "b" and "c" could be such that the system 8 has no intrinsic dynamics, or, although not having intrinsic dynamics, it is very close to having them. When the system 8 is in this last condition, it is said that it is close to a so-called critical point, i.e., it is maximally responsive to external stimuli.
Figure 10 shows a brain-computer interface system 33 which differs from the system 1 in that it comprises a plurality of devices suitable for acquiring respective physiological signals associated with the person's brain activity 2. Said devices, previously identified with the expression "acquisition means" may include, by way of example, respective electrodes for electroencephalography or respective optodes for near-infrared spectroscopy. Assuming that the physiological signals are acquired by means of respective electrodes 34 for electroencephalography (shown, in figure 10, applied to the scalp of the person 2), said signals correspond to respective electroencephalographic signals. In figure 10 the electrodes 34 are shown, by way of example, in a number equal to three.
Each of the signals acquired by the electrodes 34 is sent to an amplifier 35 which, like the amplifier 5, besides converting said signal, can convert it to the digital domain. Each of the signals thus amplified is transmitted to a filter 36, preferably of the band-pass type, in order to attenuate the signal components corresponding to non-relevant activities of the person 2. More precisely, like the filter 6, the filters 36 preferably attenuate both the frequencies that are too low (because they are the artifacts of the movements of the person 2, or rather of non-neural physiological activity), and the frequencies that are too high (because they are artificial muscular activity of the person 2). Each of the signals thus filtered is sent to a nonlinear transform 37 (for example, a sigmoidal function) to reshape its distribution conveniently. In particular, similarly to what occurs in the system 1, each of the filtered signals is transformed so as to emphasize certain ranges of the dynamics of the said signal, that is, so as to highlight the signal components most relevant for brain-computer interfacing, such as zero-crossing events, in order to recognize brain states including imaginary actions.
The signals thus transformed are fed into a nonlinear dynamical network 38 comprising a plurality of nonlinear dynamical systems 8 (shown in figure 10, by way of example, in a number equal to three, "Sys1", "Sys2" and " Sys3 ", being three electrodes 34). In other words, each of the signals thus transformed is fed into the respective system 8 of the network 38 (each of which including at least two, and preferably at least three, equations 9). The systems 8 of the network 38 are therefore respectively suitable for processing the signals transformed by the nonlinear transforms 37. Each signal transformed by one of the nonlinear transforms 37 is therefore sent as input to a respective system 8 of the network 38. Each system 8 of the network 38 comprises at least one equation 9 specifying a time course of a variable as function not only of said variable and/or of at least another variable also included in another equation 9 of said system 8 (as in system 1), but preferably also of at least another variable also included in another equation 9 of another system 8 of the network 38, so that not only each equation 9 of a system 8 of the network 38 is interdependent with at least another equation 9 of said system 8, but each system 8 of the network 38 is also interdependent with at least another system 8 of the network 38.
Each of the transformed signals received as input by a system 8 of the network 38 corresponds to at least one variable of the equations 9 of the said system 8. The variable of the i-th transformed signal can for example be identified with "si".
In each system 8 of the network 38, at least one of the equations 9 comprises said variable "si" multiplied by the previously mentioned "first coefficient". More precisely, in the i-th system 8 of the network 38, the first coefficient multiplied by the variable "si" can for example be identified with "ki" (corresponding to the coupling coefficient between the brain activity of the person 2 associated with the i-th transformed signal and the i-th system 8 of the network 38).
In each system 8 of the network 38, each of the equations 9 further comprises at least one variable not corresponding to the transformed signal (that is, that it is not "si") and not included in an equation 9 of another system 8 of the network 38, multiplied by the previously mentioned "second coefficient". More precisely, in the i-th system 8 of the network 38, the second coefficients are for example three and are respectively identifiable with the letters "ai", "bi" e "ci" (corresponding to the mutual coupling coefficients between the variables of the equations 9 of the i-th system 8 of the network 8).
In each system 8 of the network 38, at least one of the equations 9 comprises at least one variable not corresponding to the transformed signal (that is, that it is not "si") and also included in an equation 9 of another system 8 of the network 38, multiplied by a third coefficient (of mutual coupling between the systems 8 of the network 38). More precisely, in the i-th system 8 of the network 38, the coefficient multiplied by a variable of the j-th system 8 of the network 38 can for example be identified with "gi,j".
For each system 8 of the network 38, the memory 10 is suitable for storing at least one value assumed by the coefficient "ki" and the memory 11 is suitable for storing, for each of the coefficients "ai", "bi" e "ci", at least one value assumed by said coefficient "ai", "bi" o "ci".
As it is possible to note in figure 10, the system 33 also includes a third memory 39 where it can be memorized, for each of the coefficients "gi,j", at least one value assumed by said coefficient "gi,j".
The signals processed in this way (from the systems 8 of the network 38) are sent to a unit 40 (denoted, in Figure 10, by "Syn1") suitable for evaluating how much each of the received input signals (thus processed by the systems 8 of the network 38) is synchronized with each of the other input signals received. Incidentally, the signal that is sent to the unit 40 corresponds to one or more of the variables of the equations 9.
The unit 40, previously identified with the expression "first means of evaluation", generates a synchronization (or, more generally, statistical interdependence) matrix concerning the signals received at the input. Said synchronization matrix is sent to the feature extractor 13 which is suitable for extracting from the signals processed by the systems 8 of the network 38 as inserted in said synchronization matrix (i.e., after having been evaluated by the unit 40), characteristics relevant to the classification of a brain state of the person 2.
Similarly to what was said for the system 1, in the system 33 the feature extractor 13 generates a feature vector which is entered in the numerical classifier 14 previously trained to the recognition, from said features extracted from the feature extractor 13, of a brain state of the person 2 including an imaginary action. In other words, the classifier 14 is suitable for examining said characteristics so as to establish whether the brain state of the person 2 (corresponding to said characteristics) comprises said imaginary action, so as to recognize or not the presence of the latter in the brain activity which are associated the physiological signals from which said characteristics have been extracted. The classifier 14, upon recognition of a brain state comprising said imaginary action, emits a command to execute a physical action possibly "materializing" said imaginary action. Said command is preferably sent to the machine 3 which performs said physical action.
The systems 8 of the network 38 tend to synchronize with the brain activity of the person 2 so that the signals sent to the feature extractor 13 are correlated with the physiological signals acquired by the electrodes 34. In each system 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" are such that said system 8:
・ it has, preferably but not necessarily, intrinsic dynamics
and
・ processes the signal "si" (received as input by said system 8) in such a way that, in the signal that is sent to the unit 40 (and, through the latter, to the feature extractor 13), one or more of said characteristics relevant to the purposes of the classification of a brain state of theperson 2 are revealed and/or enhanced.
・ it has, preferably but not necessarily, intrinsic dynamics
and
・ processes the signal "si" (received as input by said system 8) in such a way that, in the signal that is sent to the unit 40 (and, through the latter, to the feature extractor 13), one or more of said characteristics relevant to the purposes of the classification of a brain state of the
In other words, in each system 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" are chosen in such a way that the signal outgoing from said system 8 and sent to the unit 40 "helps" the feature extractor 13 to distinguish certain characteristics of the input signals, so as to facilitate the task of the classifier 14.
By way of example only, two Rossler systems coupled bi-directionally (i.e., symmetrically) at their variable "x(t)", and which receive the signals "s1 (t)" and "s2 (t)" coupled respectively to their variables "x1 (t)" and "x2 (t)" have the following form
dx1(t)/dt = - y1(t) - z1(t) + k1・(s1(t) - x1(t)) + g・(x2(t) - x1(t))
dy1(t)/dt = x1(t) + a1・y1(t)
dz1(t)/dt = b1 + z1(t)・(x1(t) - c1)
dx2(t)/dt = - y2(t) - z2(t) + k2・(s2(t) - x2(t)) + g・(x1(t) - x2(t))
dy2(t)/dt = x2(t) + a2・y2(t)
dz2(t)/dt = b2 + z2(t)・(x2(t) - c2)
dx1(t)/dt = - y1(t) - z1(t) + k1・(s1(t) - x1(t)) + g・(x2(t) - x1(t))
dy1(t)/dt = x1(t) + a1・y1(t)
dz1(t)/dt = b1 + z1(t)・(x1(t) - c1)
dx2(t)/dt = - y2(t) - z2(t) + k2・(s2(t) - x2(t)) + g・(x1(t) - x2(t))
dy2(t)/dt = x2(t) + a2・y2(t)
dz2(t)/dt = b2 + z2(t)・(x2(t) - c2)
Preferably, but not necessarily, if at least one of the systems 8 of the network 38 comprises at least three differential equations 9, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" in said system 8 are chosen in such a way that said system 8 comprising at least three differential equations 9 has a chaotic behavior.
Preferably, but not necessarily, in one or more systems 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" are chosen in such a way that in one or more said systems 8 of the network 38 emergence phenomena occur.
Preferably, but not necessarily, the signals transformed by the nonlinear transforms 37, in addition to being sent to the systems 8 of the network 38, are sent to a unit 41 (indicated, in Figure 10, by "Syn2") suitable for evaluating how much each of the input signals received (transformed by the nonlinear transforms 37 but not processed by the systems 8 of the network 38) is synchronized with each of the other signals received at the input. The unit 41, previously identified with the expression "second evaluation means", generates a second synchronization matrix concerning the signals received at the input. Said second synchronization matrix is sent to the feature extractor 13 together with the synchronization matrix generated by the unit 40. The feature extractor 13 is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 not only from the signals processed by the systems 8 of the network 38 as inserted in the synchronization matrix generated by the unit 40, but also from the signals transformed by the nonlinear transforms 37 and not processed by the systems 8 of the network 38 as inserted in said second synchronization matrix (i.e.. after having been evaluated by unit 41).
According to a variant of the system 33, the feature extractor 13 receives as input (and is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 from) not only the signals processed by the systems 8 of the network 38 as inserted in the synchronization matrix generated by the unit 40 and the signals transformed by the nonlinear transforms 37 and not processed by the systems 8 of the network 38 as inserted in the synchronization matrix generated by the unit 41, but also the signals processed by the systems 8 of the network 38 but not evaluated by the unit 40 and/or the signals transformed by the nonlinear transforms 37 and not processed by the systems 8 of the network 38, nor evaluated by the unit 41.
According to another variant of the system 33, the latter does not comprise the unit 40 and/or the unit 41. According to this variant the feature extractor 13 receives as input (and is therefore preferably suitable for extracting features relevant to the classification of a brain state of the person 2) the signals processed by the systems 8 of the network 38 and preferably also the signals transformed by the nonlinear transforms 37 and not processed by the systems 8 of the network 38.
According to another variant of the system 33, the feature extractor 13 does not receive input signals not processed by the systems 8 of the network 38, regardless of whether or not they have been evaluated by the unit 41.
Like the system 1, the system 33 preferably, but not necessarily, comprises the device 15 and the supervision unit 16 which also receives the signals transformed by the nonlinear transforms 37 but not processed by the systems 8 of the network 38. The unit 16 is suitable for updating, in each of the systems 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" according to the evaluation of the correspondence (between the imaginary action and the physical action) communicated by the person 2.
According to another variant of the system 33, the latter does not include the nonlinear transforms 37. According to this variant:
・ the systems 8 of thenetwork 38 process the signals as filtered by the filters 36;
・ thefeature extractor 13, in addition to receiving as input the signals processed by the systems 8 of the network 38, optionally evaluated by the unit 40, if present, preferably (but not necessarily) receive as input also the signals filtered by the filters 36 but not processed by the systems 8 of the network 38, optionally evaluated by the unit 41, if present. The feature extractor 13, according to this variant of the system 33, is therefore preferably suitable for extracting characteristics relevant to the classification of a brain state of the person 2 not only from the signals filtered by the filters 36 after having been processed by the systems 8 of the network 38 and possibly evaluated by the unit 40, if present, but also by the signals filtered by the filters 36 and not processed by the systems 8 of the network 38 and possibly evaluated by the unit 41, if present;
・ theunit 16, if present, also receives the signals filtered by the filters 36 but not processed by the systems 8 of the network 38.
・ the systems 8 of the
・ the
・ the
According to another variant of the system 33 wherein said devices suitable for acquiring respective physiological signals associated with the brain activity of the person 2 comprise the electrodes 34 (for electroencephalography), the system 33 comprises at least one spatial filter suitable for reconstructing, from the signals as filtered by the filters 36, a plurality of physiological signals as would be acquired if the electrodes 34 were placed on the cerebral cortex of the person 2. According to this variant, the systems 8 of the network 38 process the signals as filtered by the filters 36 after having been treated by said spatial filter, and possibly transformed if nonlinear transforms are present.
The system 33 preferably, but not necessarily, includes in addition to or as an alternative to the device 15, a device 42 (previously identified with the expression "further acquisition means") applicable to the person 2 for the acquisition of at least one further physiological signal not directly associated with the brain activity of the person 2. The system 33 preferably, but not necessarily, comprises a device 43 (previously identified with the expression "monitoring means") suitable for autonomously detecting an execution of the previously mentioned physical action by the machine 3. The device 42 includes, by way of example, an electrode for detecting the heartbeat of the person 2. The device 43 includes, by way of example, a computerized video camera. Both devices 42 and 43 communicate with the supervision unit 16 so that the latter can evaluate, from an examination of the aforementioned further physiological signal, the correspondence between the imaginary action conceived by the person 2 or performed by the mind of the latter and the physical action performed by the machine 3. The unit 16 is suitable for updating, in each of the systems 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" as a function of the evaluation of this correspondence carried out by the unit 16.
Also the system 1, like the system 33, in addition to or as an alternative to the device 15, could include the devices 42 and 43 connected with the supervision unit 16.
According to a variant of the system 33, the supervision unit 16 also receives the signals as processed by the systems 8 of the network 38 as input and is suitable to update, in each of the systems 8 of the network 38, the value of the coefficient "ki", the value of each of the coefficients "ai", "bi" e "ci" and the value of each of the coefficients "gi,j" also as a function of the aforesaid signals as processed by the systems 8 of the network 38.
Figure 11 shows an example of a two-dimensional array 45 whose elements express an average synchronization value between twenty-one electroencephalographic signals acquired by the person 2 respectively by means of twenty-one electrodes 34 (i.e., whose elements express an average synchronization value between each of said twenty-one electroencephalographic signals and each of the other twenty of said twenty-one electroencephalographic signals). More precisely, the signals of which the matrix in figure 11 expresses an average synchronization value correspond to the twenty-one said signals as well as respectively sent to twenty-one systems 8 of the network 38 after respectively being amplified by twenty-one amplifiers 35, respectively filtered by twenty-one filters 36 and respectively transformed by twenty-one nonlinear transforms 37.
For the avoidance of doubt, according to this example, the system 33 comprises twenty-one electrodes 34 for acquiring, respectively, twenty-one electroencephalographic signals which are respectively amplified by twenty-one amplifiers 35, filtered by twenty-one filters 36, transformed by twenty-one nonlinear transforms 37 and sent to twenty-one nonlinear dynamical systems 8 belonging to the network 38.
Figure 12 shows a two-dimensional matrix 46 whose elements express a synchronization difference (calculated by comparing two mental states of interest, in this case, a state of rest and the imaginary execution of repeated opening and closing of the hands) between the twenty-one signals of which the matrix 45 expresses an average synchronization value (i.e., whose elements express a synchronization difference between each of said twenty-one electroencephalographic signals and each of the other twenty of said twenty-one electroencephalographic signals).
Figure 13 shows an example of a two-dimensional array 47 whose elements express an average synchronization value between twenty-one signals respectively sent by the twenty-one systems 8 of the network 38 to the unit 40 following the respective input reception of the twenty-one signals of which the matrix 45 expresses an average synchronization value. In other words, the twenty-one signals of which the matrix 47 expresses an average synchronization value are respectively the result of the processing, by the twenty-one systems 8 of the network 38, of the twenty-one signals of which the matrix 45 expresses an average synchronization value. In the example of figure 13, the parameters "gi,j" are set to zero. This means that, in this example, there is no coupling between the systems 8 of the network 38. Every electroencephalographic signal acquired by the electrodes 34 is therefore processed by one of the systems 8 without the latter being in any way correlated with the other twenty systems 8 of the network 38 (which process the other twenty electroencephalographic signals).
In the example of figure 13, in each of the systems 8 of the network 38 the parameter "ki" was chosen greater than zero but not so large as to overwhelm the spontaneous activity of the system 8. As a consequence of this fact, the distribution of the synchronization between the output signals is rather similar to that observed for the input signals in figure 11. As previously said, this corresponds to the preferred mode of use of the systems 8. In other words, the signals which are sent to the systems 8 of the network 38 are respectively fed into the said systems 8 in a sufficiently strong manner to influence the behavior of the latter without, however, overcoming their intrinsic dynamics. Moreover, in each of the systems 8 of the network 38 the values assigned to the parameters "ai", "bi" e "ci" are suitably chosen (preferably by means of an initial training phase) so that the signals leaving the systems 8 help the feature extractor 13 to distinguish certain characteristics introduced into the network 38 so as to facilitate the work performed by the classifier 14.
Figure 14 shows a two-dimensional matrix 48 whose elements express a synchronization difference (calculated by comparing two mental states of interest, in this case, a state of rest and the imaginary execution of repeated opening and closing of hands) between the twenty-one signals of which the matrix 47 expresses an average synchronization value.
Although in this example there is no synchronization between the systems 8 of the network 38, it is possible to notice a certain similarity between the matrix 45 and the matrix 47, and between the matrix 46 and the matrix 48. This similarity is due to the fact that the systems 8 of the network 38 are individually coupled to the input signals, but not between them.
Figure 15 shows a second example of a two-dimensional array 49 whose elements express an average synchronization value between twenty-one signals respectively sent by the twenty-one systems 8 of the network 38 to the unit 40 following the respective reception as input of the twenty-one signals of to which the matrix 45 expresses an average synchronization value. Similarly to what has been said for the matrix 47, the twenty-one signals of which the matrix 49 expresses an average synchronization value are the result of the processing, by the twenty-one respective systems 8 of the network 38, of the twenty-one signals of which the matrix 45 expresses an average synchronization value. In the example of figure 15, the parameters "gi,j" have been chosen larger than zero, but not so large so that the activity of a system 8 of the network 38 overwhelms the activity of the other systems 8 of the network 38, so that there is a coupling between the systems 8 of the network 38. This example corresponds to the preferred mode of use of the system 33. Furthermore, similarly to what has been said with reference to the example of figure 13, in each of the systems 8 of the network 38 the parameter "ki" has been chosen greater than zero but not so large as to overwhelm the spontaneous activity of the system 8 has been chosen (the signals that are sent to the systems 8 of the network 38 are in other words introduced into said systems 8 in a sufficiently strong manner to influence the behavior of the latter without, however, overwhelming its intrinsic dynamics). Moreover, in each of the systems 8 of the network 38 the values assigned to the parameters "ai", "bi" e "ci" are also appropriately chosen (preferably through an initial training phase) so that the signals leaving the systems 8 help the feature extractor 13 to distinguish certain characteristics introduced into the network 38 so as to facilitate the work carried out by the classifier 14. It should be noted, in this case, since the systems 8 are coupled together, the average level of synchronization in the matrix 49 is higher. In addition to this, it is possible to note that the synchronization distribution is markedly different with respect to the matrices 45 and 47, and this is precisely because the systems 8 interact with each other, combining the signals received individually as input with via interactions of the both the attractive and repulsive type.
Figure 16 shows a two-dimensional matrix 50 whose elements express a synchronization difference (calculated by comparing two mental states of interest, in this case, a state of rest and the imaginary execution of repeated opening and closing of the hands) between the twenty-one signals of which the matrix 49 expresses an average synchronization value. As observed for the matrix 49, it is noted that the distribution is clearly different with respect to the matrices 46 and 48, and this for the same reasons indicated above. Incidentally, we note that in this case the difference between the two conditions has a more regular distribution, and focuses on some signals, which specifically correspond to those recorded above the sensory-motor cerebral cortex.
Incidentally, similarly to what was said for the system 1, in the system 33, although it is preferable that the systems 8 of the network 38 have intrinsic dynamics, the values assigned to the parameters "ai", "bi" e "ci" could be such that one or more systems 8 of the network 38 do not have intrinsic dynamics, or, although not having intrinsic dynamics, they are very close to having them (that is, in the so-called critical point).
On the basis of the description provided for a preferred example embodiment, it is obvious that some changes may be made by those skilled in the art without departing from the scope of the invention as defined by the following claims.
Claims (17)
- Brain-computer interface system (1, 33) comprising:
・ means (4, 34) applicable to a person (2) for the acquisition of at least one physiological signal associated with the brain activity of said person (2);
・ means (5, 35) for amplifying said physiological signal;
・ means (6, 36) for filtering said amplified signal;
・ means (13) for extracting, from said filtered signal, characteristics relevant to the classification of a brain state of said person (2);
・ at least one numerical classifier (14) of said characteristics extracted from said extraction means (13),
said classifier (14) being trained in the recognition, from said features extracted from said extraction means (13), of a brain state of said person (2) comprising an imaginary action,
said classifier (14) being suitable, upon recognition of a brain state comprising said imaginary action, to emit at least one execution command of a physical action,
said brain-computer interface system (1, 33) being characterized in that it further comprises:
・ at least one nonlinear dynamical system (8) for processing said filtered signal and sending the processed signal (30) to said extraction means (13),
said extraction means (13) being means for extracting said features from said filtered signal after having been processed by said nonlinear dynamical system (8),
said nonlinear dynamical system (8) comprising at least two differential equations (9),
each of said equations (9) specifying a time pattern of a variable as a function of said variable and/or at least another variable also included in another of said equations (9), so that each of said equations (9) is interdependent with at least another of said equations (9),
said filtered signal corresponding to at least one of said variables of said equations (9),
at least one of said equations (9) comprising said variable corresponding to said filtered signal, multiplied by a first coefficient (k, ki),
each of said equations (9) comprising at least one of said variables not corresponding to said filtered signal, multiplied by a second coefficient (a, b, c, ai, bi, ci),
said signal (30) processed and sent to said extraction means (13) corresponding to one or more of said variables of said equations (9),
・ a first memory (10) for storing at least one value assumed by said first coefficient (k, ki);
・ a second memory (11) for storing, for each of said second coefficients (a, b, c, ai, bi, ci), at least one value assumed by said second coefficient (a, b, c, ai, bi, ci),
the value of said first coefficient (k, ki) and the value of each of said second coefficients (a, b, c, ai, bi, ci) being such that said nonlinear dynamical system (8) processes said filtered signal in such a way that, in said signal (30) sent to said extraction means (13), one or more of said characteristics relevant to the purposes of the classification of a brain state of said person are reveled and/or enhanced. - Brain-computer interface system (33) according to claim 1, characterized in that it comprises:
・ a plurality of said acquisition means (34), respectively for acquiring a plurality of physiological signals associated with the brain activity of said person (2);
・ a plurality of said amplification means (35), respectively for the amplification of said plurality of physiological signals;
・ a plurality of said filtering means (36), respectively for filtering said amplified signals;
・ a nonlinear dynamical network (38) comprising a plurality of said nonlinear dynamical systems (8), respectively for processing said filtered signals and sending the processed signals to said extraction means (13),
said extraction means (13) being means for extracting said features from said filtered signals after having been processed by the nonlinear dynamical systems (8) of said network (38),
each nonlinear dynamical system (8) of said network (38) comprising at least one differential equation (9) specifying a time course of a variable as a function of:
- said variable
and/or of
- at least one other variable also included in another of said equations (9) of said nonlinear dynamical system (8)
and/or of
- at least one other variable also included in one of said equations (9) of another nonlinear dynamical system (8) of said network (38)
in such a way that each linear dynamical system (8) of said network (38) is interdependent with at least one other nonlinear dynamical system (8) of said network (38),
each of said filtered signals corresponding to at least one of said variables of said equations (9) of a nonlinear dynamical system (8) of said network (38),
in each nonlinear dynamical system (8) of said network (38):
- at least one of said equations (9) comprising said variable corresponding to one of said filtered signals, multiplied by said first coefficient (ki);
- each of said equations (9) comprising at least one of said variables not corresponding to one of said filtered signals and not included in one of said equations (9) of another nonlinear dynamical system (8) of said network (38), multiplied by said second coefficient (a, b, c, ai, bi, ci);
- at least one of said equations (9) comprising at least one of said variables not corresponding to one of said filtered signals and also included in one of said equations (9) of another nonlinear dynamical system (8) of said network (38), multiplied by a third coefficient (gi,j);
- said first memory (10) being suitable for storing at least one value assumed by said first coefficient (ki);
- said second memory (11) being suitable for memorizing, for each of said second coefficients (a, b, c, ai, bi, ci), at least one value assumed by said second coefficient (a, b, c, ai, bi, ci),
said signals sent to said extraction means (13) corresponding to said variables of said equations (9);
・ a third memory (39) for storing, for each of said third coefficients (gi,j), at least one value assumed by said third coefficient (gi,j),
for each nonlinear dynamical system (8) of said network (38), the value of said first coefficient (ki), the value of each of said second coefficients (a, b, c, ai, bi, ci) and the value of each of said third coefficients (gi,j) being such that said nonlinear dynamical system (8) processes said filtered signal in such a way that, in said signal sent to said extraction means (13) one or more of said characteristics relevant to the classification of the brain state of that person are revealed and/or enhanced. - Brain-computer interface system (1, 33) according to claim 1 or 2, characterized in that said extraction means (13) are means for extracting said features
not only:
・ from said filtered signal after having been processed by said nonlinear dynamical system (8)
or
・ from said filtered signals after having been processed by the nonlinear dynamical systems (8) of said network (38), if said linear dynamical systems (8) are more than one,
but also:
・ from said filtered signal but not processed by said nonlinear dynamical system (8)
or
・ from said signals filtered but not processed by the nonlinear dynamical systems (8) of said network (38), if said linear dynamical systems (8) are more than one. - Brain-computer interface system (1, 33) according to claim 1 or 2, characterized in that it comprises:
・ a nonlinear transform (7) for:
- the transformation of said filtered signal so as to emphasize certain ranges of the dynamics of said signal, and
- sending the transformed signal (20) to said nonlinear dynamical system (8),
said nonlinear dynamical system (8) being suitable for processing said filtered signal after having been transformed by said nonlinear transform (7),
or, if said linear dynamical systems (8) are more than one:
・ a plurality of nonlinear transforms (37), respectively for:
- the transformation of said filtered signals so as to emphasize certain ranges of the dynamics of said signals, and
- sending the transformed signals to the nonlinear dynamical systems (8) of said network (38),
the nonlinear dynamical systems (8) of said network (38) being respectively suitable for processing said filtered signals after having been transformed by said nonlinear transforms (37). - Brain-computer interface system (1, 33) according to claim 4, characterized in that said extraction means (13) are not only means of extracting said features:
・ from said transformed signal (20) after having been processed by said nonlinear dynamical system (8)
or
・ from said transformed signals after having been processed by the nonlinear dynamical systems (8) of said network (38), if said linear dynamical systems (8) are more than one,
but also:
・ from said transformed signal (20) but not processed by said nonlinear dynamical system (8)
or
・ from said transformed but not processed signals from the nonlinear dynamical systems (8) of said network (38), if said linear dynamical systems (8) are more than one. - Brain-computer interface system (33) according to claim 2, or according to claim 3 when dependent on claim 2, or according to claim 4 when dependent on claim 2, or according to claim 5, characterized in that it comprises first evaluation means (40) for:
・ the reception as input of said signals processed by the nonlinear dynamical systems (8) of said network (38),
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a first synchronization matrix concerning said input signals,
said extraction means (13) being means for extracting said characteristics from said signals processed by the nonlinear dynamical systems (8) of said network (38) as inserted in said first synchronization matrix. - Brain-computer interface system (33) according to claim 6, characterized in that it comprises second evaluation means (41) for:
・ the reception as input of said filtered and possibly transformed signals if said nonlinear transforms (37) are present, but not processed by the nonlinear dynamical systems (8) of said network (38),
・ the evaluation of how much each of said input signals is synchronized with each of the others of said received input signals, and
・ the generation of a second synchronization matrix concerning said received input signals,
said extraction means (13) being means for extracting said features not only from said signals processed by nonlinear dynamical systems (8) of said network (38) as inserted in said first synchronization matrix, but also by said filtered signals, and possibly transformed if said nonlinear transforms (37) are present, but not processed by the nonlinear dynamical systems (8) of said network (38) as inserted in said second synchronization matrix. - Brain-computer interface system according to claim 7, characterized in that said extraction means (13) are not only means of extracting said features:
・ from said signals processed by the nonlinear dynamical systems (8) of said network (38) as inserted in said first synchronization matrix,
and
・ from said filtered and possibly transformed signals if said nonlinear transforms (37) are present but not processed by the nonlinear dynamical systems (8) of said network (38) as inserted in said second synchronization matrix,
but also:
・ from nonlinear transforms (37) are present but not processed by the nonlinear dynamical systems (8) of said network (38) as inserted in said second synchronization matrix,
and/or
・ from said filtered and possibly transformed signals if said nonlinear transforms (37) are present, but not processed by the nonlinear dynamical systems (8) of said network (38), nor evaluated by said second evaluation means (41). - Brain-computer interface system according to claim 2, or according to claim 3 when dependent on claim 2, or according to claim 4 when dependent on claim 2, or according to claim 5 when claim 4 is dependent on claim 2, or according to claim 6 when dependent on claim 2, or according to claim 7 when claim 6 is dependent on claim 2, or according to claim 8 when claim 6 is dependent on claim 2,
characterized in that said plurality of acquisition means comprises a plurality of electrodes for electroencephalography,
said brain-computer interface system comprising further filtering means including at least one spatial filter suitable for reconstructing, from said filtered signals, a plurality of physiological signals as would be acquired if said electrodes were placed on the cerebral cortex of said person,
the nonlinear dynamical systems (8) of said network (38) being respectively suitable for processing said filtered signals after having been processed by said spatial filter, and optionally transformed if said nonlinear transforms (37) are present. - Brain-computer interface system (1, 33) according to one of the preceding claims, characterized in that said nonlinear dynamical system (8), or at least one of the nonlinear dynamical systems (8) of said network (38) if said dynamical systems linear (8) are more than one, it includes at least three of said equations (9).
- Brain-computer interface system (1, 33) according to claim 10, characterized in that, in said nonlinear dynamical system (8) or in at least one of the nonlinear dynamical systems (8) of said network (38) comprising at least three of said equations (9) if said linear dynamical systems (8) are more than one, the value of said first coefficient (k, ki), the values of said second coefficients (a, b, c, ai, bi, ci) and, if present, the values of said third coefficients (gi,j), are such that said nonlinear dynamical system (8), or said at least one of the nonlinear dynamical systems (8) of said network (38) comprising at least three of said equations (9) if said linear dynamical systems (8) are more of one, has a chaotic behavior.
- Brain-computer interface system (1, 33) according to one of the preceding claims, characterized in that, in said nonlinear dynamical system (8) or in at least one of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (k, ki), the values of said second coefficients (a, b, c, ai, bi, ci) and, if present, the values of said third coefficients (gi,j) are such that in said nonlinear dynamical system (8), or in said at least one of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, emergent phenomena occur.
- Brain-computer interface system (1, 33) according to one of the preceding claims, characterized in that, in said nonlinear dynamical system (8) or in at least one of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (k, ki), the values of said second coefficients (a, b, c, ai, bi, ci) and, if present, the values of said third coefficients (gi, j) are such that said nonlinear dynamical system, or said at least one of the nonlinear dynamical systems of said network if said linear dynamical systems are more than one, has intrinsic dynamics.
- Brain-computer interface system (1, 33) according to one of the preceding claims, characterized in that it comprises:
・ a machine (3) for receiving said command from said classifier (14) and performing said physical action;
・ a supervision unit (16) for updating, in said nonlinear dynamical system (8) or in each of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (k, ki), the value of each of said second coefficients (a, b, c, ai, bi, ci) and, if present, the value of each of said third coefficients (gi,j). - Brain-computer interface system (1) according to claim 14, characterized in that it comprises feedback means (15) operable by a user (2) for communication, to said supervision unit (16), of a proper evaluation of the correspondence between said imaginary action and said physical action,
said supervision unit (16) being suitable for:
・ receiving as input:
- said filtered signal, and possibly transformed (20) if said nonlinear transform (7) is present, but not processed by said nonlinear dynamical system (8)
or, if said linear dynamical systems (8) are more than one,
- said filtered and possibly transformed signals if said nonlinear transforms (37) are present, but not processed by the nonlinear dynamical systems (8) of said network (38), nor evaluated by said second evaluation means (41), if present;
・ updating, in said nonlinear dynamical system (8) or in each of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (ki), the value of each of said second coefficients (a, b, c, ai, bi, ci) and, if present, the value of each of said third coefficients (gi,j) according to the evaluation communicated by said user (2). - Brain-computer interface system (33) according to claim 14, characterized in that it comprises:
・ further acquisition means (42) applicable to said person (2) for the acquisition of at least one further physiological signal not directly associated with the brain activity of said person (2);
・ monitoring means (43) of said machine (3) for detecting an execution of said physical action by said machine (3),
said supervision unit (16) being suitable for:
・ evaluating, from an examination of said further physiological signal, a correspondence between said imaginary action and said physical action, and
・ updating, in said nonlinear dynamical system (8) or in each of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (ki), the value of each of said second coefficients (a, b, c, ai, bi, ci) and, if present, the value of each of said third coefficients (gi,j) according to the evaluation carried out by said supervision unit (16). - Brain-computer interface system (1, 33) according to claim 15 or 16, characterized in that said supervision unit (16) is suitable for:
・ receiving as input also:
- said signal processed by said nonlinear dynamical system (8)
or
- said signals processed by the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one;
・ updating, in said nonlinear dynamical system (8) or in each of the nonlinear dynamical systems (8) of said network (38) if said linear dynamical systems (8) are more than one, the value of said first coefficient (ki), the value of each of said second coefficients (a, b, c, ai, bi, ci) and, if present, the value of each of said third coefficients (gi,j) also as a function of said signal processed by said nonlinear dynamical system (8) or, if said linear dynamical systems (8) are more than one, of said signals processed by nonlinear dynamical systems (8) of said network (38).
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2022504528A JP7390610B2 (en) | 2019-08-01 | 2019-08-01 | Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with human brain activity |
PCT/JP2019/030295 WO2021019776A1 (en) | 2019-08-01 | 2019-08-01 | "Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/JP2019/030295 WO2021019776A1 (en) | 2019-08-01 | 2019-08-01 | "Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021019776A1 true WO2021019776A1 (en) | 2021-02-04 |
Family
ID=67876050
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2019/030295 WO2021019776A1 (en) | 2019-08-01 | 2019-08-01 | "Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" |
Country Status (2)
Country | Link |
---|---|
JP (1) | JP7390610B2 (en) |
WO (1) | WO2021019776A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115005842A (en) * | 2022-08-09 | 2022-09-06 | 之江实验室 | Frequency-modulated brain-computer interface chip input impedance enhancing method and system |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6732064B1 (en) * | 1997-07-02 | 2004-05-04 | Nonlinear Solutions, Inc. | Detection and classification system for analyzing deterministic properties of data using correlation parameters |
US20070213786A1 (en) | 2005-12-19 | 2007-09-13 | Sackellares James C | Closed-loop state-dependent seizure prevention systems |
WO2009149126A2 (en) * | 2008-06-02 | 2009-12-10 | New York University | Method, system, and computer-accessible medium for classification of at least one ictal state |
US20100198098A1 (en) | 1997-01-06 | 2010-08-05 | Ivan Osorio | System for the prediction, rapid detection, warning, prevention, or control of changes in activity states in the brain of a subject |
US20100292752A1 (en) | 2009-02-17 | 2010-11-18 | Bardakjian Berj L | System and method for cognitive rhythm generation |
WO2011123059A1 (en) | 2010-03-31 | 2011-10-06 | Agency For Science, Technology And Research | Brain- computer interface system and method |
-
2019
- 2019-08-01 WO PCT/JP2019/030295 patent/WO2021019776A1/en active Application Filing
- 2019-08-01 JP JP2022504528A patent/JP7390610B2/en active Active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100198098A1 (en) | 1997-01-06 | 2010-08-05 | Ivan Osorio | System for the prediction, rapid detection, warning, prevention, or control of changes in activity states in the brain of a subject |
US6732064B1 (en) * | 1997-07-02 | 2004-05-04 | Nonlinear Solutions, Inc. | Detection and classification system for analyzing deterministic properties of data using correlation parameters |
US20070213786A1 (en) | 2005-12-19 | 2007-09-13 | Sackellares James C | Closed-loop state-dependent seizure prevention systems |
WO2009149126A2 (en) * | 2008-06-02 | 2009-12-10 | New York University | Method, system, and computer-accessible medium for classification of at least one ictal state |
US20100292752A1 (en) | 2009-02-17 | 2010-11-18 | Bardakjian Berj L | System and method for cognitive rhythm generation |
WO2011123059A1 (en) | 2010-03-31 | 2011-10-06 | Agency For Science, Technology And Research | Brain- computer interface system and method |
Non-Patent Citations (9)
Title |
---|
C.J. STAM: "Nonlinear dynamical analysis of EEG and MEG: review of an emerging field", CLIN NEUROPHYSIOL, vol. 116, 2005, pages 2266 - 301, XP005070618, doi:10.1016/j.clinph.2005.06.011 |
L. MINATIA. NIGRIC. ROSAZZAM.G. BRUZZONE: "Thoughts turned into high-level commands: Proof-of-concept study of a vision-guided robot arm driven by functional MRI (fMRI) signals", MED ENG PHYS, vol. 34, 2012, pages 650 - 8, XP028502870, doi:10.1016/j.medengphy.2012.02.004 |
L. MINATIM. FRASCAN. YOSHIMURAL. RICCIP. O?WIECIMKAY. KOIKEK. MASUH. ITO: "Current-Starved Cross-Coupled CMOS Inverter Rings as Versatile Generators of Chaotic and Neural-Like Dynamics Over Multiple Frequency Decades", IEEE ACCESS, vol. 7, 2019, pages 54638 - 54657, XP011723384, doi:10.1109/ACCESS.2019.2912903 |
L. MINATIM. FRASCAN. YOSHIMURAY. KOIKE: "Versatile locomotion control of a hexapod robot using a hierarchical network of nonlinear oscillator circuits", IEEE ACCESS, vol. 6, 2018, pages 8042 - 8065 |
L. MINATIN. YOSHIMURAM. FRASCAS. DRO?D?Y. KOIKE: "Warped phase coherence: An empirical synchronization measure combining phase and amplitude information", CHAOS, vol. 29, 2019, pages 021102 |
L. MINATIN. YOSHIMURAY. KOIKE: "Hybrid Control of a Vision-Guided Robot Arm by EOG, EMG, EEG Biosignals and Head Movement Acquired via a Consumer-Grade Wearable Device", IEEE ACCESS, vol. 4, 2016, pages 9528 - 9541 |
L.A. FARWELLE. DONCHIN: "Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials", ELECTROEN-CEPHALOGR CLIN NEUROPHYSIOL, vol. 70, 1988, pages 510 - 23 |
S. BOCCALETTI: "The synchronization of chaotic systems", PHYS. REP., vol. 366, 2002, pages 1 - 101 |
VAIBHAV GANDHI ET AL: "EEG denoising with a recurrent quantum neural network for a brain-computer interface", NEURAL NETWORKS (IJCNN), THE 2011 INTERNATIONAL JOINT CONFERENCE ON, IEEE, 31 July 2011 (2011-07-31), pages 1583 - 1590, XP031970873, ISBN: 978-1-4244-9635-8, DOI: 10.1109/IJCNN.2011.6033413 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN115005842A (en) * | 2022-08-09 | 2022-09-06 | 之江实验室 | Frequency-modulated brain-computer interface chip input impedance enhancing method and system |
CN115005842B (en) * | 2022-08-09 | 2022-11-15 | 之江实验室 | Frequency-modulated brain-computer interface chip input impedance enhancing method and system |
Also Published As
Publication number | Publication date |
---|---|
JP2022551370A (en) | 2022-12-09 |
JP7390610B2 (en) | 2023-12-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
KR101585150B1 (en) | multi-mode brain-computer interface system based on brain connectivity | |
Stephygraph et al. | Wireless mobile robot control through human machine interface using brain signals | |
Wu et al. | Controlling a human–computer interface system with a novel classification method that uses electrooculography signals | |
KR20080074099A (en) | Detection of and interaction using mental states | |
Murugappan et al. | Inferring of human emotional states using multichannel EEG | |
Xu et al. | Affect recognition using EEG signal | |
Lotte et al. | Brain–computer interface contributions to neuroergonomics | |
Hammer et al. | Predominance of movement speed over direction in neuronal population signals of motor cortex: intracranial EEG data and a simple explanatory model | |
Bagh et al. | Hilbert transform-based event-related patterns for motor imagery brain computer interface | |
Gjoreski et al. | An inter-domain study for arousal recognition from physiological signals | |
Bousseta et al. | EEG efficient classification of imagined hand movement using RBF kernel SVM | |
Padmavathi et al. | A review on EEG based brain computer interface systems | |
Ehrlich et al. | A neuro-based method for detecting context-dependent erroneous robot action | |
Francis et al. | Brain-computer interfacing for wheelchair control by detecting voluntary eye blinks | |
Aler et al. | Evolving spatial and frequency selection filters for brain-computer interfaces | |
Patnaik et al. | Deep RNN learning for EEG based functional brain state inference | |
WO2021019776A1 (en) | "Brain-computer interface system suitable for synchronizing one or more nonlinear dynamical systems with the brain activity of a person" | |
Chaudhry et al. | A prosthetic arm based on electroencephalography by signal acquisition and processing on MATLAB | |
Cabrera et al. | What makes a gesture a gesture? Neural signatures involved in gesture recognition | |
Shepelev et al. | A novel neural network approach to creating a brain–computer interface based on the EEG patterns of voluntary muscle movements | |
KR101630398B1 (en) | Method and apparatus for providing service security using subliminal stimulus | |
Ogino et al. | An EEG-based robot arm control to express human emotions | |
Arslan et al. | Subject-dependent and subject-independent classification of mental arithmetic and silent reading tasks | |
Turnip et al. | Utilization of EEG-SSVEP method and ANFIS classifier for controlling electronic wheelchair | |
Jain et al. | EEG signal classification for real-time neuro marketing applications |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
ENP | Entry into the national phase |
Ref document number: 2022504528 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 19765569 Country of ref document: EP Kind code of ref document: A1 |