JP2009531077A - Apparatus and method for real time control of effectors - Google Patents

Abstract

The present invention relates to an effector real-time control device, which derives an electrode for identifying an input signal from neuronal activity of a patient's brain, and calculates a control signal for the effector from the input signal. And a data processing apparatus with an evaluation unit. In this case, the evaluation unit is configured for training to determine the prediction model, and is configured to calculate the control signal from the input signal by taking the prediction model. At that time, neuronal activity arises from the natural neural information processing process of the brain.
[Selection] Figure 1

Description

The present invention relates to an apparatus and method for real time control of effectors.

Past medical techniques for patients with severe paralysis have been extremely inadequate. Motor nerve function prosthesis, which has been studied in recent years, is intended for paralyzed patients whose organic healing is no longer possible. In these patients, the cortex or motor cortex, an area important for voluntary movement control, is at least partially normal, but the nerve connection to the muscle is interrupted or the muscle / body is no longer Does not exist. The main cause of such paralysis is ischemic stroke or intracerebral hemorrhage (“stroke”).

The most serious cases are patients with so-called confinement syndrome, where complete skeletal muscle paralysis (eg, amyotrophic lateral sclerosis (ALS) / muscular atrophy) or voluntary motor loss due to stroke of the brainstem is there. Such patients are declared fully passive with full consciousness. Similarly, patients who have lost their limbs or patients who are inferior to the lower body can be considered. These paralysis and disability prevent patients from performing their intended movements. Here and below, movement refers not only to movements that mean walking, but also to movements of each muscle, such as movements of arms, imitation, or speech.

For this reason, attempts have been made to restore or improve athletic ability by consciously controlling the prosthesis using own brain signals. The core of such a neural prosthesis is the so-called brain machine interface (BMI). Its function is to translate the patient's neuronal activity when issuing commands to the prosthesis.

Conventional BMI systems are based on electrodes that are attached to the head and record brain waves (EEG). Patients are taught to intentionally strive to strengthen or weaken certain aspects of recorded brain activity.

Here, three different systems are known, each utilizing specific neuronal activity.

A) Slow Cortical Potentials (SCPs) (cortical slow potential)
SCPs are slow voltage changes across the cortex between 0.5 and 10 seconds. In doing so, negative SCPs are typically associated with other factors of motor or cortical activity, and conversely, positive SCPs are associated with attenuated cortical activity. Typically, during weeks and months of training, patients can learn conscious increases and decreases in SCPs. The “increase” or “decrease” information that can be identified from the outside by EEG can be used for simple cursor operation after the end of training. The disadvantage is that even with long-term rigorous training, there is very little information transmission. That is, only one byte is transmitted in a few seconds during which one SCP continues. In addition, this type of communication makes the patient tired.

B) P300
Auditory, visual or somatosensory stimuli, rare or particularly important potentials produced by normal stimuli, are typically about 300 to stimuli (typically in parietal cortex EEG). The peak of activity occurs with a delay of milliseconds. For example, showing some characters to the patient, and showing these characters sequentially in flash, P300 is strongest at the character desired by the patient. Making this procedure a little faster by showing the characters in columns or columns reveals them from the strongest column or line in P300. The disadvantages of this method are that you can only choose from the choices presented, and that many presentations are required with enough focus to choose just one letter.

C) Mu and β rhythms 8-12 Hz Mu activity and 13-30 Hz β activity occur strongly throughout the somatosensory and motor areas. Mu and β rhythms are usually attenuated by exercise and exercise preparation, while the rhythm is strongest after exercise and at rest. After weeks of training, subjects were able to control the strength of these rhythms, eventually showing that they allowed two-dimensional cursor control independent of the Mu and β rhythms. . In that respect, it is one step ahead of SCPs, but this is only a measure, and the drawbacks are not eliminated. In addition, since the movement affects the Mu rhythm, the accuracy of the control is reduced. Since these methods require sufficient concentration to intentionally generate the relevant brain activity, both in training and in actual use, the patient is quite fatigued. Therefore, patients need to consider that in addition to paralysis, they must also fight another major health problem.

There is nothing to do with healing when it comes to using another prosthesis to replace lost motor skills. Furthermore, each time it takes a relatively long time to detect brain activity, it can only transmit a small amount of information (typically 1-20 bytes per minute). Thereby, the information transmission width of BMI becomes extremely small. In this way, not only complex movements, but even simple movements cannot be controlled. Even in daily activities such as writing a simple letter, if you do not have considerable concentration every day, you will spend a lot of time.

In addition, it is well known that the prosthesis can be controlled by muscle activity that still allows optional control. One disadvantage of this is that the patient cannot express his intention in the usual way. For example, patients must get used to communicating by blinking. Second, the area of use of these methods is limited and probably will not help patients with complete confinement syndrome.

The object of the present invention is therefore to create the possibility of communication and motor skills, in which the effector with high information transmission rate is controlled by the patient's brain activity in a safe and simple way for the patient.

This problem is solved by the device according to claim 1 or the method according to claim 13 or 21. Other advantageous forms of the invention are defined in the subclaims.

In doing so, the method of the invention first uses the original or original signal transmitted for the function that the conventional BMI system uses any signal or intentionally trained signal to perform. It starts with the recognition that it has nothing to do with biological signals. Thus, the training often takes a long time, hindering or at least making it quite difficult to control complex and natural movements. Thus, the present invention starts from the principle of using a natural activity sample that the patient has already learned before losing his motor skills. It should be supplemented in order to understand the method of the invention that the “natural neural information processing process” must be understood to include the central nervous system information processing itself associated with movement. This is done even for healthy people and does not require training. In contrast, “artificial neuron processes” are limited and require special learning to control effectors. Or, even if it corresponds to a natural activity of the brain, that activity originally had nothing to do with the exercise that had to be learned before training.

The advantage of this invention is that it translates a natural motion signal into the corresponding prosthetic motion by accessing the desired motion from all the complex and diverse natural motions in a straightforward and easily trainable way It is a point that can be done. Training is related to direct movement control in a familiar way, so it is motivated, purposeful and does not feel immediately fatigued. Moreover, since the activity sample learned as a child can be used, it can be easily performed as a whole. Patients who use this method do not feel the transmission of their information artificially or have a sense of hard learning.

This facilitates the use and acceptance of the prosthesis, in that respect it can be said to be a considerably improved treatment in the sense of a true step towards healing. Overall, patients, apart from all the personal benefits of recreating their expressive ability, are improved in terms of economics, patient independence, social participation, workability, and less dependent on social security and care. Is done.

Advantageously, the data processing device comprises an input signal receiving unit and a control unit for supplying control signals to the effector. This auxiliary unit allows the data processing device to communicate with information input and output components.

Advantageously, the lead electrode, in particular the LFP (Local Field Potential) electrode, the ECoG (Electrocortical electroencephalogram) electrode or the EEG (Electroencephalography) electrode identifies the electromagnetic input signal from the neuronal population activity. It is configured to do so. This enhances stability against vibration and other changes during long-term derivation when access to the derived brain region is appropriate, and enables accurate motion prediction.

Preferentially, the lead-out electrode is non-invasive, in which case the working lead-out electrode is mounted outside the head surface, subcutaneously, under the calvaria on the brain tissue or without damaging the brain tissue. Is inserted inside. Invasive methods can reach individual neurons (SUA, single unit recording) or neuron groups (LFP), but damage brain tissue, often with unobservable results. In addition, the use of invasive electrodes is limited due to the risk of infection and surgery.

It is advantageous that the active derivation electrode receives motion-related input signals from the brain area responsible for motion (especially for information processing when imaging, assembling, executing or controlling motion in the motor cortex). It is to identify. Another input signal, such as a visual signal, can also fully control the effector. However, input signals that are particularly relevant to the movement of the brain region that acts to control the movement provide optimal input data for the movement control.

Preferentially, one cycle of one standard exercise becomes one training stage, and an input signal is recorded as training data in each training stage. Thus, neuronal activity is recorded because a single standard movement (demonstrated, imaged and executed in front of you) forms the basis of the movement data bank. This is not only used to directly improve the identification of prediction models, but for example when developing new prediction methods that were not yet known at the time of training or improving existing ones. Used for calibration.

More preferentially, the training phase is a demonstration of standard movement in front of the eyes, imaging and / or execution by the patient. These three stages are described in more detail later, but this allows for proper and purposeful training. In this case, the flexibility of the brain is used. This not only improves the prediction accuracy of the evaluation unit, but by fading back the actual movement of the prosthesis, the patient also improves the imaging of the intended movement. . As a result, the intended movement and the actual movement are quickly combined into one another.

Advantageously, the state of the patient at each training stage is classified and recorded, and the state, in particular the health state, concentration or fatigue state, is determined by each training data when determining the predictive model. Affect the adjustment. The inventor has been able to point out that important activities depend on such state parameters. If the prediction model takes them into account, the prediction model becomes more accurate, and the systematic error according to the patient's condition can be adjusted correctly.

Advantageously, the external environment at each training stage is classified and recorded, and its status, in particular the patient's status position (sleeping, sitting, standing), lighting or environment is Influencing the adjustment of each training data when determining the prediction model. Since these external environments affect related activities as well, the prediction model is more accurate when this is taken into account.

Preferentially, part of the training data remains unerased and erased in response to an abnormal signal. This abnormal signal is a standard motion that is specifically determined. If the patient sees an incorrect move that is different from what he intended, the patient needs to correct the appropriate training data. The patient can do it very easily if it can be corrected with a standard exercise that has been learned reliably. Another input process that is abnormal and disturbing is not necessary for the patient in subsequent training, and in extreme cases, the patient uses it for paralysis of the patient It will not be possible at all.

Advantageously, the predictive model uses standard motion as an interpolation point in setting new motion. It is almost impossible to train all possible movements. By using interpolation, the standard motion that must be trained can be minimized to cover the full range of possible motion.

Advantageously, the standard motion is converted into the form of frequency and phase modulation of the time-frequency-analysis band of the input signal. These parameters have been found to be useful for quick and reliable predictions.

Preferentially, the effector is a prosthesis, a patient's own body part, or a data processing device. Each of these effectors has its application area and advantages. Your body part is considered for use if it is still normal, and the computer allows complex behavior when there is little demand for information, such as activation of an emergency signal.

It is advantageous that the control signal has a target setting. This is converted into action or motion by the effector, including intermediate goal settings not included in the control signal. In doing so, the patient does not have to control all the intermediate parts that may make correct prediction difficult when making the intended movement. Simple physical laws often determine the exact movement, and the details of the movement can be determined unpredictably by electronics in the evaluation unit or effector.

Preferentially, an amplifying device is provided between the lead-out electrode and the receiving unit, which extracts and / or amplifies the input signal. The neuronal signal that the electrode derives often needs to be prepared before its evaluation can begin. Thereby, the accuracy of the prediction is increased.

The method according to the invention has other similar features and advantages, but such features and advantages described in the subclaims are examples and not final.

FIG. 1 is a structural diagram of a neuroprosthesis according to the invention. Details of each component are described in detail below. A lead electrode 1 for deriving neuronal activity in the human cerebral cortex 2 is inserted under the patient's skull or, in another embodiment, attached to the scalp or the head surface.

The lead-out electrode 1 measures neuronal activity and forwards it via the signal interface 3 as an electromagnetic input signal to an amplifying device 4 preferentially formed as a multi-channel amplifying device. This amplifying apparatus amplifies and extracts the electromagnetic input signal of the lead-out electrode 1 with high temporal resolution, and performs an evaluation chip, computer, or similar system that performs signal processing on the processed signal in real time. Forward to 5. Therefore, the signals are classified by the prediction model in real time, and an appropriate control signal is created. At the functional explanation level, the behavioral intention from the activity signal is now identified for a particular voluntary exercise that the patient can no longer perform.

The effector 6 is controlled by the control signal. What is featured here is a classic prosthesis that mechanically mimics a body part and also has its own body part. In addition, virtual commands are also conceivable, such as commands to a computer cursor or menu select. Conversely, the effector 6 can send an effector status signal to the system 5 to allow effector controlled fadeback.

There are various derivation methods in which the activity of neurons is electromagnetically measured by the derivation electrode 1. At that time, the lead-out electrode 1 is formed as a multi-electrode in actual use.

Usually, the voltage is recorded. At this time, the lead-out electrode 1 is inserted into the body or brain tissue, and the invasive method and the non-invasive method must be distinguished depending on whether the brain tissue is inevitably damaged. Invasive methods include single unit recording (SUA) in which the lead electrode 1 is attached near or penetrates an individual neuron, and the electric field potential that the lead electrode 1 is determined by the adjacent neuron. There is a local electric field potential (LFP) that measures from the periphery of the electrode. Invasive methods can reach deeper layers of the brain and can obtain spatially resolved detailed measurement data. However, this can give the patient a brain tissue disorder with unpredictable results. Furthermore, in order to always receive data from the same neuron, it is difficult for the lead-out electrode 1 to remain stably at that position.

Non-invasive methods include electroencephalography (EEG). This is a method of attaching the lead-out electrode 1 to the outside of the skull and can be used particularly easily, but only (spatial) low resolution data can be obtained. Although all the methods mentioned can be used for the present invention, in a preferred embodiment, the cortical potential is measured (by ECoG). In this case, the lead-out electrode 1 is a thin film electrode, as shown in FIG. 2, which is implanted sub-durally or epidurally over one or more selected areas of the cerebral cortex. Optionally, it is also conceivable to store the lead-out electrode 1 outside the skull. When used in a plurality of regions, an appropriate number of lead-out electrodes 1 are used in the form described here.

Inventor's experimental results show that such thin films may be advantageous for BMI use over thick electrodes that are typically used in clinical diagnostics. In a special embodiment, the lead-out electrode 1 is inserted into a brain groove (acupuncture). Thereby, it is possible to reach brain regions that are not on the external surface without damaging brain tissue. This arrangement is explained in detail in so-called related applications.

Next, the form of the film electrode as the preferential lead electrode 1 will be described in more detail with reference to FIG. The lead-out electrode 1 has a carrier 1a made of a flexible material or an elastic material. As the material, polyimide or silicon is used because it is compatible or biocompatible, easy to process, and hardly damaged. However, other materials are suitable as long as they have the necessary flexibility and biocompatibility, i.e., they do not adversely affect the brain even after long-term use. Furthermore, the material must naturally be non-conductive. Also, it must be easy to give individual shapes to the carrier, for example, it can be easily cut and shaped. Finally, the carrier must be sufficiently elastic and thin. The thinness is mainly 1 cm or less. The carrier edge is rounded to avoid damaging the tissue.

A series of electrodes 1c are attached to the carrier, and these electrodes are individually connected to the cable 1b, and a signal is transmitted to the outside through the cable. The carrier 1a is roughly shown as a rectangle. In use, it will often be advantageous to adapt this to the derived area.

The electrodes 1c are arranged in a matrix as contact points. The circuit 1e inside the carrier 1b is individually connected to each electrode 1c, and each circuit 1e is connected to the signal exchange cable 1b without overlapping. The expert has knowledge about the manufacturing method of such a circuit 1e and its arrangement method.

Electrodes can also be manufactured from a variety of materials. The main ones are gold, platinum, metal alloys or conductive plastics and semiconductors. The carrier 1a can be employed from a size smaller than 1 cm to a size larger than 10 cm. The electrode contacts are attached at a typical density, from one electrode contact per square centimeter to over 10,000 electrode contacts. Signal resolution is improved with a high-density electrode contact, but naturally, not only the manufacturing cost of the electrode 1c but also the amplification time and control calculation time increase. It should be noted here that the energy demand is high when a large number of electrode paths are wirelessly transmitted.

Of course, if necessary, the arrangement may be different from the arrangement shown in FIG. Here, the points on the surface can be arranged in any way.

It is advantageous that this carrier does not damage tissue as opposed to invasive electrodes. Thereby, the signal is detected stably for a longer period of time. This is because electrodes that invade brain tissue cause local tissue destruction, leading to the disappearance of local neuronal activity. Moreover, the carrier 1a with the electrode 1c can also be made very small (1 cm or less) according to application. In this case, the operation of inserting the carrier 1a into the patient can be performed at a very low cost, and the load on the patient is extremely small.

In the operation in which the lead-out electrode 1 is inserted, a special preoperative diagnosis and an operation plan are required. The most important point is to determine the exact target area of the transplant. This cannot be determined a priori because the human brain is individually highly neuroanatomical. The only exception would be if you would like to insert without deciding the location individually. Since general brain mapping is known, it is known where a particular functional area is. Furthermore, in the special mapping example of motor and somatosensory areas, the human body is replicated locally and individual body parts are assigned to separate cortical areas. However, this level of prior knowledge is still insufficient for individual patients.

Therefore, prior to surgery, functional magnetic resonance imaging (fMRI) is used to accurately locate the target area for each individual patient. Here, while the patient controls, images, and observes the effector, the brain-specific activation is measured, and the location of the transplant can be determined with high positional accuracy. it can. To improve localization, subsequent electroencephalogram measurements with source reconstruction are performed during a similar motor paradigm (effector control trial, image or observation).

FIG. 3 is a preferred embodiment of the signal interface 3. As a method, it is also possible to selectively carry out data transfer via a cable that has been used as standard in neurosurgical diagnosis. However, connecting a cable to the body surface for a long time increases the risk of infection and is unattractive in terms of appearance and practicality. In the embodiment described here, signal transmission between the electrode and the amplification device takes place by inductive energy transmission without a percutaneous cable connection.

The wireless signal transmission system 3 is divided into an upper part and a lower part of the skin surface 3a. The external transmission / reception unit outside the body (shown here at the top of the skin surface 3a) is representatively shown with only the coil 3b. The external transmission / reception unit can transmit only data to the amplifying device 4 or the data processing system 5 by wireless or direct cable connection. Another embodiment is also conceivable in which the amplification device 4 and / or the data processing system 5 are partly or completely included in a chip that is attached to the skull surface or other suitable body part. In each case, which embodiment is preferred or feasible depends on the complexity of use. At present, small transmission / reception units can transmit / receive to / from the external amplifying device 4 or the data processing system 5 from almost any distance (mobile radio, Bluetooth, WLAN) without any technical problems. It is.

The mentioned transmission method can also be used for data exchange with the effector 6. When controlling a paralyzed natural body part, another two-part signal transmission interface similar to that described here can be inserted into each body part. Since the external transmission / reception unit can be easily accessed, it can be adapted to advanced technology and can be exchanged without performing a new operation.

A multifunction chip 3c, which is an internal transmission / reception unit, is inserted below the skin surface 3a, corresponding to the external transmission / reception unit. The multi-function chip 3c includes a receiving unit 3c1, a transmitting unit 3c2, and an optional battery unit 3c3. The signal is supplied from the electrode 1c of the carrier 1a to the transmission unit 3c2 or the reception unit 3c1 via the cable 1b.

In operation, the coil 3b of the external transmitting / receiving unit inductively transmits the control signal for the energy and possibly the lead-out electrode 1 to the receiving unit 3c1 by means of a high-frequency signal. Such a control signal may be an on-off switch command, a check on the energy state of the battery 3c3, or the like. The interface is basically also suitable for transmitting stimulation signals to the lead-out electrode 1.

The multi-function chip 3c identifies the modulated control or the mentioned stimulus signal in a manner well known in the communication technology. The energy for the necessary arithmetic operations by the control unit of the multi-function chip 3c is received from the high frequency signal. Alternatively, the accumulator can be inductively charged by the battery 3c3 or a high frequency signal. Therefore, the energy supply is temporally decoupled from the interface transmission. In this case, of course, the charging signal and the stimulation signal must be separated by separation of time frame or frequency band.

Conversely, the signal of the measurement electrode 1c is transmitted to the transmission unit 3c2 via the cable 1b, where it is preferentially designed for the coil 3b or reception in the signal band of 402 to 405 MHz of MICS (Medical Implantable Service Band). In addition, it is transmitted to the analog of the coil 3b displayed as a representative.

So far, the transmission interface has been described so that the transmission performance of the transmission unit 3c2 reaches only the coil 3b of the external transmission / reception unit. Optionally, the transmission unit 3c2 can also transmit directly to the amplification device 4 that is not attached to the skull surface, if possible. In this case, the energy supply of the multi-function chip 3c is ensured by a long-life battery (current technology is not sufficient), by induction by the method described above, or by charging by using the body's own energy source. It can be carried out.

Thereafter, the amplifying device 4 receives the input signal of the output electrode 1 outside the skull. The amplifying device 4 processes the input signal by a known method. At this time, in addition to amplification, a high-pass filter, a low-pass filter, or a band-pass filter is used (for example, Savitzky-Golay filter, Butterworth filter, Chebychev filter). For real-time transmission, a high temporal resolution is advantageous and an extraction rate of 200 Hz or higher is ideal, but lower values are not excluded. The input signal thus processed is then transferred to the system 5 for evaluation.

The function of the system 5 is to calculate a control signal for the effector 6 from the input signal. In that case, a distinction can be made between two phases: a training phase and a usage phase. In this case, use is always interrupted by new training to improve or to learn further control. The training serves to determine the predictive model so that the system 5 then detects the patient's motion intention during use and calculates the appropriate control signal.

The purpose of the training is to input the data required for standard exercise classification into the system 5 in order to determine the prediction model. To that end, a single standard exercise is selected at each training stage, and then the patient either demonstrates the standard exercise in one of three scenarios, images it, or controls the effector 6. Try. Undoubtedly, this evokes an input signal similar to that of natural, self-implemented control. Of course, if you have a paralysis that still has (residual) movement, you can also use the actual movement. This can be considered as a preparation especially when the cause of paralysis does not occur suddenly, such as slowly progressing illnesses before amputation or muscle atrophy.

The training paradigm has an important aspect of invention. The patient does not have to mobilize high concentration to evoke the state of neuronal activity indicated by the evaluation process, which is essentially natural, as in the traditional method described at the beginning. Under certain conditions, it doesn't work to control the function you want to control. Instead, patients use exercise samples that have already been learned in their lives, quite naturally. Thereby, subjectively, during training and in use, the patient is doing nothing other than calling on healthy muscles as before the disease.

During individual training phases that are repeated many times, the system 5 can memorize incoming brain signals and later assign them to specific movements.

This will be explained in more detail using an arm / hand prosthesis. The standard exercise to be trained is determined by the patient, either through selection from a computer menu or communication with an assistant. For example, in the standard exercise example using the mentioned arm / hand prosthesis, the hand is opened-the hand is closed, the arm is moved in various directions, returned, or the force is changed to close the hand, There is a possibility.

Here, three different training data sets are created. First, the natural standard exercise selected by the subject is shown again. From the derived brain signal, a data record for the first training data set of the matrix is created about 1 second before and after the end of the standard exercise. As the remaining selected standard exercises that must be learned are demonstrated, other data sets are created that are recorded together to form the first complete training data set.

Thereafter, the computer program repeatedly expresses to the subject that the standard motion is imaged in turn, during which the derived brain signals create a second training data set. In the third stage, the computer program tells the subject to perform a standard exercise. The brain signals derived during that time create a third training data set.

Training is optimized by displaying accuracy with feedback. Using that accuracy, the predictive model can correctly assign brain signals derived during each movement to the displayed standard movement. In that way, the subject directly knows how well he / she learns the movement and the subject's brain can adapt to the effector 6. In other words, training not only improves the system 5, but also improves the patient due to neuronal flexibility. Importantly, this is not based on difficult behavioral samples that are artificially created. The patient's subjective sensation feels a much more natural process, such as swimming or biking, which is usually the same as acquiring new motor skills. Due to the efficiency of feedback learning, in other procedures, the data of the third training set is even more important than the other two training sets.

The third stage is repeated as the final stage of the training, but this time with different subjective conditions (eg ready and unprepared, awake and sleepy) and objective conditions ( For example, lighting, posture: sitting, standing, sleeping). This expands the third training set, which further improves the accuracy of training results and predictions and also allows for the reliable assignment of brain signals in various environments. Subjective conditions are also called states, and objective conditions are also called environments.

FIG. 4 shows the measurement of the activity of a specific neuron for one similar movement and shows it over time when the experimental animal is in three different states. As can be seen from the three graphs, the neuroactivity or signal samples are clearly different, especially during exercise execution. Thus, the measured signal samples are classified with respect to the movements and conditions that must be trained and stored, for example, in a data bank.

In use, the patient's intended neuronal activity and stored signal samples are compared to sense the imaged motion that the patient intends to perform on the effector. Here, external and internal conditions are also detected if possible.

When using the Bayesian method, which is a general probability theory, the model of motion prediction B from neuronal activity N is
P (B | N) = P (B) * P (N) ⁻¹ * P (N | B)
Considering state Z for optimal motion prediction,
P (B | N, Z) = P (B) * P (Z | B) * P (Z- ¹ ) * P (N | Z) ^-1 * P (N | B, Z).

It is advantageous to train the system 5 under various conditions and use signal samples for conditions and imaged motion as shown in FIG. Here, in the animals "Subject 1" and "Subject 2", the intended motion was correctly decoded for the various trained prediction models, regardless of the number of neurons (signal samples) used. Probabilities are shown. The black bars are the decoding probabilities when used in known or trained conditions (states). The dark gray bar corresponds to the case where the state is unknown, where all three trained states are combined, almost like "one trained big condition". The light gray bar indicates a case that is a different condition from the training.

Obviously, the best results are consistently known when the condition is known and there is training data belonging to it (ie, there is exercise training data). However, when many neurons (signal samples) are measured at the same time, the dark gray bars show almost as good numbers. That is, by combining training data in three different states, it is possible to correct the loss of specificity of the training data that can be used.

When only the training data obtained in another state can be used (light gray bars), it is consistently clearly bad. For example, in the case of the target 1, the accuracy is less than 70% even with 100 neurons. On the other hand, 97% or 96% when there is training data in the same state or when all state training data is combined.

FIG. 10 summarizes the mechanism of action of the system 5 when used in the “normal mode”. Once the condition or condition X is known, brain neuronal activity is measured. If there is training data in the system for condition X or state X, these data are used to calculate and transmit motion prediction or motion control signals to effector 6. On the other hand, when there is no training data regarding the condition X in the system, the training data of a similar condition is used for calculating the motion prediction. If condition X is unknown because condition X is not detected, all training data in the system is used to calculate motion predictions.

As described above, the more states and training data assigned to them, the more stored in the system 5, the better the motion prediction.

In addition, the patient can always start a new training program for a new standard exercise on his / her own or run a previously learned program in a new situation. Eventually, the patient can be trained with a “normal / unnormal signal” that allows training to be restarted. Thereby, the patient can operate to tell the system 5 that the prediction accuracy is not sufficient and a new training is required. This restart can of course be ordered from a computer menu, etc., regardless of the standard movement. However, if the effector 6 is the patient's only means of communication, he should have a reliable way to communicate to the system 5 with his own learned “normal” standard motion. Another abnormal signal is a natural / natural brain error signal that occurs if the assignment is incorrect.

FIG. 6 shows an example of a conversion diagram when converting from an input signal to an effector control signal using training data. On the left side, an example of voltage progress of the three electrodes 1c is shown. These voltage signals are first amplified and extracted as an input signal by the amplification device 4. The extraction function can also be transferred to the system 5. As an example of an extraction method (details are described in the context of the amplification device 4), the pressure signal is extracted in bandpass, then averaged over a small time frame and divided into short time frames. The activity is then evaluated in a mathematical manner. The predictive model is determined on the one hand by selecting a mathematical method and on the other hand by calibration with training data. Thereby, the prediction of the intended movement is possible via the system 5 (“Intention Prediction”).

Typical mathematical methods include (1) signal pre-processing, or a) filtering (eg, low pass or band pass), b) time and frequency analysis (eg, Fourier transform or multi-tapering) and / or Or c) segmentation and averaging over time frames; (2) decoding or discriminant analysis of preprocessed signals (linear, square or regularization) or support vector machines (linear or radial basis functions). The methods listed here are not all. In particular, in addition to so-called discriminant analysis and support vector machines, for example, linear filters and Kalman filters can be used for continuous motion decoding.

The effector control signal shown on the right is the result. Here, as an example, two effector devices (two motors) and outputs applied to them according to the rotational speed are shown here.

It should be emphasized that the predictive model is not limited to detecting a limited number of standard motions. Other movements can be detected by interpolating and extrapolating recorded neuronal correlations trained with standard movements.

The effector 6 converts the effector control signal sent from the system 5 into motion. It should be reiterated that movement must be understood in a comprehensive sense. Exercise includes control and imitation of utterances.

The effector 6 can be considered in the three groups described above. That is, a mechanical device such as a robot, a robot arm, or a prosthesis, an own body part, or an electrical device controlled by a computer virtual command, such as a computer, a mobile wireless device, or a home appliance.

The first case of a prosthesis artificially mimicking a body part is described in more detail below using the hand prosthesis shown in FIG. Of course, any type of prosthesis can be controlled.

An effector control signal is transmitted from the system 5 to the effector 6 via the effector input cable 6a1.

The prosthesis has a rotation system 6b1 for turning the hand. The motor control of the rotation system 6b1 rotates the prosthesis according to the effector control signal. In addition, the prosthesis has a motor and controlled capture system 6b2, which opens and closes fingers according to effector control signals. It must be mentioned that not all control from neuron data is communicated in detail. Instead, if the system 5 can only predict the intended movement, it will of course be possible to determine the individual steps required.

In order to perform feedback, a pressure sensor 6c is attached to the finger portion. The effector status signal measured by the sensor is fed back to the system 5 via the effector output cable 6a2. These data can be fed back to the brain as stimulation data in the extension of the present invention. However, the system 5 should also be able to check the actual state of the prosthesis. This is because the internal display of the system 5 does not necessarily match. This is not due to the inaccuracy of the system 5, but may be caused by a failure acting from the outside, such as a prosthesis collision. By checking the actual condition, the system again recognizes the initial position of the correct motion reference.

Finally, the prosthesis is fitted with a cover similar to the appearance of a human hand. Here, the hand prosthesis is not limited to opening and closing, and it is also possible to perform complex movements by means of the prosthesis that has advanced technically in the present invention.

In the second group of effectors 6 particularly related to medical treatment, when only the neuron connection between the brain and the body part is interrupted, the body part is controlled as the effector 6 by functional electrical stimulation. In this case, either normal nerve cells of the body part are stimulated or muscle fibers are stimulated directly. Similarly, possible feedback should be done through a still-normal body baroreceptor, stretch receptor, or other selector, or using an auxiliary sensor as described in the prosthesis control case above. Can do. Similarly, in the case of partial palsy where (slightly) athletic ability is still left, it is conceivable to assist these remaining movements with motorized mechanical devices.

The third group of “virtual” effectors 6 is particularly large. This allows you to control computer cursors or menu selections, switch lights on, send emergency calls, etc.

Of particular interest is the control of virtual prostheses. In this case, the body part is three-dimensionally displayed on the screen and controlled by the neuronal activity of the patient or subject. This greatly facilitates proper prosthesis adjustment and selection.

If such control data and stimulus data are used in a form that is capable of computer processing, the spatial distance is no longer taken into account by the Internet or similar networks. Thus, the controlling effector need not be in the immediate vicinity of the individual controlling it and need not be directly connected in space. In this way, it will be possible to virtually display and control a prosthesis or robot in another location. Medical applications include surgeon telesurgery, and in the military it is possible to accurately control a robot without sending a person to a dangerous location. Alternatively, it can be used in places where humans cannot enter, such as contaminated areas, nuclear facilities, the deep sea, and space.

FIG. 8 summarizes the method according to the invention for controlling the motion of the effector 6 incorporated in the living body. This method includes the following steps.
Step 810, receiving an input signal representative of neuronal activity in a living brain.
Step 850 obtains a motion control signal for the effector 6 based on the input signal.
In doing so, signal samples are detected from the input signal (step 820). Then, a motion control signal is calculated from the signal sample based on the prediction model. In steps 830 and 840, the detected signal samples are compared with the stored training signal samples according to a predetermined prediction model. This training signal sample is representative of the neuronal activity created by the living brain with respect to the trained standard motion, and the stepped motion of the corresponding effector 6 is assigned to each standard motion ( Stage 830).
This stepped motion is then used to obtain a motion control signal assigned to the training signal sample most similar to the signal sample detected according to the comparison process (step 840).

FIG. 9 shows a method for constructing a data bank based on the invention having prediction model data for use in motion control of an effector 6 incorporated in a living body. This method includes the following steps.
Step 910, receiving an input signal that is representative of neuronal activity created by the living brain with respect to the normal movement of the living body that must be trained.
Step 920, storing an input signal in a data bank as a training signal sample.
Step 930 assigns stepwise motion control signals to the training signal samples according to the respective standard motion.

The present invention will now be described in detail with reference to the accompanying figures for further features and advantages. The figure is as follows.
Summary of invention Plan view of carrier surface with multiple electrodes attached according to preferred form of lead-out electrode Schematic diagram of signaling interface for an embodiment of the invention Neuronal signals for movement in three different subject states Decoding probabilities for three different variations taking into account the subject's condition in the prediction model Example of conversion diagram of neuron signal / data in control signal for effector using prediction model Figure showing a hand prosthesis as an example of an effector that can be controlled by one form of the invention Flowchart for using the method according to the invention Flowchart of the inventive method for building a data bank Summary of systems and methods according to the invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lead electrode 1a Carrier 1b Cable 1c (Individual) electrode 1e Circuit 2 Cerebral / cortex 3 Signal interface 3a Skin surface 3b Coil 3c of external transmission / reception unit Multi-function chip 3c1 Reception unit 3c2 Transmission unit 3c3 Battery / accumulator 4 Amplifier 5 Evaluation And central control system 6 Effector 6a1 Effector input cable 6a2 Effector output cable 6b1 Rotation system 6b2 Capture system 6c Pressure sensor 6d Cover

Claims (35)

It has a lead-out electrode (1) for receiving an input signal that is representative of the neuronal activity of the living brain (2), and acquires a motion control signal for the effector (6) based on the input signal A motion control device for an effector (6) incorporated in a living body having a data processing device (5) with an evaluation unit, wherein the evaluation unit detects a signal sample from an input signal, and the detected signal sample The motion control signal is calculated for the effector (6) from and the detected signal sample is compared with the stored training signal sample according to a predetermined prediction model. The training signal sample is representative of the neuronal activity generated by the living brain with respect to the trained standard movement. , Each stepped motion of the corresponding effector (6) is assigned to their standard motion, where the stepped motion is used to obtain the motion control signal assigned to the training signal samples detected in the comparison process. A motion control device for an effector (6) incorporated in a living body. 2. The device according to claim 1, wherein the data processing device comprises an input signal receiving unit and a control unit for supplying control signals to the effector (6). An information processing process in which the deriving unit has at least one deriving electrode (1), and this deriving electrode starts from the brain region in charge of movement (particularly in the case of imaging, assembling, executing or controlling movement in the motor area) The apparatus according to claim 1 or 2, wherein the apparatus is configured to identify an input signal relating to movement. A lead electrode (1), in particular a local field potential electrode, a cortical electroencephalography electrode or an electroencephalography electrode, is configured to identify an electromagnetic input signal from neuronal population activity; The apparatus of claim 3. Biological conditions, especially body position, health, concentration or fatigue, are classified and stored during each training phase, which affects the weighting of each training data when determining the predictive model Item 5. The apparatus according to any one of Items 1 to 4. 6. The evaluation unit is formed such that the external environment is classified and stored during each training phase, and its state affects the weighting of each training data when determining the predictive model. The apparatus as described in any one of. The evaluation unit is configured so that the detected signal samples remain unaccounted for in the calculation of the motion control signal, where the abnormal signal is a natural error signal of the brain. The apparatus according to claim 1. The evaluation unit is configured so that the detected signal samples remain unaccounted for in the calculation of the motion control signal, where the abnormal signal is a natural error signal of the brain. The device according to claim 1. 9. Apparatus according to any one of the preceding claims, wherein the predictive model uses standard motion as an interpolation point when assembling a new motion control signal. 10. Apparatus according to any one of the preceding claims, wherein the standard motion is converted into the form of frequency modulation and phase modulation of the time-frequency-analysis band of the input signal. The device according to any one of the preceding claims, wherein the effector (6) is a prosthesis, a patient's own body part or a data processing device. 12. The motor control signal according to claim 1, wherein the motor control signal has a target setting, and the motor control signal can be converted into an action or a motion by the effector (6), including intermediate target settings not included in the control signal. The device described in 1. Device according to any one of the preceding claims, wherein an amplifying device (4) is provided between the lead-out electrode (1) and the receiving unit (5) and extracts and / or amplifies the input signal. Incorporated into the body,
Receives input signals that are representative of neuronal activity in the living brain,
Obtaining a motion control signal for the effector (6) based on the input signal;
In this case, a signal sample is detected from the input signal, and a motion control signal is calculated from the signal sample based on the prediction model, and is detected according to the predetermined prediction model. A comparison is made between the signal sample and the stored training signal sample, which is representative of the neuronal activity generated by the living brain with respect to the trained standard motion, and each corresponding effector ( The stepped motion of 6) is assigned to those standard motions, where the stepped motion is used to obtain the motion control signal assigned to the training signal sample most similar to the signal sample detected according to the comparison process. A motion control device for the effector (6). The method of claim 14, wherein the method is performed in real time. For the calculation, one state of the living body is detected. In this case, a predetermined state of the living body is detected based on the prediction model, and these states are associated with the neuron activity and the training signal. The method according to claim 14 or 15. The method according to any one of claims 14 to 16, wherein based on the predictive model, the external environment of the living body is detected, and their state is correlated with neuronal activity and training signals. The input signal is related to movement, and represents neuronal activity in the brain region that controls the movement. In this case, neuronal activity is the case where the movement is imaged, assembled, executed, or controlled in the motor area of the brain. The method according to claim 14, which corresponds to an information processing process. 19. The detected signal sample remains in an unacceptable state when receiving a non-normal signal when calculating a motion control signal, wherein the non-normal signal is a particularly determined standard motion. The method described in one. 20. The detected signal sample remains in an unacceptable state when receiving a non-normal signal when calculating a motion control signal, wherein the non-normal signal is a natural error signal of the brain. The method described in one. 15. A method according to any one of claims 20 to 14, wherein the predictive model uses standard motion as an interpolation point when assembling motion control signals. A data bank construction method based on prediction model data for use in motion control of an effector (6) incorporated in a living body,
Receives input signals that are representative of neuronal activity generated by the body's brain with respect to the standard body movements that must be trained,
Save the input signal in the data bank as a training signal sample,
Assign stepwise motion control signals to training signal samples according to their standard motion
A method for constructing a data bank having the steps of: 23. The method according to claim 22, wherein in training, a cycle of a standard body movement forms a training phase, and during each training phase, the input signal is recorded as a training signal sample. 24. A method according to claim 22 or 23, wherein one training stage is a demonstration, image and / or execution of a standard exercise. The state of the organism is stored in the classification and data bank during each training phase, which affects the weighting of each training signal sample when determining the predictive model, where the state is in particular 25. A method according to any one of claims 22 to 24, wherein the method is body posture, health status, concentration and / or fatigue. 26. The method of any one of claims 22-25, wherein a standard exercise training series is performed in various situations. 27. A method according to any one of claims 22 to 26, wherein the detected signal samples are discarded in response to a received abnormal signal without being stored. 28. A method according to any one of claims 22 to 27, wherein the stored training signal samples are erased from the data bank upon receipt of the received abnormal signal. 29. A method according to claim 27 or 28, wherein the abnormal signal is a predetermined standard movement or is representative of neuronal activity of a natural error signal. 30. The input signal is an electromagnetic input signal of collective activity of neurons, in particular an input signal derived from a local field potential, a cortical electroencephalogram or an electroencephalography derivation. The method as described in any one. 31. A method according to any one of claims 14 to 30, wherein the predictive model uses standard motion as an interpolation point when assembling a new motion control signal. 31. A method according to any one of claims 14 to 30, wherein neuronal activity corresponding to the standard motion is converted into a form of frequency modulation and phase modulation of the time-frequency-analysis band of the input signal. The method according to any one of claims 14 to 32, wherein the effector (6) to be controlled is a prosthesis, a body part of the living body itself, or a data processing device. 34. Any one of claims 14 to 33, wherein the motion control signal has a goal setting, and the motion control signal is converted into action or motion by the effector (6), including intermediate goal settings not included in the control signal. The method described in 1. The method according to any one of claims 14 to 34, wherein the prediction model has learning ability.

Images