Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
  • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
  • G09B19/00—Teaching not covered by other main groups of this subclass
  • G09B19/003—Repetitive work cycles; Sequence of movements
  • G09B19/0038—Sports
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
  • A61B5/25—Bioelectric electrodes therefor
  • A61B5/251—Means for maintaining electrode contact with the body
  • A61B5/256—Wearable electrodes, e.g. having straps or bands
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
  • A61B5/316—Modalities, i.e. specific diagnostic methods
  • A61B5/369—Electroencephalography [EEG]
  • A61B5/372—Analysis of electroencephalograms
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
  • A61B5/316—Modalities, i.e. specific diagnostic methods
  • A61B5/369—Electroencephalography [EEG]
  • A61B5/375—Electroencephalography [EEG] using biofeedback
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
  • A61B5/316—Modalities, i.e. specific diagnostic methods
  • A61B5/369—Electroencephalography [EEG]
  • A61B5/377—Electroencephalography [EEG] using evoked responses
  • A61B5/378—Visual stimuli
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
  • A61B5/316—Modalities, i.e. specific diagnostic methods
  • A61B5/369—Electroencephalography [EEG]
  • A61B5/384—Recording apparatus or displays specially adapted therefor
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/74—Details of notification to user or communication with user or patient; User input means
  • A61B5/7405—Details of notification to user or communication with user or patient; User input means using sound
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/74—Details of notification to user or communication with user or patient; User input means
  • A61B5/742—Details of notification to user or communication with user or patient; User input means using visual displays
  • A61B5/744—Displaying an avatar, e.g. an animated cartoon character
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
  • G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
  • G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
  • G09B23/285—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine for injections, endoscopy, bronchoscopy, sigmoidscopy, insertion of contraceptive devices or enemas
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
  • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
  • A61B2503/10—Athletes
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
  • A61B2503/20—Workers
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
  • A61B5/024—Measuring pulse rate or heart rate
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/48—Other medical applications
  • A61B5/4806—Sleep evaluation
  • A61B5/4809—Sleep detection, i.e. determining whether a subject is asleep or not
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/74—Details of notification to user or communication with user or patient; User input means
  • A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
• • A—HUMAN NECESSITIES
  • A63—SPORTS; GAMES; AMUSEMENTS
  • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
  • A63B2214/00—Training methods

Definitions

• the present invention generally relates to systems and methods directed to neuroenhancement of trainees and, more particularly, to neurofeedback training in order to improve a subject's performance.
• Brain stimulation has been used to train the brain to enhance physical motion activities of the subject. It has been shown that, by applying electric current (externally) over the specific brain regions, physical performance increases. This has been shown, for example, with athletes. During the usage, brain stimulation is quite similar to the result of brain training that is achieved organically with training. The main disadvantage of brain stimulators is the time the effect they carry last. With the device removal, it is reported that the training's impact remains only for ⁇ 20 minutes.
• US 2019/0247662 discloses a method of facilitating a skill learning process or improving performance of a task, comprising: determining a brainwave pattern reflecting neuronal activity of a skilled subject while engaged in a respective skill or task; processing the determined brainwave patern with at least one automated processor; and subjecting a subject training in the respective skill or task to brain entrainment by a stimulus selected from the group consisting of one or more of a sensory excitation, a peripheral excitation, a transcranial excitation, and a deep brain stimulation, dependent on the processed temporal patern extracted from brainwaves reflecting neuronal activity of the skilled subject.
• Enhancing performance with direct intervention over brain activity is done commercially in two main ways.
• the first is brain stimulation.
• brain stimulation is quite similar to the result of brain training that is achieved organically with training.
• the main disadvantage of stimulators is the time the effect they carry last. With the device removal, it is reported that the training's impact remains only for ⁇ 20 minutes. It means that in most competitive sports, the advantage of the intervention fades before it is needed. This process can be also considered as a form of doping, which is problematic for competitive sports.
• Nasa reported that, during 2004 in the United States, pilot error was listed as the primary cause of 78.6% of fatal general aviation accidents, and as the primary cause of 75.5% of general aviation accidents overall.
• Pilot errors may be classified as:
• pilot error typically accounts for just over half of worldwide accidents with a known cause.
• the recent hiatus in air travel has caused an increase in pilot errors (https://www.latimes.com/business/storv/2021-01-29/airline-pilots-flight-errors-pandemic) and only serves to highlight the need for additional and effective systems of training.
• the system and method of the present invention is easily adaptable to improve pilot's performance and assessment. “There is growing interest for implementing tools to monitor cognitive performance in naturalistic work and everyday life settings.
• fNIRS functional near infrared spectroscopy
• the aforesaid system comprises: (a) an electroencephalographic sensor arrangement attachable to a head of said trainee; (b) a processor configured for receiving and analyzing electroencephalographic signals obtained from said trainee in response to said visual stimulus displayed to said trainee; (c) a memory' storing instructions when executed by said processor perform: (i) instructing said trainee to imagine executing said sports, surgical or aviation motion action; (ii) measuring electroencephalographic signals on said electroencephalographic sensor arrangement; (iii) calculating at least one characteristic selected from the following: (1) a concentration index; (2) a motor control index; (3) an alertness index; (iv) providing said trainee with a feedback pattern based on at least one said concentration, motor control, alertness and motion readiness; (v) recurring steps c to e if needed.
• a further object of the invention is to disclose the trainee instructed to imagine executing said motion activity in response to displaying said visual stimulus.
• a further object of the invention is to disclose the instructions of displaying said visual stimulus, measuring electroencephalographic signals on said electroencephalographic sensor arrangement and providing said feedback pattern performed in a consecutive manner.
• a further object of the invention is to disclose the memory comprising an instruction of calculating said concentration index as a ratio of change of electroencephalographic signals at parietal-zone and frontal-zone electrodes at alfa-, beta- and theta- frequencies obtained from said electroencephalographic signals at parietal-zone and frontal-zone electrodes measured at rest.
• a further object of the invention is to disclose the memory comprising an instruction of calculating said motor control index as a ratio of change of electroencephalographic signals at sensorimotor zone electrodes, at Mu-frequency obtained from said trainee in response to said visual stimulus below said electroencephalographic signals at sensorimotor zone electrodes measured at rest.
• a further object of the invention is to disclose the memory comprising an instruction of for calculating said alertness index as a ratio of change of electroencephalographic signals at parietal-zone electrode at alfa-frequency obtained from said trainee with open eyes over said electroencephalographic signals at parietal-zone with closed eyes.
• a further object of the invention is to disclose the memory comprising an instruction of analyzing at least one of said concentration index, motor control index and alertness index of said trainee or a group of said trainees and presenting training progress data in a chronological manner.
• a further object of the invention is to disclose the feedback pattern selected from the group consisting of a static avatar, a dynamic avatar, a text message, a sound pattern, a tactile pattern and any combination thereof.
• a further object of the invention is to disclose the feedback pattern relating to a visual environment related to said motion action.
• a further object of the invention is to disclose the visual environment selected from the group consisting of a soccer stadium, a baseball stadium, a basketball hall, a rugby stadium, an athletic stadium and any combination thereof.
• a further object of the invention is to disclose the memory' comprising an instruction of calculating an integral index of sports readiness as a compound of at least two indexes selected from the group consisting of said concentration index, motor control index and alertness index and normalized by a sum thereof.
• a further object of the invention is to disclose method of testing and training a brain capability of a trainee to plan and executing motion activity.
• the aforesaid method comprises steps of: (a) providing said system according to claim 1 for testing and training a brain capability of planning and executing sports, surgical or motion actions; (b) instructing said trainee to imagine executing said motion action; (c) measuring electroencephalographic signals on said electroencephalographic electrode arrangement; (d) calculating said concentration index, motor control index and alertness index; e) providing said trainee with a feedback pattern characterizing at least one of said concentration, motor control, alertness and sports readiness; and (f) recurring steps c to e if needed.
• a further object of the invention is to disclose the method comprising a step of providing a display configured for providing a visual stimulus to said trainee.
• a further object of the invention is to disclose the steps of displaying said visual stimulus, measuring electroencephalographic signals on said electroencephalographic sensor arrangement and providing said feedback pattern which are performed in a consecutive manner.
• a further object of the invention is to disclose the step of calculating said concentration index comprising calculating a ratio of change of electroencephalographic signals at parietal-zone and frontal -zone electrodes at alfa-, beta- and theta- frequencies obtained from said trainee in response to said visual stimulus over said electroencephalographic signals at parietal-zone and frontal-zone electrodes measured at rest.
• a further object of the invention is to disclose the step of calculating said motor control index comprising calculating a ratio of change of electroencephalographic signals at sensorimotor zone electrodes at Mu-frequency obtained from said trainee in response to said visual stimulus below said electroencephalographic signals at sensorimotor zone electrodes measured at rest.
• a further object of the invention is to disclose the step of calculating said alertness index comprising calculating a ratio of excess of electroencephalographic signals at parietal-zone electrode at alfa-frequency obtained from said trainee with open eyes over said electroencephalographic signals at parietal-zone with closed eyes.
• a further object of the invention is to disclose the method comprising a step of analyzing at least one of said concentration index, motor control index and alertness index of said trainee or a group of said trainees and presenting training progress data in a chronological manner.
• a further object of the invention is to disclose the method comprising a step of calculating an integral index of sports readiness, surgical readiness or pilot readiness as a compound of at least two indexes selected from the group consisting of said concentration index, motor control index and alertness index and normalized by a sum thereof.
• the systems and methods of the present invention are able to furnish coaches and athletes with relevant information for designing of optimal training protocols (e.g., how 7 long can a player maintain high focus; what are the periodic changes of it during the dayAveek/month, etc.).
• information relating to athletes’ brain states of different traits e.g., concentration, motor cortex capacity
• coaches in determining conditions of trainee athletes or team players and establishing competition or game strategy or in scouting process of new team members.
• sports technologies mainly target athletes' performance through physical fitness, flexibility, and other body training types.
• the primary organ that controls movement which is the brain, was just unreachable, at least, to train directly.
• the user's benefit from using a solution based on the current invention includes: (1) performance enhancement, (2) improved return to play after injuries, and (3) a unique approach to new personal Key Performance Indicators KPIs.
• the user learns at first how to enhance specific brain patterns that control his/her movement.
• the brain networks' readiness potential is faster to react when needed ⁇ which can lead to an increase in performance rate and accuracy.
• the main advantages of neurofeedback training emerge with time - constant adaptations of these networks come in reshaping these neural networks. As mentioned above, these neural changes are similar to the natural changes that follow traditional training but are performed in higher intensity without the risk of injuries or fatigue.
• the player can also observe his/her performance over different sessions, track his/her performance in the game, and compare it to his/her in-the-field performance.
• the present invention differs from the prior in providing long-term effects which are data- driven. Brain performance is enhanced by leaming/training rather by using external stimuli. It should be emphasized that any type of leaming/training provides the effects, which last a prolonged period while the effects of the external stimuli are limited by a very short time.
• the feedback provided by the system of the preset invention relates to the current brain state of the tested trainee. From the abovementioned feedback, the trainee can learn how to control the level of activation of these specific brain regions with their own will. These changes are saved in the brain the same way as in any learning process.
• the system of the present invention is configured for recording electroencephalographic signals relating to activation of particular brain regions related to movement such as the primary motor cortex. This is the reason for the use of high-quality EEG hardware - to ensure optimal data collected. By collecting this unique type of data from the user, analyzing it online and offline, we create a training environment that is highly personal which drives the user to increase their performance.
• the system of the present invention is functioning as a Brain-Computer Interface (BCI) for sports and training and includes the following elements: (1) a Brain signal recording arrangement, (2) real-time signal analysis software, and (3) a user’s front end (training environment in game form). Data is available on the ‘’brain at work” (Paras uraman
• Fig. 1 presenting a schematic diagram of system for testing and training a brain capability of planning and executing sports actions.
• Numeral 10 refers to an trainee to be tested.
• System 100 comprises memory unit 50 storing instructions for processing unit 40.
• the aforesaid processing unit is connected to electroencephalographic sensor arrangement 20 attachable to a head of said trainee such electroencephalographic signals generated on the head surface of trainee 10 are detectable.
• Trainee 10 is instructed to imagine executing a predetermined sports action 15 in response to displaying a visual stimulus on display 30. Displaying the visual stimulus to trainee 10 is performed concurrently with measuring electroencephalographic signals on electroencephalographic sensor arrangement 20.
• processing unit calculates a concentration index, a motor control index and an alertness index (described in in detail below). Then a feedback message characterizing at least one said concentration, motor control, alertness and sports readiness are provided to trainee 10.
• the feedback in the form of a static avatar, a dynamic avatar, a text message, a sound pattern, a tactile pattern is in the scope of the present invention.
• an exemplary electroencephalographic cap is made of an elastic synthetic fabric that comes in various sizes.
• the aforesaid cap holds the sensors exactly above tire brain regions of interest.
• the standard EEG cap is known as the “10-20 system”.
• the “International 10-20 system” is a recognized method to describe the scalp electrodes’ location. This standard testing system ensures a subject’s study outcomes (clinical or research) could be compiled, reproduced, and effectively analyzed and compared using the scientific method.
• the system is based on the relationship between the location of an electrode and the underlying area of the brain, specifically the cerebral cortex.
• each EEG sensor is recording the electric fields that are underneath it.
• the neurons communicate with one another with changes in electric charges. It creates a difference in the electric field around them. That is the reason we can decode the brain’s functions by analyzing these changes.
• the electroencephalographic sensor arrangement comprises sensors attachable at:
• frontal zone electrode locates on the midline of the frontal lobe
• parietal zone electrode locates on the midline of the parietal lobe
• ground and reference sensors are used to collect the signal.
• the surface under the sensors is recommended to be cleaned prior to measurement and brought in electric contact the skin surface by means of a conductive gel in order to maximize signalnoise ratio, if necessary'.
• the sensors are connected to an amplifier that increases the electric signal’s magnitude.
• the amplifier is attached to the cap using a cord. It transmits the amplified electroencephalographic signals to a computer, smartphone, or tablet connected by a USB connector using a wireless connection.
• EEG signal amplifiers are usable in the present invention.
• the EEG’s data is streamed to the processing unit and structured with software according to a pre-set sensor montage.
• noise removal features are applied, such as removing the electric network’s static electric disturbances (notch filter).
• the software allows the user to examine each sensor ’s connectivity level, which will enable them to add more gel if the impedance is too high or detect any sensor malfunctioning.
• the EEG data is being streamed using LSL protocol (Lab Streaming Layer), picked up by the data-analyzing code.
• LSL protocol Lab Streaming Layer
• Method 200 starts with providing system 100 (Fig. 1) for testing and training a brain capability of planning and executing sports actions described above (step 210).
• step 210 After instructing said trainee to imagine executing the predetermined sports action in response to displaying a visual stimulus (step 220), the aforesaid visual stimulus is displayed to the trainee to be tested concurrently with measuring electroencephalographic signals on the electroencephalographic sensor arrangement (step 230).
• the obtained electroencephalographic signals are processed and the concentration index, motor control index and alertness index are calculated (step 240).
• a feedback message characterizing at least one of concentration, motor control, alertness and sports readiness step 250. Steps 220-250 are recurred if needed.
• EEG data is received over an LSL socket.
• the data analysis includes filtration.
• Hie EEG rawdata is analyzed in time windows with a shift (e.g., 500 or 1,000 samples with 50% shift) in each cycle.
• the goal of the analysis is to extract relevant brain function features that contribute to successful performance of motion control
• the level of a person’s concentration and activity level in the motor cortex can be deducted from fluctuations of the power of certain frequency bands. We detect it with at least five scalp electrodes.
• Raw data is first filtered using an HR filter, with half-power frequencies for a frequency range [ alpha: 8 to 1 1 Hz; beta: 16-22Hz; and theta 4-7Hz] on the data from sensors attached to parietal zone (alpha) and frontal zone (beta and theta).
• the concentration index for each cycle is the ratio of the powers of beta, theta and alpha.
• Threshold - accuracy At the beginning of each session, the system will determine a baseline that characterizes each trainee. The trainee will sit still in front of an instructed simulation for several minutes (2 minutes by default) to create an open-eyes baseline. The system collects the indices during the baseline collection and is used to set an trainee threshold. That would set the user’s customized boundary. If exceeded, the system can determine that the user’s concentration is high enough to give him positive feedback.
• the aforesaid threshold can be set as a sum of values of lower bound and a compound of difficulty level and difference between upper and lower bounds
• Upper bound is the average of the indices.
• Lower bound is the average minus two standard deviations of the index, and the Difficulty level is a value set by the user to adjust the challenge level to the user’s abilities.
• the indices are continuously calculated and compared with the trainee’s threshold baseline through the backend module.
• the concentration index’s current value exceeds the threshold set for it.
• the module will send the game simulation a message) to grant positive user feedback.
• the user will notice it through shrinkage of a circle indicating the target of the kick, making the shot more accurate.
• intermittent feedback is given (in the soccer game - the ball would be kicked accurately towards the target).
• the algorithm of calculation of the motor control index is the following:
• raw data is filtered using an HR filter, with half-power frequencies for a frequency range [Mu: 12Hz to 15Hz] for sensorimotor zone sensors, such as C3, Cz and C4 above the motor cortex.
• the bandpower function calculates the power of the Mu frequency band and the filtered data from these channels.
• the index from sensorimotor zone sensors, such as C3 and C4 is later used to evaluate current Mu desynchronization which is an action preceding movement.
• Threshold - Kick power all Mu power indices calculated during the acquisition of open-eyes baseline data from the sensorimotor zone, such as C3 and C4 electrodes are used to evaluate the trainee pattern of Mu rhythm of the specific user.
• the average Mu power of locations C3 and C4 (above the left motor cortex that controls the right-side limbs and vice versa) is used as the motor brain activity threshold.
• Training on the basis of in-game feedback during the game the current Mu power is continuously compared with the average value of the data collected in the open-eyes baseline. Specifically, the feedback is defined as positive if the sum of momentary Mu power and compound of difficulty' lever and STD of MU at baseline is smaller than the mean MU at baseline.
• Mu power represents the momentary' Mu value
• Difficulty level is a value set by' the user to adjust the challenge level to the user’s abilities
• STD of Mu at baseline is the standard deviation of all Mu indices collected at open-eyes baseline. Suppose the current Mu power plus a portion of the standard deviation is lower than the average Mu at baseline. In that case, there is an activation of the motor cortex - and positive feedback is given.
• the circular bar In the soccer game training, the circular bar is gradually becoming full, and the color changes from red to green (via yellow and orange). The fill of the bar indicates to the user how to perform the neural action of activating these regions better. If the user succeeds in “ch arging” this bar during one trial, then the kick’s power would be strong enough to score.
• the similar criteria are applicable to training and scouting in other sports games requiring quick reaction and shooting accuracy such as basketball, hockey, golf, American football, and other.
• Ability to mobilize physical vigor and mental alertness is also significant to be estimated in training process in different athletic disciplines such as long/high jumps and javelin/hammer/discus throwing.
• the algorithm of calculation of the alertness (sleep) index is the following:
• the power of the alpha band in the central parietal sensor is coll ected at closed- eyes baseline (several minutes (2 minutes by default), immediately after open-eyes baseline collection) and compared to the alpha power open eyes baseline.
• Alpha power at this location is known as relating to the user’s level of arousal; higher alpha power is usually associated with tiredness.
• Threshold- Sleep detection A moving average for both open-eyes and closed-eyes baseline data is calculated and compared. First, it is based on averaging of one time-window, then the interval increases until the moving average of closed eyes is at least higher in 2 standard deviations than the alpha values at open-eyes state. The moving average with the minimal time interval size that satisfies this condition is set as the threshold for “sleep detection”.
• the current alpha power in the central parietal sensor is continuously compared to the threshold. If it exceeds the threshold, a warning sign is sent (in the soccer game’s case - the digits of the countdown are painted red). If sleep index was detected a few times during one trial, the trial is disqualified (in the soccer game - the player either misses the ball with his leg or even faint on the grass, a voice message stating “Hey! Wake up!”).
• the soccer game is only the first example - but the same concept would be applied in different gaming and training environments.
• advanced ML tools can be used, such as clustering algorithms, SVM classifiers, and Artificial Neural Networks to create a powerful pattern detection mechanism that would be highly user-specific, robust to EEG noise and would provide users with rich data-driven insights.
• the final element of the system the present invention is the training environment for the user.
• Tire interface exposes the user to a real-time representation of his brain activity 7 . It is essential in a neurofeedback learning process and is closing the loop that began in the brain data acquired using the EEG.
• current research indicates that a feedback environment for this kind of neural training is much more efficient in a proper learning environment. Therefore, for each type of sport, unique virtual environments should be developed. The goal is to allow people to train in a familiar environment, which will enable them to transfer their learning efficiently into the "real-world’, such as sports competitions for athletes.
• the interface segment of the present invention is a computer-game environment designed to be highly contextual.
• our first environment is a football (soccer) trainer, and the first task is to kick free shots to the goal.
• Figs 4a to 4i presenting an exemplarily changing visual stimulus with improving the tested index.
• the trainee is instructed to sit still, focus on the monitor, and imagine that he or she is kicking the ball to a target set on the screen.
• the system gives the player 8 seconds.
• the neural activity in the case of real and imagery ‘kick’ is very similar. If our system detects strong activation on specific bands in specific brain areas related to this action, an indicator of kick power will increase (image below). This correlation between neural activation and a symbol presented on the screen allows the user to gain intuitive control of the activity of this neural network that is most important in controlling his leg during a kick.
• the brain is an organ that changes continuously, neuron connectivity changes, networks of neurons are created or enhanced when constant activation takes place. These changes are the building blocks of learning, and they are the reason we get better with training and repetition.
• Figs 5a to 5d presenting another embodiment of the present invention.
• the trainee can control in this game is how accurate the kick will be.
• the kick accuracy is controlled by the level of engagement and focus of the player.
• the system recognizes that the user is focused - a circle around the target will gradually lock on the target. Only when the white circle is locked on the target - the ball will be shot to the goal frame.
• the second form of feedback is referred to as ‘intermittent feedback’ - the result of their efforts - if they managed to control both types of input to a sufficient level - the kick that will be taken at the end of the 8 seconds will be successful, and they would score. If only one condition is satisfied, then the player misses. If the player could not reach sufficient concentration level during this trial (1 kick), then the ball would be kicked not accurately. If the user could not get enough power by the motor imagery attempts - the kick would be too weak, and the goalkeeper will stop it.
• Scoring a goal is, by nature, very rewarding for a soccer player.
• a general ‘rule’ is that neural activities that lead to reward will result in stronger connectivity and an increase in resources related to this activity. These changes will increase the player’s speed and accuracy in performing motor actions, granting him a competitive advantage.
• the users can observe their performance and track changes in their ability to concentrate or activate the brain’s motor areas in the progression of the play.
• the interface will allow the coach to design the training session - how many repetitions, locations, which leg is used, etc. Later the athlete and the coach can observe the performance the player achieved in the recent session, compare it to past sessions or other important analyses using a specially designed interface that is presented at the end of a session. It shows statistics based on the results of the training. It allows the users, coaches, or supervisors to have the overall picture of the player’s abilities and improvement using our trainee brain-data.
• the objective of this invention is to create a basis for development of a series of neuro -interface applications for different sports, recovery, and training applications that will allow users to improve their motor performance. These products will be designed with attentiveness to the needs of every target group of users.
• the procedure of the present invention is applicable to basketball, American football, hockey, racing, golf, tennis, and other sports, as well as to rehabilitation process.
• the information relating to surgeons ’ brain states of different traits can be helpfill for surgical instructors and surgical directors in determining conditions of trainee surgeons or testing aptitude of potential surgeons in embarking on this exacting career requiring very specific manual dexterity skills
• Such systems and methods described in the present invention when appropriately adapted, are able to furnish surgeons and dentists with relevant information for designing of optimal training protocols (e.g., how long can a surgeon maintain high focus; what are the periodic changes of high focus periods during the day/week/month, etc.).
• Information relating to surgeon's brain states of different traits can be helpful for surgical educators, proctors and supervisors in determining conditions, professional suitability and assessments of trainee surgeons or operating room (OR) personnel and surgical teams.
• the surgical PI doctor's benefit from using a solution based on the current invention includes: (1) performance enhancement and (2) a unique approach to new personal Key Performance Indicators KPIs.
• KPIs for surgeons can include a well-defined performance measure that is used to observe, analyze. optimize, and transform a surgeon's process to increase satisfaction for both patients and healthcare providers alike. These metrics are commonly used by care facilities to compare their performance to other care facilities and identify areas of improvement. For example, surgical operating room error rate measures the number of mistakes made by the surgeon when treating a patient. The error rate can be represented by (Number of Treatment Errors / Total Treatments) * 100
• Embodiments of the present invention disclose a system fortesting and training a brain capability of planning and executing motion activity.
• the aforesaid system comprises: (a) an electroencephalographic sensor arrangement attachable to a head of said trainee; (b) a processor configured for receiving and analyzing electroencephalographic signals obtained from said trainee in response to said visual stimulus displayed to said trainee; (c) a memory storing instructions when executed by said processor perform: (i) instructing said trainee to imagine executing said sports motion action; (ii) measuring electroencephalographic signals on said electroencephalographic sensor arrangement; (iii) calculating at least one characteristic selected from the following: (1) a concentration index; (2) a motor control index; (3) an alertness index; (iv) providing said trainee with a feedback pattern based on at least one said concentration, motor control, alertness and motion readiness; (v) recurring steps c to e if needed.
• the implementation of system and methods herein disclosed for testing and training a brain capability of planning and executing surgical motion activity can be used to demonstrate construct validity.
• the system of the present invention can be used to test and train psychomotor, visuo-spatial, and perceptual abilities, and can be used to correlate positively with objective tests of such fundamental abilities that have already been shown to predict surgical performance.
• Functional involvement of psychomotor ability in the adaptation, consolidation, and development of skills in endoscopic surgery has been demonstrated (Gallagher AG, McClure N, McGuigan J, et al.) An ergonomic analysis of the fulcrum effect in acquisition of endoscopic skills.
• the ES3 is composed of four principal components: N a Silicon Graphics Incorporated computer which serves as the simulation host platform; N a haptic system controller PC which performs the requisite high rate control of a physical instrument handle associated with a set of virtual surgical instruments; N a virtual voice recognition instructor PC which responds to spoken commands controlling the simulator; N an electro-mechanical platform which houses a physical replica of an endoscope, a mechanically linked surgical hand tool handle, and a mannequin of the external head anatomy
• the subject simulates grasping tissue, transferring it from one gripper to the other, running the bowel by using hand over hand transfer, removing a tool from the operating field and reinserting it accurately, cauterising three subtargets, and maintaining objects within the target box while cauterising three consecutive subtargets
• Outcomes and predictions of the present system and methods of the invention can be evaluated for internal validity and consistency and with the herein described ES3 , and linked with other, standardised measures of cognitive and psychomotor skill for trainee student surgeons.
• Quantitatively defined types of error can be based on the metrics specified to include: - incorrect manoeuvres, with violation of tissue or instrument tolerances; - correctly performed instrument manoeuvres that are out of sequence or inappropriate for that part of an operation; - inefficient force patterns or application, and inefficient manoeuvres or sequences of manoeuvres; - inappropriate variability in technical performance; - inappropriate “dwell time” or “lack of progress,” indicating indecision or confusion.
• the present system and method having been correlated appropriately with standard systems such as the ES3 can be used to provide, contextually accurate analysis and feedback to the student surgeon, for error recognition and correction, in addition to objective comparison
• the database is the fundamental unit that integrates the project.
• the metrics component identified quantifiable measures which then become the fields for the database.
• the system of the present invention acquires measurements during training and can submit data in an automated and standardised format to a database, which may be web based.
• Data may be provided on outcomes that represent an overall assessment of technical skill for an trainee surgeon. These statistics, when assessed together with other archived measures of cognitive and interpersonal skill, may also provide a first order metric for the global assessment of competency.
• the present system and method may be used to provide data in a recursive, and iterative feedback cycle to support the training and assessment of the surgeons and OR teams Datasets for new operations may be acquired and models may be based on them. Support analysis — based on demographics, training, and performance — across many simulated procedures or groups of surgeons to define parameters of competency, skills, and training for credentialling, regulatory, and policy purposes to appropriate surgical boards and societies, and to state and federal agencies may be implemented.
• the trainee or qualified surgeon learns at first how to enhance specific brain patterns that control his/her movement.
• the brain networks' readiness potential is faster to react when needed - which can lead to an increase in performance rate and accuracy.
• the main advantages of neurofeedback training emerge with time - constant adaptations of these networks come in reshaping these neural networks. As mentioned above, these neural changes are similar to the natural changes that follow traditional training but are performed in higher intensity without the risk of injuries or fatigue.
• the player can also observe his/her performance over different sessions, track his/her performance in the game, and compare it to his/her in-the-field performance.
• the present invention differs from the prior in providing long-term effects which are data- driven. Brain performance is enhanced by leaming/training rather by using external stimuli. It should be emphasized that any type of leaming/training provides the effects, which last a prolonged period while the effects of the external stimuli are limited by a very short time.
• the feedback provided by the system of the preset invention relates to the current brain state of the tested trainee. From the abovementioned feedback, the trainee can learn how to control the level of activation of these specific brain regions with their own will. These changes are saved in the brain the same way as in any learning process.
• the system of the present invention is configured for recording electroencephalographic signals relating to activation of particular brain regions related to movement such as the primary motor cortex. This is the reason for the use of high-quality EEG hardware - to ensure optimal data collected. By collecting this unique type of data from the user, analyzing it online and offline, we create a training environment that is highly personal which drives the user to increase their performance.
• the system of the present invention is functioning as a Brain-Computer Interface (BCI) for surgical training and includes the following elements: (1) a Brain signal recording arrangement, (2) real-time signal analysis software, and (3) a user’s front end (training environment in an OR form).
• BCI Brain-Computer Interface
• Fig. 1 presenting a schematic diagram of system for testing and training a brain capability of planning and executing surgical activities and manoeuvres
• Numeral 10 refers to an trainee to be tested.
• System 100 comprises memory unit 50 storing instructions for processing unit 40.
• the aforesaid processing unit is connected to electroencephalographic sensor arrangement 20 attachable to a head of said trainee such electroencephalographic signals generated on the head surface of trainee 10 are detectable.
• Trainee 10 is instructed to imagine executing a predetermined surgical action 15 in response to displaying a visual stimulus on display 30. Displaying the visual stimulus to trainee 10 is performed concurrently with measuring electroencephalographic signals on electroencephalographic sensor arrangement 20.
• processing unit calculates a concentration index, a motor control index and an alertness index (described in in detail below). Then a feedback message characterizing at least one said concentration, motor control, alertness and sports readiness are provided to trainee 10.
• the feedback in the form of a static avatar, a dynamic avatar, a text message, a sound pattern, a tactile pattern is in the scope of the present invention.
• an exemplary electroencephalographic cap is made of an elastic synthetic fabric that comes in various sizes.
• the aforesaid cap holds the sensors exactly above tire brain regions of interest.
• the standard EEG cap is known as the “10-20 system”.
• the “International 10-20 system” is a recognized method to describe the scalp electrodes’ location. This standard testing system ensures a subject’s study outcomes (clinical or research) could be compiled, reproduced, and effectively analyzed and compared using the scientific method.
• the system is based on tire relationship between tire location of an electrode and the underlying area of the brain, specifically the cerebral cortex.
• each EEG sensor is recording the electric fields that are underneath it.
• the neurons communicate with one another with changes in electric charges. It creates a difference in the electric field around them. That is the reason we can decode the brain’s functions by analyzing these changes.
• the electroencephalographic sensor arrangement comprises sensors attachable at:
• frontal zone electrode locates on the midline of the frontal lobe
• parietal zone electrode locates on the midline of the parietal lobe
• ground and reference sensors are used to collect the signal.
• the sensors are connected to an amplifier that increases the electric signal’s magnitude.
• the amplifier is attached to the cap using a cord. It transmits the amplified electroencephalographic signals to a computer, smartphone, or tablet connected by a USB connector using a wireless connection.
• EEG signal amplifiers are usable in the present invention.
• the EEG’s data is streamed to the processing unit and structured with software according to a pre-set sensor montage.
• noise removal features are applied, such as removing the electric network’s static electric disturbances (notch filter), so the software allows the user to examine each sensor ’s connectivity level, which will enable them to add more gel if the impedance is too high or detect any sensor malfunctioning.
• the EEG data is being streamed using LSL protocol (Lab Streaming Layer), picked up by the data-analyzing code.
• LSL protocol Lab Streaming Layer
• Method 200 starts with providing system 100 (Fig. 1) for testing and training a brain capability of planning and executing surgical actions described above (step 210).
• step 210 After instructing said trainee to imagine executing the predetermined surgical action in response to displaying a visual stimulus (step 220), the aforesaid visual stimulus is displayed to the trainee to be tested concurrently with measuring electroencephalographic signals on the electroencephalographic sensor arrangement (step 230).
• the obtained electroencephalographic signals are processed and the concentration index, motor control index and alertness index are calculated (step 240).
• a feedback message characterizing at least one of concentration, motor control, alertness and surgical readiness (step 250). Steps 220-250 are recurred if needed.
• EEG data is received over an LSL socket.
• the data analysis includes filtration.
• the EEG rawdata is analyzed in time windows with a shift (e.g., 500 or 1,000 samples with 50% shift) in each cycle.
• the goal of the analysis is to extract relevant brain function features that contribute to successful performance of motion control
• the level of a person’s concentration and activity level in the motor cortex can be deducted from fluctuations of the power of certain frequency bands. In the present system it is detected with at least five scalp electrodes. Calculating of the “concentration” (or - brain engagement) index is performed according to the following algorithm:
• Raw data is first fil tered using an IIR filter, wi th half-power frequencies for a frequency range [ alpha: 8 to 11 Hz; beta: 16-22Hz; and theta 4-7Hz] on the data from sensors attached to parietal zone (alpha) and frontal zone (beta and theta).
• the concentration index for each cycle is the ratio of the powers of beta, theta and alpha.
• Threshold - accuracy At tire beginning of each session, the system will determine a baseline that characterizes each trainee. The trainee will sit still in front of an instructed surgical activity or OR simulation for several minutes (2 minutes by default) to create an open-eyes baseline. The system collects the indices during the baseline collection and is used to set an traineeized threshold. That would set the user’s customized boundary. If exceeded, the system can determine that the user’s concentration is high enough to give him positive feedback.
• the aforesaid threshold can be set as a sum of values of lower bound and a compound of difficulty level and difference between upper and lower bounds
• Upper bound is the average of the indices.
• Lower bound is the average minus two standard deviations of the index, and the Difficulty level is a value set by the user to adjust the challenge level to the user’s abilities.
• the indices are continuously calculated and compared with the trainee’s threshold baseline through the backend module.
• the concentration index’s current value exceeds the threshold set for it.
• the module will send the game simulation a message) to grant positive user feedback.
• the user will notice it through shrinkage of a circle indicating the target of the suturing, dissection or ablation procedure , making the surgical activity more accurate.
• intermittent feedback is given (a satisfactory stitching sequence)).
• the algorithm of calculation of the motor control index is the following: First, raw data is filtered using an HR filter, with half-pow er frequencies for a frequency range [Mu: 12Hz to 15Hz] for sensorimotor zone sensors, such as C3, Cz and C4 above the motor cortex.
• the bandpower function calculates the power of the Mu frequency band and the filtered data from these channels.
• the index from sensorimotor zone sensors, such as C3 and C4 is later used to evaluate current Mu desynchronization which is an action preceding movement.
• Threshold -power exerted on the surgical manual instrument all Mu power indices calculated during the acquisition of open-eyes baseline data from the sensorimotor zone, such as C3 and C4 electrodes are used to evaluate the trainee pattern of Mu rhythm of the specific user.
• the average Mu power of locations C3 and C4 (above the left motor cortex that controls the right-side limbs and vice versa) is used as the motor brain activity threshold.
• Training on the basis of operation or procedure feedback during the operation or procedure the current Mu power is continuously compared with the average value of the data collected in the open-eyes baseline. Specifically, the feedback is defined as positive if the sum of momentary Mu power and compound of difficulty lever and STD of MU at baseline is smaller than the mean MU at baseline.
• Mu power represents the momentary Mu value
• Difficulty level is a value set by the user to adjust the challenge level to the user’s abilities
• STD of Mu at baseline is the standard deviation of all Mu indices collected at open-eyes baseline. Suppose the current Mu power plus a portion of the standard deviation is lower than the average Mu at baseline. In that case, there is an activation of the motor cortex - and positive feedback is given.
• the circular bar In surgical training, the circular bar is gradually becoming full, and the color changes from red to green (via yellow 7 and orange). The fill of the bar indicates to the user how to perform the neural action of activating these regions better. If the user succeeds in “charging” this bar during one trial, then the power exerted on the surgical manual instrument would be strong enough to complete the predetermined surgical action. Similar criteria are applicable to training and selection in all branches of surgery requiring quick and decisive accuracy
• Ability to mobilize physical vigor and mental alertness is also crucial in the training process for different types of surgery, from general surgery', orthopedic, cardiac, thoracic, neurosurgery, ophthalmic surgery', dental surgery , veterinary' surgery and other types of surgical training.
• Tire power of the alpha band in the central parietal sen sor is collected at closed- eyes baseline (several minutes (2 minutes by default), immediately after open-eyes baseline collection) and compared to the alpha power open eyes baseline.
• Alpha power at this location is known as relating to the user’s level of arousal; higher alpha power is usually associated with tiredness.
• Threshold- Sleep detection A moving average for both open-eyes and closed-eyes baseline data is calculated and compared. First, it is based on averaging of one time-window, then the interval increases until the moving average of closed eyes is at least higher in 2 standard deviations than the alpha values at open-eyes state. The moving average with the minimal time interval size that satisfies this condition is set as the threshold for “sleep detection”.
• the current alpha power in the central parietal sensor is continuously compared to the threshold. If it exceeds the threshold, a w arning sign is sent (in the case of surgery' - tire digits of the countdown may be painted red). If sleep index was detected a few times during one trial, the trial is disqualified (in the case of an operation, the surgeon makes an error, the procedure on the screen is incomplete, an alert is given and a voice message may call out “Hey! Wake up!”).
• advanced ML tools can be used, such as clustering algorithms, SVM classifiers, and Artificial Neural Networks to create a powerful pattern detection mechanism that w'ould be highly user-specific, robust to EEG noise and would provide users with rich data-driven insights.
• the final elem ent of the system the present invention is the training environment for the user.
• the interface exposes the user to a real-time representation of his brain activity. It is essential in a neurofeedback learning process and is closing the loop that began in the brain data acquired using the EEG.
• current research indicates that a feedback environment for this kind of neural training is much more efficient in a proper learning environment. Therefore, for each type of surgery', unique virtual environments are developed. The goal is to allow people to train in a familiar environment, which will enable them to transfer their learning efficiently into the ‘real-world’ of surgery in the OR.
• the interface segment of the present invention is a computer-game surgery environment designed to be highly contextual.
• our first environment is an intestinal operation where a task might be to excise a growth in the intestine and suture the intestine securely, safely and effectively.
• Figs 4a to 4i presenting an exemplarily changing visual stimulus with improving the tested index.
• the surgeon is instructed to sit still, focus on the monitor, and imagine that he or she is excising a target tumour on the screen.
• the system gives the surgeon a predetermined number of seconds to complete the action.
• the neural activity in the case of real and imagery excisions is very similar. If our system detects strong activation on specific bands in specific brain areas related to this action, an indicator of power exerted upon the manual surgical instrument will increase (image below). This correlation between neural activation and a symbol presented on the screen allows the user to gain intuitive control of the activity of this neural network that is most important in controlling his hand during the procedure.
• the brain is an organ that changes continuously, neuron connectivity changes, networks of neurons are created or enhanced when constant activation takes place. These changes are the building blocks of learning, and they are the reason we get better with training and repetition.
• Figs 6 presenting another embodiment of the present invention, which could be the operation of surgical devices in laparoscopy surgery.
• the trainee can control in this game is how accurate the laser beam will be. The accuracy is controlled by the level of engagement and focus of the player.
• the system recognizes that the user is focused - a circle around the target will gradually lock on the target. Only when the white circle is locked on the target - will the laser intensity increase to the surgical level required to ablate tissue.
• the second form of feedback is referred to as ‘intermittent feedback’ - the result of their efforts if they managed to control both types of input to a sufficient level - the operative manual pressure on the surgical instrument that will be taken at the end of tire predetermined time allocation will be successful, and the instrument mediated action would achieve the intended result.. If only one condition is satisfied, then the surgeon fails. If the surgeon could not reach sufficient concentration level during this trial then the surgical action would be inaccurately executed . If the user could not exert enough power by the motor iagcry atempts - the tissue would not be excised correctly (for example).
• the users can observe their performance and track changes in their ability to concentrate or activate the brain’s motor areas in the progression of the operation.
• the interface will allow the surgical educators to design the training session - how many repetitions, locations, complications, simulated emergencies are presented. Later the trainee surgeon and the trainer or proctor can observe the performance of the surgeon achieved in the recent session, compare it to past sessions or other important analyses using a specially designed interface that is presented at the end of a session. It shows statistics based on the results of the training. It allows the surgeons under training and the supervisors, educators or proctors to have the overall picture of the surgeon's abilities and improvement using our traineeized brain-data. AVIATION TESTING AND TRAINING
• Pilots deal with an uncertain environment and face complex interaction with the flightdeck (Causse et al.. 2013; Cakir et al interfere 2016; Revnal et al.. 2016).
• WM pilots' working memory
• ATC air traffic control
• This activity indeed requires mentally storing flight parameters (e.g., heading, altitude, speed) to follow the adequate flight path.
• human working memory is fundamentally limited (Baddelev. 1992; Miller. 1994) and easily overwhelmed when task demand is excessive (Durantin et al.. 2014a).
• Human factor studies emphasized that a variety of environmental stressors may negatively impact pilots' ability to execute ATC clearances (Billings and Cheanev.
• BCI brain computer interface
• Electroencephalography EEG
• fNIRS functional near infrared spectroscopy
• Tire present invention differs from the prior art in the domain of pilot training in that the present invention provides long-term effects on pilots which are data-driven. Brain performance is enhanced by learning/training rather by using external stimuli. It should be emphasized that any type of learning/training provides the effects, which last a prolonged period while the effects of the external stimuli are limited by a very short time.
• Tire feedback provided by the system of the preset invention relates to the current brain state of the tested trainee. From the abovementioned feedback, the trainee can learn how to control the level of activation of these specific brain regions with their own will. These changes are saved in the brain the same way as in any learning process.
• the system of the present invention is configured for recording electroencephalographic signals relating to activation of particular brain regions related to movement such as the primary motor cortex. This is the reason for the use of high-quality EEG hardware - to ensure optimal data collected. By collecting this unique type of data from the user, analyzing it online and offline, we create a training environment that is highly personal which drives the user to increase their performance.
• the system of the present invention is functioning as a Brain-Computer Interface (BCI) for pilot training and includes the following elements: (1) a Brain signal recording arrangement, (2) real-time signal analysis software, and (3) a user’s front end (training environment in a cockpit representational form).
• BCI Brain-Computer Interface
• Fig. 1 presenting a schematic diagram of system for testing and training a brain capability of planning and executing piloting activities and manoeuvres.
• Numeral 10 refers to an trainee to be tested.
• System 100 comprises memory unit 50 storing instructions for processing unit 40.
• the aforesaid processing unit is connected to electroencephalographic sensor arrangement 20 attachable to a head of said trainee such electroencephalographic signals generated on the head surface of trainee 10 are detectable.
• Trainee 10 is instructed to imagine executing a predetermined flight control action 15 in response to displaying a visual stimulus on display 30. Displaying the visual stimulus to trainee 10 is performed concurrently with measuring electroencephalographic signals on electroencephalographic sensor arrangement 20. According to the instructions stored in memory unit 50, processing unit calculates a concentration index, a motor control index and an alertness index (described in in detail below). Then a feedback message characterizing at least one said concentration, motor control, alertness and flight readiness are provided to trainee 10.
• the feedback in the form of a static avatar, a dynamic avatar, a text message, a sound pattern, a tactile pattern is in the scope of the present invention.
• an exemplar ⁇ ' electroencephalographic cap is made of an elastic synthetic fabric that comes in various sizes.
• the aforesaid cap holds the sensors exactly above the brain regions of interest,
• the standard EEG cap is known as the “10-20 sy stem”.
• the “International 10-20 system” is a recognized method to describe the scalp electrodes’ location. This standard testing system ensures a subject’s study outcomes (clinical or research) could be compiled, reproduced, and effectively analyzed and compared using the scientific method.
• the system is based on the relationship between the location of an electrode and the underlying area of the brain, specifically the cerebral cortex.
• each EEG sensor is recording the electric fields that are underneath it.
• the neurons communicate with one another with changes in electric charges. It creates a difference in tire electric field around them. That is the reason we can decode the brain’s functions by analyzing these changes,
• the electroencephalographic sensor arrangement comprises sensors attachable at:
• frontal zone electrode locates on the midline of the frontal lobe
• parietal zone electrode locates on the midline of the parietal lobe
• ground and reference sensors are used to collect the signal.
• the surface under the sensors is recommended to be cleaned prior to measurement and brought in electric contact the skin surface by means of a conductive gel in order to maximize signalnoise ratio, if necessary.
• the sensors are connected to an amplifier that increases the electric signal’s magnitude.
• the amplifier is attached to the cap using a cord. It transmits the amplified electroencephalographic signals to a computer, smartphone, or tablet connected by a USB connector using a wireless connection.
• EEG signal amplifiers are usable in the present invention.
• Tire EEG’s data is streamed to the processing unit and structured with software according to a pre-set sensor montage.
• noise removal features are applied, such as removing the electric network’s static electric disturbances (notch filter).
• the software allows the user to examine each sensor ’s connectivity level, which will enable them to add more gel if the impedance is too high or detect any sensor malfunctioning.
• Tire EEG data is being streamed using LSL protocol (Lab Streaming Layer), picked up by the data-analyzing code.
• LSL protocol Lab Streaming Layer
• Method 200 starts with providing system 100 (Fig. 1) fortesting and training abrain capability of planning and executing piloting actions described above (step 210).
• step 210 After instructing said trainee to imagine executing the predetermined pilot action in response to displaying a visual stimulus (step 220), the aforesaid visual stimulus is displayed to the trainee to be tested concurrently with measuring electroencephalographic signals on the electroencephalographic sensor arrangement (step 230).
• the obtained electroencephalographic signals are processed and the concentration index, motor control index and alertness index are calculated (step 240).
• a feedback message characterizing at least one of concentration, motor control, alertness and flight readiness step 250. Steps 220-250 are recurred if needed.
• EEG data is received over an LSL socket.
• the data analysis includes filtration.
• the EEG rawdata is analyzed in time windows with a shift (e.g., 500 or 1,000 samples with 50% shift) in each cycle.
• the goal of the analysis is to extract relevant brain function features that contribute to successful performance of motion control of cockpit and pilot controls.
• the level of a person’s concentration and activity level in the motor cortex can be deducted from fluctuations of the power of certain frequency bands. In the present system it is detected with at least five scalp electrodes.
• Raw data is first filtered using an HR filter, with half-power frequencies for a frequency range [ alpha: 8 to 11 Hz; beta: 16-22Hz; and theta 4-7Hz] on the data from sensors attached to parietal zone (alpha) and frontal zone (beta and theta).
• the concentration index for each cycle is the ratio of the powers of beta, theta and alpha.
• Threshold accuracy: At the beginning of each session, the system will determine a baseline that characterizes each trainee. The pilot trainee will sit still in front of an instructed flight activity or simulation for several minutes (2 minutes by default) to create an open-eyes baseline. Tire system collects the indices during the baseline collection and is used to set an trainee actualized threshold which would set the user’s customized boundary. If exceeded, the system can determine that the user’s concentration is high enough to give the trainee positive feedback.
• the aforesaid threshold can be set as a sum of values of lower bound and a compound of difficulty level and difference between upper and lower bounds
• Upper bound is the average of the indices.
• Lower bound is the average minus two standard deviations of the index, and the Difficulty level is a value set by the user to adjust the challenge level to the user’s abilities.
• the indices are continuously calculated and compared with the pilot trainee’s threshold baseline through the backend module.
• the concentration index’s current value exceeds the threshold set for it.
• the module will send the flight simulation a message) to grant positive user feedback.
• the user will notice take off, landing or in flight procedure execution through shrinkage of a circle indicating the target, making the piloting and aircraft handling procedure more accurate.
• intermittent feedback is given (a satisfactory and safe approach resulting from appropriate throttle and flaps coordination).
• the algorithm of calculation of the motor control index is the following:
• raw data is filtered using an HR filter, with half-power frequencies for a frequency range [Mu: 12Hz to 15Hz] for sensorimotor zone sensors, such as C3, Cz and C4 above the motor cortex.
• the bandpower function calculates the power of the Mu frequency band and the filtered data from these channels.
• the index from sensorimotor zone sensors, such as C3 and C4 is later used to evaluate current Mu desynchronization which is an action preceding movement.
• the average Mu power of locations C3 and C4 (above the left motor cortex that controls the right-side limbs and vice versa) is used as the motor brain activity threshold.
• Training on the basis of operation or procedure feedback during the operation or procedure the current Mu power is continuously compared with the average value of the data collected in the open-eyes baseline. Specifically, the feedback is defined as positive if the sum of momentary Mu power and compound of difficulty lever and STD of MU at baseline is smaller than the mean MU at baseline.
• Mu power represents the momentary Mu value
• Difficulty level is a value set by the user to adjust the challenge level to the user’s abilities
• STD of Mu at baseline is the standard deviation of all Mu indices collected at open-eyes baseline. Suppose the current Mu power plus a portion of the standard deviation is lower than the average Mu at baseline. In that case, there is an activation of the motor cortex - and positive feedback is given.
• the algorithm of calculation of the alertness (sleep) index is the following:
• Alpha power at this location is known as relating to the user’s level of arousal; higher alpha power is usually associated with tiredness.
• Threshold- Sleep detection A moving average for both open-eyes and closed-eyes baseline data is calculated and compared. First, it is based on averaging of one time-window, then the interval increases until the moving average of closed eyes is at least higher in 2 standard deviations than the alpha values at open-eyes state. The moving average with the minimal time interval size that satisfies this condition is set as the threshold for “sleep detection”.
• the current alpha pow er in the central parietal sensor or other appropriate location is continuously compared to tire threshold.
• the pilot makes an error such as an overshoot, the procedure on the screen is incomplete, an alert is given and a voice message may call out “Hey! Wake up, Danger!”.
• advanced ML tools can be used, such as clustering algorithms, SVM classifiers, and Artificial Neural Networks to create a powerful pattern detection mechanism that would be highly user-specific, robust to EEG noise and would provide users with rich data-driven insights.
• the final element of the system the present invention is the training environment for the user.
• the interface exposes the user to a real-time representation of his brain activity. It is essential in a neurofeedback learning process and is closing the loop that began in the brain data acquired using the EEG.
• current research indicates that a feedback environment for this kind of neural training is much more efficient in a proper learning environment. Therefore, for each type of flying, unique virtual environments are be developed. The goal is to allow pilots to train in a familiar environment, which will enable them to transfer their learning efficiently into the ‘real-world’ of flying and controlling an aircraft.
• the interface segment of the present invention is a computer-game environment designed to be highly contextual.
• our first environment is an intestinal operation where a task might be to excise a growth in the intestine and suture the intestine securely, safely and effectively.
• Figs 4a to 4i presenting an exemplarily changing visual stimulus with improving the tested index.
• the pilot is instructed to sit still, focus on the monitor, and imagine that he or she is executing a landing on the screen.
• the system gives the pilot a predetermined number of seconds.
• the neural activity in the case of real and imaginary flight activities is very similar. If our system detects strong activation on specific bands in specific brain areas related to this action, an indicator of power exerted upon the joystick or other manual flight control will increase (image below).
• This correlation between neural activation and a symbol presented on the screen allows the user to gain intuitive control of the activity of this neural network that is most important in controlling the operating arm and hand during the controlling of the joystick procedure.
• the brain is an organ that changes continuously, neuron connectivity changes, networks of neurons are created or enhanced when constant activation takes place. These changes are the building blocks of learning, and they are the reason we get better with training and repetition.
• the second form of feedback is referred to as ‘intermittent feedback’ - the result of their efforts - if they managed to control both types of input to a sufficient level - the operative manual pressure on the surgical instrument that will be taken at the end of the predetermined time allocation will be successfill, and the flight control mediated action would achieve the intended result. If only one condition is satisfied, then the pilot fails.. If the pilot could not reach a sufficient concentration level during this trial then the flight control action would be inaccurately executed. If the pilot could not exert enough power by the motor imagery attempts - the flight would become unstable and failure may ensue.
• the pilots can observe their performance and track changes in their ability to concentrate or activate the brain’s motor areas in the progression of the operation.
• the interface will allow the pilot instructors to design the training session - how many repetitions, locations, complications, simulated emergencies are presented. Later the trainee pilot and the instructor can observe the performance of the pilot n achieved in the recent session, compare it to past sessions or other important analyses using a specially designed interface that is presented at the end of a session. It shows statistics based on the results of the training. It allows the pilots under training and the instructors to have the overall picture of the pilot's abilities and improvement using our trained brain-data.
• the present invention provides the system and method for testing and training a brain capability of planning and executing motion activity. It should be emphasized that the disclosed invention is usable in any kind of human motion activity such as sport, surgery, aviation, post- traumatic and post-disease rehabilitation.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Engineering & Computer Science (AREA)
• General Health & Medical Sciences (AREA)
• Medical Informatics (AREA)
• Physics & Mathematics (AREA)
• Public Health (AREA)
• Psychiatry (AREA)
• Heart & Thoracic Surgery (AREA)
• Molecular Biology (AREA)
• Surgery (AREA)
• Animal Behavior & Ethology (AREA)
• Pathology (AREA)
• Biomedical Technology (AREA)
• Veterinary Medicine (AREA)
• Biophysics (AREA)
• Psychology (AREA)
• Social Psychology (AREA)
• Hospice & Palliative Care (AREA)
• Child & Adolescent Psychology (AREA)
• Developmental Disabilities (AREA)
• Educational Technology (AREA)
• Business, Economics & Management (AREA)
• General Physics & Mathematics (AREA)
• Epidemiology (AREA)
• Primary Health Care (AREA)
• Theoretical Computer Science (AREA)
• Educational Administration (AREA)
• Radiology & Medical Imaging (AREA)
• Mathematical Analysis (AREA)
• Entrepreneurship & Innovation (AREA)
• Chemical & Material Sciences (AREA)
• Medicinal Chemistry (AREA)
• Algebra (AREA)
• Computational Mathematics (AREA)
• Pulmonology (AREA)
• Mathematical Optimization (AREA)
• Mathematical Physics (AREA)
• Pure & Applied Mathematics (AREA)
• Rehabilitation Tools (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=88646354

Family Applications (1)

Country Status (4)

Cited By (2)

Citations (4)

Family Cites Families (2)

• 2023
  • 2023-05-03 JP JP2024565110A patent/JP2025516336A/ja active Pending
  • 2023-05-03 EP EP23799385.2A patent/EP4519888A4/en active Pending
  • 2023-05-03 US US18/862,183 patent/US20250302353A1/en active Pending
  • 2023-05-03 WO PCT/IL2023/050453 patent/WO2023214413A1/en not_active Ceased

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events