US20180239430A1 - Brain activity measurement and feedback system - Google Patents

Brain activity measurement and feedback system Download PDF

Info

Publication number
US20180239430A1
US20180239430A1 US15/555,561 US201615555561A US2018239430A1 US 20180239430 A1 US20180239430 A1 US 20180239430A1 US 201615555561 A US201615555561 A US 201615555561A US 2018239430 A1 US2018239430 A1 US 2018239430A1
Authority
US
United States
Prior art keywords
head
eeg
sensors
wearer
stimulation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US15/555,561
Other languages
English (en)
Inventor
Tej TADI
Gangadhar GARIPELLI
Daniel PEREZ MARCOS
Nicolas BOURDAUD
Gerardo de Jesus CHAVEZ CASTANEDA
Leandre BOLOMEY
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mindmaze Holding SA
Original Assignee
Mindmaze Holding SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mindmaze Holding SA filed Critical Mindmaze Holding SA
Publication of US20180239430A1 publication Critical patent/US20180239430A1/en
Assigned to MINDMAZE HOLDING SA reassignment MINDMAZE HOLDING SA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: BOURDAUD, NICHOLAS, MR.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/0036—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room including treatment, e.g., using an implantable medical device, ablating, ventilating
    • A61B5/0478—
    • A61B5/0482—
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/25—Bioelectric electrodes therefor
    • A61B5/279—Bioelectric electrodes therefor specially adapted for particular uses
    • A61B5/291—Bioelectric electrodes therefor specially adapted for particular uses for electroencephalography [EEG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • A61B5/375—Electroencephalography [EEG] using biofeedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6813—Specially adapted to be attached to a specific body part
    • A61B5/6814—Head
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L31/00—Semiconductor devices sensitive to infrared radiation, light, electromagnetic radiation of shorter wavelength or corpuscular radiation and specially adapted either for the conversion of the energy of such radiation into electrical energy or for the control of electrical energy by such radiation; Processes or apparatus specially adapted for the manufacture or treatment thereof or of parts thereof; Details thereof
    • H01L31/02—Details
    • H01L31/0232—Optical elements or arrangements associated with the device
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01L—SEMICONDUCTOR DEVICES NOT COVERED BY CLASS H10
    • H01L33/00—Semiconductor devices having potential barriers specially adapted for light emission; Processes or apparatus specially adapted for the manufacture or treatment thereof or of parts thereof; Details thereof
    • H01L33/48—Semiconductor devices having potential barriers specially adapted for light emission; Processes or apparatus specially adapted for the manufacture or treatment thereof or of parts thereof; Details thereof characterised by the semiconductor body packages
    • H01L33/58—Optical field-shaping elements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/16—Details of sensor housings or probes; Details of structural supports for sensors
    • A61B2562/164—Details of sensor housings or probes; Details of structural supports for sensors the sensor is mounted in or on a conformable substrate or carrier
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/16—Details of sensor housings or probes; Details of structural supports for sensors
    • A61B2562/166—Details of sensor housings or probes; Details of structural supports for sensors the sensor is mounted on a specially adapted printed circuit board
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/053—Measuring electrical impedance or conductance of a portion of the body
    • A61B5/0531—Measuring skin impedance
    • A61B5/0533—Measuring galvanic skin response
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1113—Local tracking of patients, e.g. in a hospital or private home
    • A61B5/1114—Tracking parts of the body
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
    • A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/30—Input circuits therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/389—Electromyography [EMG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/398—Electrooculography [EOG], e.g. detecting nystagmus; Electroretinography [ERG]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4836—Diagnosis combined with treatment in closed-loop systems or methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/486—Bio-feedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/683—Means for maintaining contact with the body
    • A61B5/6831—Straps, bands or harnesses
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7285—Specific aspects of physiological measurement analysis for synchronising or triggering a physiological measurement or image acquisition with a physiological event or waveform, e.g. an ECG signal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/743—Displaying an image simultaneously with additional graphical information, e.g. symbols, charts, function plots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/744—Displaying an avatar, e.g. an animated cartoon character
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
    • A61B5/7445—Display arrangements, e.g. multiple display units

Definitions

  • the present invention relates to a head-mounted device to measure or monitor cortical activity of a user, and optionally to generate stimuli and/or to provide feedback to the user.
  • Applications for use of the invention may include gaming, training, learning of sensory-motor skills, diagnosis or treatment of neurological injury or disease.
  • Neurological injury which follows a stroke often manifests as hemiparesis or other partial paralysis of the body.
  • Current rehabilitation procedures are often based on exercises performed by the impaired body part, the movement of which is tracked in real-time to provide feedback to the patient and/or a medical practitioner.
  • Computer controlled mechanical actuation systems have been used to track a position of, and force applied by, a body part such an arm of a patient as a predetermined movement pattern is executed by the patient. To reduce patient fatigue such systems can support the patient, for example by actuators which can assist during execution of the movement.
  • a disadvantage of such devices is that they can be complicated and expensive.
  • US 2011/0054870 discloses a VR based system for rehabilitation of a patient, wherein a position of a body part of a patient is tracked by a motion camera. Software is used to create a motion avatar, which is displayed to the patient on a monitor. In an example, if a patient moves only a right arm when movement of both arms are prescribed, then the avatar can also display motion of the left arm.
  • a drawback of certain VR based systems is that they only measure the response of the body part to an instructed task. Accordingly, they do not directly measure cortical activity in response to a displayed movement of a body part, only the way in which an area of the brain can control a body part. This may lead to areas of the brain being treated other than those which are damaged, or at least an inability to directly monitors a particular area of the brain. Moreover, the patient is not fully immersed in the VR environment since they look to a separate monitor screen to view the VR environment.
  • VR based systems with brain monitoring and motion tracking are described, the main drawback of known systems being that they do not reliably nor accurately control synchronization between stimulation or action signals and brain activity signals, which may lead to incorrect or inaccurate processing and read out of brain response signals as a function of stimuli or actions.
  • the cost of devices is of course also an important issue in consumer applications.
  • the cost of diagnosis or treatment is important, whereby such costs are affected not only by the cost of devices used per se, but also the cost of using the devices, including: ease of set up, manipulation and of obtaining and interpreting results; reuse and sterilization.
  • EEG head-mounted electroencephalogram
  • Conventional head-mounted electroencephalogram (EEG) sensing devices used in medical or research applications typically comprise a textile cap with a plurality of sensors clipped to the cap.
  • the cap typically is provided with orifices or other fixing means in the positions corresponding to the electrode placement model used.
  • the electrodes are then wired to an amplifying and computing device.
  • Chin straps are provided to stably and accurately adjust and position the cap so that the electrodes are correctly positioned and pressed against the user's head.
  • the positioning of the cap, connection of electrodes and the chip strap or other fixing devices render such conventional systems uninteresting for many consumer applications and tedious even for medical applications.
  • Drawbacks include: not visually aesthetic; overheating of the electrode contact due to reduced air-flow; time-consuming placement of the cap because of adjustments to anatomical inion-nasion distance to ensure 10-20 system and complex set-up (e.g., placement of electrodes, contacts, connectivity etc.); the need for separate electronics and cap electrodes to allow washing of the electrodes.
  • a medical EEG head cap that avoids individual wire connections by mounting and connecting electrodes to a flexible circuit is disclosed in U.S. Pat. No. 4,967,038 and U.S. Pat. No. 5,038,782.
  • Another medical EEG headpiece with elastic straps is disclosed in EP0541393. All three of these prior systems suffer from many of the drawbacks mentioned above.
  • EEG head-mounted devices are disclosed in US 2015/0011857, US 2013/0172721, WO 2013/124366, US2011/0282231, and US2013/0303874, however that also suffer at least some of the drawbacks including the difficulty to adjust for different head sizes, non-optimal placement of electrodes, difficulty to allow washing of the electrodes/head set for re-use, discomfort to wear and/or high manufacturing costs.
  • physiological parameter measurement and motion tracking system e.g., movements head and body
  • a physiological parameter measurement and motion tracking system e.g., movements head and body
  • a brain activity measurement and feedback system that can generate a plurality of stimuli signals of different sources (e.g. visual, auditive, touch sensory, electric, magnetic) and/or that can additionally measure a plurality of physiological response signals of different types (e.g. body part movement, eye movement, galvanic skin response).
  • sources e.g. visual, auditive, touch sensory, electric, magnetic
  • physiological response signals e.g. body part movement, eye movement, galvanic skin response
  • a head set comprising an EEG sensing device comprising EEG sensors configured to be mounted on a head of a wearer so as to position the EEG sensors at selected positions of interest over the wearers scalp.
  • the EEG sensing device comprises a sensor support and a flexible electronic circuit assembled to the sensor support, the EEG sensors connected to the flexible electronic circuit.
  • the sensor support and flexible circuit comprise a central stem configured to extend along a center plane of the top of the head in a direction from nasion to inion, a front lateral branch extending laterally from the central stem configured to extend across a front portion of a wearer's head, a center lateral branch extending laterally from the central stem configured to extend across a top portion of a wearer's head essentially between the wearer's ears, and a rear lateral branch extending laterally from the central stem configured to extend across a back portion of a wearer's head.
  • the sensor support comprises a base wall and side walls extending along edges of the base wall to form an essentially flat “U” shaped channel in which the flexible circuit is assembled.
  • the flexible circuit comprises orifices adjacent the EEG sensor contacts or electrodes, wherein the EEG sensor orifices overlap the flexible circuit orifices such that a through passage between a top surface and a bottom surface of the sensor support is provided to allow conductive gel to be inserted from a top side of the head set while it is positioned on a wearer's head.
  • the through passages could also serve, alternatively or in addition, as access passages for inserting complementary electrodes or other sensors, for instance to test or calibrate the electrodes of the head set, or to add additional sensors or stimulation devices, such as temperature sensors and NIRS, tDCS, tRNS, tACS devices.
  • the through passages may advantageously be used for access by Near Infrared Light emitters and receivers (NIRS) placed over or in these through passages or in other extra orifices.
  • NIRS Near Infrared Light emitters and receivers
  • Near infrared spectroscopy can be very useful to measure blood oxygen levels of the brain tissue. This way one can co-register the electrical and blood-oxygen levels.
  • Each of the lateral branches further comprises extensions extending in a front to rear, or in a rear to front direction.
  • the EEG sensors are positioned in discrete spaced apart positions along the stem, branches and extensions, for instance in positions according to the international 10-20 system, or according to other EEG positioning systems.
  • the center lateral branches comprise extensions extending both in a front to rear and in a rear to front direction such that the center lateral branches can be tensioned to both the front lateral branches and the rear lateral branches.
  • a flexible sealing material is filled over the flexible circuit in the channel in order to seal the electrical circuit tracks and components on the flexible circuit within the channel.
  • the sensor support is a single piece part.
  • the sensor support is molded or formed from a flexible polymeric material.
  • the flexible circuit comprises a single piece flexible substrate.
  • the head set sensor support may further comprise tensioner anchors configured to anchor elastic tensioners between positions in the stem, branches and extensions of the EEG sensing device, and also between the EEG sensing device and a head mount frame support.
  • a base wall of the sensor support comprises EEG sensor orifices to allow access to the EEG sensor contacts or electrodes on the flexible circuit.
  • each EEG sensor on the flexible circuit is positioned a discrete EEG signal amplifier configured to amplify the brain electrical activity signal picked up by the corresponding EEG sensor.
  • the EEG sensor may comprise according to the variant, comprise passive or active sensors.
  • the EEG sensors may comprise electrodes in the form of conductive circuit pads on a surface of a substrate of the flexible circuit intended to face the wearers scalp, or protruding conductive compressible elements mounted on the flexible substrate and electrically connected to a circuit trace of a substrate of the flexible circuit.
  • the EEG sensors may contact the scalp through a wet contact, e.g. using a soft gel, or a semi-solid structure acting in a similar manner to a wet contact, or a dry contact depending on the variant.
  • EEG sensors according to certain variants may not be in direct contact with the scalp, for instance sensors based on near infrared light or capacitive charging
  • the head set may comprise a head-mounted display (HMD) fixed to a head mount frame support and configured to be positioned over the eyes of a wearer of the headset.
  • HMD head-mounted display
  • the HMD may comprise a display unit having display means in the form of an electronic screen configured for positioning in front of the wearer's eyes to present visual information to the wearer and optionally further components providing feedback, stimulation or information to the wearer.
  • the HMD may house various sensing devices and information capture and transmission devices, such as one or more cameras, depth sensors, head movement sensing unit, wireless communication device to interconnect the headset to external electronic devices and computing systems in a wireless fashion, and an on-board power supply for autonomous operation of the head set.
  • various sensing devices and information capture and transmission devices such as one or more cameras, depth sensors, head movement sensing unit, wireless communication device to interconnect the headset to external electronic devices and computing systems in a wireless fashion, and an on-board power supply for autonomous operation of the head set.
  • the headset may further incorporate a plurality of sensors configured to measure different physiological parameters, selected from a group consisting of Electrocorticogram (ECOG) sensors, eye movement sensors, and head movement sensing unit.
  • EOG Electrocorticogram
  • the headset may advantageously further incorporate one or a plurality of brain or nerve stimulation devices, for instance Functional Electrical Stimulation (FES) devices, comprising for instance electrodes configured for trans-cranial alternating current stimulation (tACS), direct current stimulation (tDCS), trans-cranial magnetic stimulation (TMS) and trans-cranial ultrasonic stimulation.
  • FES Functional Electrical Stimulation
  • tACS trans-cranial alternating current stimulation
  • DCS direct current stimulation
  • TMS trans-cranial magnetic stimulation
  • ultrasonic stimulation trans-cranial ultrasonic stimulation.
  • the headset may further incorporate a position/motion detection system operable to detect a position/motion of a body part of a user, the position/motion/detection system comprising one or more colour cameras, and a depth sensor.
  • a physiological parameter measurement system comprising a control system, a sensing system, and a stimulation system
  • the sensing system comprising one or more physiological sensors including at least brain electrical activity sensors mounted in the head set.
  • the stimulation system may comprise one or more stimulation devices including at least a visual stimulation system.
  • the control system may comprising an acquisition module configured to receive sensor signals from the sensing system, and a control module configured to process the signals from the acquisition module and control the generation of stimulation signals to one or more devices of the stimulation system.
  • the control system further comprises a clock module wherein the control system is configured to time stamp signals related to the stimulation signals and the sensor signals with a clock signal from the clock module, enabling the stimulation signals to be synchronized with the sensor signals by means of the time stamps.
  • the time stamped signals related to the stimulation signals may be content code signals received from the stimulation system.
  • the system may advantageously comprise a display register configured to receive display content representing a final stage before the display content is activated on the display, the display register being configured to generate a display content code signal for transmission to the control system, a time stamp being attached to the display content code signal by the clock module.
  • the sensing system may comprise physiological sensors selected from a group comprising Electromyogram (EMG) sensors, Electrooculography (EOG) sensors, Electrocardiogram (ECG) sensors, Inertial Sensors (INS), Body temperature sensor, Galvanic skin sensor, pulse oximetry sensor, and respiration sensors.
  • EMG Electromyogram
  • EOG Electrooculography
  • ECG Electrocardiogram
  • INS Inertial Sensors
  • Body temperature sensor Galvanic skin sensor
  • INS Galvanic skin sensor
  • respiration sensors respiration sensors
  • the stimulation system may comprise stimulation devices selected from a group comprising audio stimulation device, Functional Electrical Stimulation (FES) devices, and haptic feedback devices, said functional electrical stimulation devices being connected to the control system and operable to electrically stimulate one or more body parts of the user.
  • FES devices may be selected from a group consisting of electrodes configured to stimulate nerves or muscles, trans-cranial alternating current stimulation (tACS), direct current stimulation (tDCS), trans-cranial magnetic stimulation (TMS) and trans-cranial ultrasonic stimulation.
  • Each stimulation device may comprise an embedded sensor whose signal is registered by a synchronization device.
  • the system may further comprise in an embodiment, a robotic system for driving movements of a limb of the user and configured to provide haptic feedback.
  • the clock module may be configured to be synchronized with clock module of other systems, including external computers.
  • the system may advantageously comprise an exercise logic unit configured to generate visual display frames including instructions and challenges to the display unit.
  • the system may advantageously comprise an events manager unit configured to generate and transmit stimulation parameters to the stimulation unit.
  • FIGS. 1 a and 1 b are schematic illustrations of prior art systems
  • FIG. 2 a is a schematic diagram illustrating an embodiment of the invention in which display content displayed to a user is synchronized with response signals (e.g. brain activity signals) measured from the user;
  • response signals e.g. brain activity signals
  • FIG. 2 b is a schematic diagram illustrating an embodiment of the invention in which audio content played to a user is synchronized with response signals (e.g. brain activity signals) measured from the user;
  • response signals e.g. brain activity signals
  • FIG. 2 c is a schematic diagram illustrating an embodiment of the invention in which a plurality of signals applied to a user are synchronized with response signals (e.g. brain activity signals) measured from the user;
  • response signals e.g. brain activity signals
  • FIG. 2 d is a schematic diagram illustrating an embodiment of the invention in which a haptic feedback system is included;
  • FIG. 2 e is a schematic diagram illustrating an embodiment of the invention in which a neuro-stimulation signal is applied to a user;
  • FIG. 3 a is a simplified schematic diagram of an embodiment of a physiological parameter measurement system according to the invention.
  • FIG. 3 b is a detailed schematic diagram of a control system of the system of FIG. 3 a;
  • FIG. 3 c is a detailed schematic diagram of a physiological tracking module of the control system of FIG. 3 b;
  • FIG. 4 a is a perspective view of an embodiment of a head-mounted display unit of a headset according to an embodiment of the invention
  • FIGS. 4 b and 4 c are front and back perspective views of a headset according to an embodiment of the invention, mounted on a head of a user;
  • FIG. 4 d is a schematic top view of a headset according to an embodiment of the invention, mounted on a head of a user;
  • FIGS. 4 e and 4 f are bottom and side views of an EEG sensing device of a head set according to an embodiment of the invention.
  • FIGS. 4 g and 4 h are perspective views of a portion of a support of the EEG sensing device according to an embodiment of the invention.
  • FIG. 4 i is a top side view of a flexible circuit of the EEG sensing device according to an embodiment of the invention.
  • FIG. 4 j is a bottom side view of a flexible circuit of the EEG sensing device according to an embodiment of the invention.
  • FIG. 5 is a top view illustrating an arrangement of electroencephalography locations and their nomenclature according to the international 10-20 system
  • FIG. 6 is a front view of an exemplary arrangement of EMG sensors on a body of a user
  • FIG. 7 is a diagrammatic view of a process for training a stroke victim using an embodiment of the system
  • FIG. 8 is a view of screen shots which are displayed to a user during the process of FIG. 7 ;
  • FIG. 9 is a perspective view of a physical setup of a physiological parameter measurement system according to an exemplary embodiment of the invention.
  • FIG. 10 is a schematic block diagram of an example stimulus and feedback trial of a physiological parameter measurement system according to an exemplary embodiment of the invention.
  • FIG. 11 is a schematic block diagram of an acquisition module of a physiological parameter measurement system according to an exemplary embodiment of the invention.
  • FIG. 12 is a diagram illustrating time stamping of a signal by a clock module of a physiological parameter measurement system according to an exemplary embodiment of the invention.
  • FIG. 13 is a data-flow diagram illustrating a method of processing physiological signal data in a control system of a physiological parameter measurement system according to an exemplary embodiment of the invention
  • FIG. 14 is a flowchart diagram illustrating a method of processing events in a control system of a physiological parameter measurement system according to an exemplary embodiment of the invention
  • FIG. 15 a is a bottom view of a portion of the headset according to an embodiment of the invention, showing an electrode orifice in detail;
  • FIG. 15 b is a top view of the portion of the headset of FIG. 15 a;
  • FIG. 15 c is a cross-sectional view through line A-A of FIG. 15 a;
  • FIG. 15 d is a perspective cross-sectional view through line D-D of FIG. 15 b;
  • FIG. 15 e is a cross-sectional view of the portion of headset of FIG. 15 a mounted on a head of a user.
  • a physiological parameter measurement and motion tracking system generally comprises a control system 12 , a sensing system 13 , and a stimulation system 17 .
  • the sensing system comprises one or more physiological sensors including at least brain electrical activity sensors, for instance in the form of electroencephalogram (EEG) sensors 22 .
  • the sensing system may comprises other physiological sensors selected from a group comprising Electromyogram (EMG) sensors 24 connected to muscles in user's body, Electrooculography (EOG) sensors 25 (eye movement sensors), Electrocardiogram (ECG) sensors 27 , Inertial Sensors (INS) 29 mounted on the user's head and optionally on other body parts such as the users limbs, Body temperature sensor, Galvanic skin sensor.
  • EMG Electromyogram
  • EOG Electrooculography
  • ECG Electrocardiogram
  • INS Inertial Sensors
  • the sensing system further comprises position and/or motion sensors to determine the position and/or the movement of a body part of the user.
  • Position and motion sensors may further be configured to measure the position and/or movement of an object in the field of vision of the user. It may be noted that the notion of position and motion is related to the extent that motion can be determined from a change in position.
  • position sensors may be used to determine both position and motion of an object or body part, or a motion sensor (such as an inertial sensor) may be used to measure movement of a body part or object without necessarily computing the position thereof.
  • at least one position/motion sensor comprises a camera 30 and optionally a distance sensor 28 , mounted on a head set 2 configured to be worn by the user.
  • the Stimulation system 17 comprises one or more stimulation devices including at least a visual stimulation system 32 .
  • the stimulation system may comprise other stimulation devices selected from a group comprising audio stimulation device 33 , and Functional Electrical Stimulation (FES) devices 31 connected to the user (for instance to stimulate nerves, or muscles, or parts of the user's brain e.g. to stimulate movement of a limb), and haptic feedback devices (for instance a robot arm that a user can grasp with his hand and that provides the user with haptic feedback).
  • FES Functional Electrical Stimulation
  • the stimulation system may further comprise Analogue to Digital Converters (ADC) 37 a and Digital to Analogue Converters (DAC) 37 b for transfer and processing of signals by a control module 51 of the control system.
  • ADC Analogue to Digital Converters
  • DAC Digital to Analogue Converters
  • Devices of the stimulation system may further advantageously comprise means to generate content code signals 39 fed back to the control system 12 in order to timestamp said content code signals and to synchronize the stimulation signals with
  • the control system 12 comprises a clock module 106 and an acquisition module 53 configured to receive content code signals from the stimulation system and sensor signals from the sensing system and to time stamp these signals with a clock signal from the clock module.
  • the control system further comprises a control module that processes the signals from the acquisition module and controls the output of the stimulation signals to devices of the stimulation system.
  • the control module further comprises a memory 55 to store measurement results, control parameters and other information useful for operation of the physiological parameter measurement and motion tracking system.
  • FIG. 3 a is a simplified schematic diagram of a physiological parameter measurement and motion tracking system 10 according to an embodiment of the invention.
  • the system 10 comprises a control system 12 which may be connected to one or more of the following units: a physiological parameter sensing system 14 ; position/motion detection system 16 ; and a head set 2 , all of which will be described in more detail in the following.
  • the physiological parameter sensing system 14 comprises one or more sensors 20 configured to measure a physiological parameter of a user.
  • the sensors 20 comprise a plurality of electroencephalogram (EEG) sensors 22 mounted in the head set configured to measure cortical activity of a user by measuring electrical activity in a brain of a user.
  • EEG sensors measure voltage fluctuations result from ionic current flows within the neurons of the brain.
  • FIG. 5 shows a top view of schematized head showing a known nomenclature of locations for electrode arrangement corresponding to the international 10-20 system, wherein the shaded groups comprise zones of primary interest in many uses of embodiments of the invention.
  • FIGS. 4 a , 4 b and 4 c shows an arrangement of electroencephalogram sensors 22 of a headset according to an embodiment of the invention, positioned on a head of a user.
  • the head set 2 comprises a head mount frame support 9 configure to position and hold the head set on a user's head, and a brain activity (EEG) sensing device 3 attached to the head mount frame support.
  • EEG brain activity
  • the head set may, in certain embodiments, further comprise a head-mounted display 19 fixed to the head mount frame support,
  • the head-mounted display (HMD) 19 is fixed to the head mount frame support and configured to be positioned over the eyes of a wearer of the headset.
  • the HMD 19 may in embodiments be removably attachable to the head mount frame support.
  • the HMD 19 comprises a display unit 32 mounted in a display unit support 36 , the display unit having display means in the form of an electronic screen configured for positioning in front of the wearer's eyes to present visual information to the wearer.
  • the HMD may further comprise other components providing feedback, stimulation or information to the wearer, such as an audio unit 33 to generate sound for the wearer.
  • the HMD may further conveniently house electronics and support further information capture devices such as one or more cameras 30 , depth sensors 28 , microphone, and various sensors such as a head movement sensing unit 40 , 50 , an eye gaze sensing unit 100 .
  • the HMD may further conveniently house wireless communication devices for wireless communication using any one or more on various known wireless communication protocols to interconnect the headset to external electronic devices and computing systems in a wireless fashion.
  • the HMD may advantageously further house an on-board power supply, in the form of a battery, for autonomous operation of the head set.
  • the head set may also advantageously be provided with one, or a plurality of connectors configured for connecting the head set to external power supply and computing systems or sensors. Further description of various sensors and components of the HMD will be provided further on in this specification.
  • the EEG sensing device 3 is fixed to the head mount frame support and configured to be positioned over the head of a wearer of the headset so as to position electrodes over the wearers scalp at the selected positions of interest for the application concerned (e.g. for medical applications, for gaming applications, for training applications, for research applications, for brain monitoring applications).
  • the electrode positions correspond to positions known from the international 10-20 system, however other systems may be used and electrode positions refined as a function of the results of research on brain electrical activity.
  • the EEG sensing device 3 comprises an EEG sensor support 4 and a flexible circuit 6 assembled to the EEG sensor support 4 .
  • the flexible circuit 6 is configured to capture and process brain activity electrical signals and comprises a flexible circuit substrate 601 on or in which are mounted EEG sensors 22 and an EEG signal processing circuit 8 comprising electronic components 8 a and circuit traces 8 b interconnecting components and sensors and interconnecting the flexible circuit to external power supply and circuits via a connection portion 41 at an end of the flexible circuit.
  • the flexible circuit comprises a pluggable electrical connector for plugging to a complementary pluggable electrical connector on the HMD.
  • a pluggable connector 41 is particularly advantageous in that it allows the user to be able take off the HMD without having to remove the electrode head set. This allows the electrical or electrochemical contacts of the electrodes to be maintained in position and undisturbed on the wearer's scalp.
  • the flexible circuit substrate may advantageously be provided in the form of a thin flexible semi-rigid polymer substrate per se well known in the art of flexible printed circuit technology.
  • the plurality of EEG sensors 22 comprise electrodes spatially positioned on the flexible circuit at the desired electrical brain activity measurement positions, and next to each electrode is positioned a discrete EEG signal amplifier 8 b .
  • the close proximity of the amplifier to the electrode allows the brain electrical activity picked up by the electrode to be amplified locally before transmission to a signal analyzing circuit in order to improve the signal to noise ratio of the measured brain activity signals.
  • the flexible circuit 6 and support 4 of the EEG sensing device are configured such that when they are bent to conforms to a generally spherical or ellipsoid three dimensional form corresponding essentially to the general morphology of a top half of a human head, the EEG sensors are positioned accurately in the desired positions according to the chosen sensor placement. Accuracy and ease of positioning the electrodes correctly and quickly is provided inter alia by the advantageous shape of the sensing device 3 , and in particular of the shape of the support 4 and the flexible circuit 6 mounted in the support.
  • the support comprises a central stem 4 a configured to extend along a center plane of the top of the head in a direction from a nose to a centre of the back of a wearers head, and extending laterally from the central stem 4 a , a front lateral branch 4 b configured to extend across a front portion of a wearer's head, a center lateral branch 4 c configured to extend across a top portion of a wearer's head essentially between the wearer's ears, and a rear lateral branch 4 d configured to extend across a back portion of a wearer's head.
  • Each of the lateral branches further comprise extensions, including back and side extensions 4 b 1 , 4 b 2 , front and rear extensions 4 c 1 , 4 c 2 and front and side extensions 4 d 1 , 4 d 2 respectively.
  • the flexible substrate 6 has a shape that conforms to the support and thus comprises a central stem 6 a , a front lateral branch 6 b , a center lateral branch 6 c , and a rear lateral branch 6 d that are mounted in corresponding portions of the support.
  • Each of the lateral branches further comprises extensions, including back and side extensions 6 b 1 , 6 b 2 , front and rear extensions 6 c 1 , 6 c 2 and front and side extensions 6 d 1 , 6 d 2 respectively that are mounted in corresponding extensions of the support 4 .
  • EEG sensors are positioned in discrete spaced apart positions along the stem, branches and extensions. As best illustrated in FIG. 4 d , the stem, branches and extensions are configured in this example to position EEG sensors at certain positions of interest according to the international 10-20 system.
  • the support 4 may advantageously be made of a flexible elastic or semi-rigid material (as opposed to a textile fabric used in conventional caps) that is flexible enough to bend out of its major plane to conform to the rounded shape of a wearer's head, but that has sufficient self-supporting rigidity in conjunction with the flexible circuit substrate mounted therein, in a direction orthogonal to the bending plane, to generally keep its shape between branches.
  • the thin flexible circuit substrate is very flexible in the direction B perpendicular to the major surface of the substrate but relatively rigid against bending in a direction P parallel to the major surface.
  • the support may be molded in a single piece, for instance injection molded, press die molded, or blow-molded, out of an elastic polymeric material such as a silicon rubber polymer or similar.
  • the support comprises a base wall 401 and side walls 402 extending along edges of the base wall to form an essentially flat “U” shaped channel 403 in which the flexible circuit 6 is inserted.
  • a flexible sealing material 5 for instance a polymeric elastic potting resin or silicon rubber based potting material may be filled over the flexible circuit in the channel, thus forming a top wall of the support, in order to seal the electrical circuit tracks and components on the flexible circuit.
  • a flexible top wall of a similar or different material to the bottom wall could also be assembled over the flexible circuit and bonded, welded or fastened with mechanical means to the base wall.
  • the base wall 401 of the support advantageously comprises EEG sensor orifices 404 to allow access to the EEG sensor contacts or electrodes 221 on the flexible circuit substrate.
  • the EEG sensor orifices may comprise a first portion 404 a below the electrode 221 and a second portion 404 b that is aligned with or overlaps an orifice 602 in the flexible circuit, such that a through passage is formed extending from the bottom surface 406 to the top surface 506 of the support 4 .
  • the through passage provides access from the top of the sensing device to the scalp of the wearer to inject a conductive gel 37 from the top of the device when the sensing device is mounted on a wearer's scalp.
  • the conductive gel 37 provides a good electrical contact between the wearer's skin and the electrodes 221 .
  • a gel- or water-based electrode could be re-wetted by injecting water in the through passages without removing the sensing device.
  • the gel 37 is able to be injected in the second portion 404 a of the orifice and spread laterally to fill the second portion 404 b of the orifice 404 to contact the electrode 221 arranged on a bottom side of the flexible circuit.
  • the through passages could also serve, alternatively or in addition, as access passages for inserting complementary electrodes or other sensors, for instance to test or calibrate the electrodes of the head set, or to add additional sensors or stimulation devices, such as temperature sensors and NIRS, tDCS, tRNS, tACS devices.
  • the through passages may advantageously be used for access by Near Infrared Light emitters and receivers (NIRS) placed over or in these through passages or in other extra orifices.
  • NIRS Near Infrared Light emitters and receivers
  • Near infrared spectroscopy can be very useful to measure blood oxygen levels of the brain tissue. This way one can co-register the electrical and blood-oxygen levels.
  • the support 4 further comprises tensioner anchors 405 , for instance in the form of fixing orifices, that allow elastic ties 7 or other forms of elastic tensioners to be anchored between positions in the stem, branches and extensions of the EEG sensing device, and also between the EEG sensing device and the head mount frame support 9 .
  • the flexible circuit substrate 601 may be provided with corresponding tensioner orifices 605 aligned with the orifices 405 in the support.
  • the elastic tensioners 7 apply a certain tension in the structure to ensure that the branches and extensions of the sensing device 3 conform in a snug fit to the shape of a wearer's head and further to ensure the correct position of the various EEG sensors 22 on the wearers scalp.
  • the stem and branch arrangement of the sensing device ensures the correct position of the front, rear and central electrodes, whereby the elastic tensioners allow a certain adjustment in the distance between center and front, and between center and rear, to conform to different head sizes.
  • the sensing device may be provided in various sizes, whereby with three different sizes more than 95% of the range of adult human head sizes may be covered with accurate electrode placement. For children additional sizes may be provided.
  • the different sizes may concern essentially the length of the central stem and central lateral branch.
  • Anchors 405 are positioned at extremities of each branch and extension, and also in opposing positions within branches to anchor the opposing end of an elastic tensioner 7 connected to an extremity, as best illustrated for instance in FIGS.
  • each elastic ties may be pre-adjusted during assembly of the sensing device and thereafter remain fixed lengths, but in some embodiments some or all of the elastic tensioners may be of adjustable lengths.
  • the elastic tensioners may be provided in different forms, for instance in the form of an elastic textile based band.
  • some or all of the tensioners 7 could be integrally formed with the support 4 as a single part.
  • some or all of the tensioners could be molded with the support 4 , for instance in the same flexible polymeric material as the support.
  • the elastic tensioners may be provided with a thinner wall or smaller width, or with serpentine sections so as to increase the elasticity of the elastic tensioners in comparison to the support portion.
  • different materials may be incorporated in different portions, for instance by overmolding or by welding inserts of another material in the injected or molded main support material, or for instance by dual component injection molding.
  • the head mount frame support 9 may have various configurations, its function being to support the sensing device under a certain elastic tension (as described above) and as a function of the embodiment, to further support the head-mounted display.
  • the head mount frame support 9 may comprise a head band with an adjustable diameter to allow a correct fit around a wearer's head, the head band being rigid against bending at least in the pulling direction of the elastic tensioners 7 .
  • Other head mount structures including helmet like structures to support the HMD and sensing device may be used within the scope of this invention.
  • the EEG electrode may be in the form of a conductive circuit pad on a surface of the flexible circuit substrate intended to face the wearer's scalp.
  • a liquid or self-supporting conductive gel or solid gel may be position on the electrode for contact with the wearer's scalp.
  • the conductive bridge between the electrical contact on the flexible circuit substrate and scalp of the wearer may be provided by a conductive bead, stud or other protruding element for a dry electrode or wet electrode contact the against the wearer's scalp.
  • the EEG electrode may also be in the form of a protruding conductive compressible element, for instance a stamped and formed sheet metal contact, mounted on the flexible substrate and electrically connected to a circuit trace of the substrate by crimping, welding, soldering or other per se known techniques for connecting electronic components on a flexible circuit board.
  • a protruding conductive compressible element for instance a stamped and formed sheet metal contact
  • EEG sensors may include sensors that do not require any direct electrical contact with the scalp (or skin) and may in particular include sensors based on (i) near infrared light and (ii) capacitive charging, according to the per se known measurement principles discussed below:
  • NIRS Near Infrared Spectroscopy
  • Capacitive electrodes are based on the measurement of charge displacements on the body surface due to brain activity. This change of the charge can in turn affect the charge on a metal plate close to the body. Since this electrical plate does not require direct electrical contact to the body, it can be isolated from the body. The measurement of the capacitive EEG (cEEG) is therefore also possible through the hair.
  • a supersensitive signal amplifier is preferably connected to this plate which amplifies the brain signal and processes it.
  • the one or more sensors 20 may additionally comprise sensors 24 configured to measure movement of a muscle of a user, for example by measuring electrical potential generated by muscle cells when the cells are electrically or neurologically activated.
  • a suitable sensor is an electromyogram EMG sensor.
  • the sensors 24 may be mounted on various parts of a body of a user to capture a particular muscular action. For example for a reaching task, they may be arranged on one or more of the hand, arm and chest.
  • FIG. 6 shows an exemplary sensor arrangement, wherein the sensors 24 are arranged on the body in: a first group 24 a on the biceps muscle; a second group 24 b on the triceps muscle; and a third group 24 c on the pectoral muscle.
  • the one or more sensors 20 may additionally comprise sensors 25 configured to measure electrical potential due to eye movement.
  • a suitable sensor is an electrooculography (EOG) sensor.
  • EOG electrooculography
  • FIG. 4 a there are four sensors that may be arranged in operational proximity to the eye of the user. However it will be appreciated that other numbers of sensors may be used.
  • the sensors 25 are conveniently connected to a display unit support 36 of the head set, for example they are affixed thereto or embedded therein.
  • the sensors 20 may alternatively or additionally comprise one or more of the following sensors: electrocorticogram (ECOG); electrocardiogram (ECG); galvanic skin response (GSR) sensor; respiration sensor; pulse-oximetry sensor; temperature sensor; single unit and multi-unit recording chips for measuring neuron response using a microelectrode system.
  • sensors 20 may be invasive (for example ECOG, single unit and multi-unit recording chips) or non-invasive (for example EEG).
  • Pulse-oximetry sensor is used for monitoring a patient's oxygen saturation, usually placed on finger tip and may be used to monitor the status of the patient. This signal is particularly useful with patients under intensive care or special care after recovery from cardio-vascular issues.
  • the information provided by the sensors may be processes to enable tracking of progress of a user.
  • the information may also be processed in combination with EEG information to predict events corresponding to a state of the user, such as the movement of a body part of the user prior to movement occurring.
  • the information provided by the sensors may be processed to give an indication of an emotional state of a user.
  • the information may be used during the appended example to measure the level of motivation of a user during the task.
  • the head set 2 of the physiological parameter sensing system 14 comprises a wireless transceiver which is operable to wirelessly transfer data to external devices, for instance to a wireless transceiver of the physiological parameter processing module 54 , or to a wireless transceiver of the skeletal tracking module 52 .
  • the position/motion detection system 16 comprises one or more sensors 26 suitable for tracking motion of the skeletal structure or a user, or part of the skeletal structure such as an arm.
  • the sensors comprise one or more cameras which may be arranged separate from the user or attached to the head set 2 . At least one, or each camera is arranged to capture the movement of a user and pass the image stream to a skeletal tracking module which will be described in more detail in the following.
  • the sensors 26 may comprise three cameras: two colour cameras 28 a , 28 b and a depth sensor camera 30 .
  • a suitable colour camera may for instance have a resolution of VGA 640 ⁇ 480 pixels and a frame rate of at least 60 frames per second. The field of view of the camera may also be matched to that of the head-mounted display, as will be discussed in more detail in the following.
  • a suitable depth camera may have a resolution of QQ VGA 160 ⁇ 120 pixels.
  • two colour cameras 28 a and 28 b and the depth sensor 30 are arranged on a display unit support 36 of the head set 2 (which is discussed in more detail below) as shown in FIG. 4 .
  • the colour cameras 28 a , 28 b may be arranged over the eyes of the user such that they are spaced apart, for example, by the distance between the pupil axes of a user which is about 65 mm. Such an arrangement enables a stereoscopic display to be captured and thus recreated in VR as will be discussed in more detail in the following.
  • the depth sensor 30 may be arranged between the two cameras 28 a , 28 b.
  • the head set 2 comprises a display unit 32 having a display means 34 a , 34 b for conveying visual information to the user.
  • the display means 34 comprises a head-up display, which is mounted on an inner side of the display unit in front of the eyes of the user so that the user does not need to adjust their gaze to see the information displayed thereon.
  • the head-up display may comprise a non-transparent screen, such an LCD or LED screen for providing a full VR environment. Alternatively it may comprise a transparent screen, such that the user can see through the display whilst data is displayed on it.
  • Such a display is advantageous in providing an augmented reality AR.
  • the display unit may comprise a 2D or 3D display which may be a stereoscopic display.
  • the image mage be an augmented reality image, mixed reality image or video image.
  • the display unit 32 is mounted to a display unit support 36 .
  • the display unit support 36 supports the display unit 32 on the user and provides a removable support for the head set 2 on the user.
  • the display unit support 36 extends around the eyes of the user as best seen in FIGS. 4 a to 4 c.
  • the display unit 32 may be separate from the head set.
  • the display means 34 comprises a monitor or TV display screen or a projector and projector screen.
  • the system 10 comprises a head movement sensing unit 40 .
  • the head movement sensing unit comprises a movement sensing unit 42 for tracking head movement of a user as they move their head during operation of the system 10 .
  • the head movement sensing unit 42 is configured to provide data in relation to the X, Y, Z coordinate location and the roll, pitch and yaw of a head of a user.
  • This data is provided to a head tracking module, which is discussed in more detail in the following, and processes the data such that the display unit 32 can update the displayed VR images in accordance with head movement. For example, as the user moves their head to look to the left the displayed VR images move to the left. Whilst such an operation is not essential it is advantageous in providing a more immersive VR environment.
  • the maximum latency of the loop defined by movement sensed by the head movement sensing unit 42 and the updated VR image is 20 ms.
  • the head movement sensing unit 42 comprises an acceleration sensing means 44 , such as an accelerometer configured to measure acceleration of the head.
  • the sensor 44 comprises three in-plane accelerometers, wherein each in-plane accelerometer is arranged to be sensitive to acceleration along a separate perpendicular plate. In this way the sensor is operable to measure acceleration in three-dimensions.
  • accelerometers include piezoelectric, piezoresistive and capacitive variants.
  • the head movement sensing unit 42 further comprises a head orientation sensing means 47 which is operable to provide data in relation to the orientation of the head.
  • a head orientation sensing means 47 which is operable to provide data in relation to the orientation of the head.
  • suitable head orientation sensing means include a gyroscope and a magnetometer 48 which are configured to measure the orientation of a head of a user.
  • the head movement sensing unit 42 may be arranged on the head set 2 .
  • the movement sensing unit 42 may be housed in a movement sensing unit support 50 that is formed integrally with or is attached to the display unit support 36 .
  • the system 10 may comprise an eye gaze sensing unit 100 .
  • the eye gaze sensing unit 100 comprises one or more eye gaze sensors 102 for sensing the direction of gaze of the user.
  • the eye gaze sensor 102 comprises one or more cameras arranged in operation proximity to one or both eyes of the user.
  • the or each camera 102 may be configured to track eye gaze by using the centre of the pupil and infrared/near-infrared non-collimated light to create corneal reflections (CR).
  • CR corneal reflections
  • other sensing means may be used for example: electrooculogram (EOG); or eye attached tracking.
  • the data from the movement sensing unit 42 is provided to an eye tracking module, which is discussed in more detail in the following, and processes the data such that the display unit 32 can update the displayed VR images in accordance with eye movement. For example, as the user moves their eyes to look to the left the displayed VR images pan to the left. Whilst such an operation is not essential it is advantageous in providing a more immersive VR environment. In order to maintain realism it has been found that the maximum latency of the loop defined by movement sensed by the eye gaze sensing unit 100 and the updated VR image is about 50 ms, however in an advantageous embodiment it is 20 ms or lower.
  • the eye gaze sensing unit 100 may be arranged on the head set 2 .
  • the eye gaze sensing unit 42 may be attached to the display unit support 36 as shown in FIG. 4 a.
  • the control system 12 processes data from the physiological parameter sensing system 14 and the position/motion detection system 16 , and optionally one or both of the head movement sensing unit 40 and the eye gaze sensing module 100 , together with operator input data supplied to an input unit, to generate a VR (or AR) data which is displayed by the display unit 32 .
  • the control system 12 may be organized into a number of modules, such as: a skeletal tracking module 52 ; a physiological parameter processing module 54 ; a VR generation module 58 ; a head tracking module 58 ; and an eye gaze tracking module 104 which are discussed in the following.
  • the skeletal tracking module 52 processes the sensory data from the position/motion detection system 16 to obtain joint position/movement data for the VR generation module 58 .
  • the skeletal tracking module 52 as shown in FIG. 3 b , comprises a calibration unit 60 , a data fusion unit 62 and a skeletal tracking unit 64 the operations of which will now be discussed.
  • the sensors 26 of the position/motion detection system 16 provide data in relation to the position/movement of a whole or part of a skeletal structure of a user to the data fusion unit 62 .
  • the data may also comprise information in relation to the environment, for example the size and arrangement of the room the user is in.
  • the sensors 26 comprise a depth sensor 30 and a colour cameras 28 a , 28 b the data comprises colour and depth pixel information.
  • the data fusion unit 62 uses this data, and the calibration unit 62 , to generate a 3D point cloud comprising a 3D point model of an external surface of the user and environment.
  • the calibration unit 62 comprises data in relation to the calibration parameters of the sensors 26 and a data matching algorithm.
  • the calibration parameters may comprise data in relation to the deformation of the optical elements in the cameras, colour calibration and hot and dark pixel discarding and interpolation.
  • the data matching algorithm may be operable to match the colour image from cameras 28 a and 28 b to estimate a depth map which is referenced with respect to a depth map generated from the depth sensor 30 .
  • the generated 3D point cloud comprises an array of pixels with an estimated depth such that they can be represented in a three-dimensional coordinate system. The colour of the pixels is also estimated and retained.
  • the data fusion unit 62 supplies data comprising 3D point cloud information, with pixel colour information, together with colour images to the skeletal tracking unit 64 .
  • the skeletal tracking unit 64 processes this data to calculate the position of the skeleton of the user and therefrom estimate the 3D joint positions.
  • the skeletal tracking unit is organized into several operational blocks: 1) segment the user from the environment using the 3D point cloud data and colour images; 2) detect the head and body parts of the user from the colour images; 3) retrieve a skeleton model of user from 3D point cloud data; 4) use inverse kinematic algorithms together with the skeleton model to improve joint position estimation.
  • the skeletal tracking unit 64 outputs the joint position data to the VR generation module 58 which is discussed in more detail in the following.
  • the joint position data is time stamped by a clock module such that the motion of a body part can be calculated by processing the joint position data over a given time period.
  • the physiological parameter processing module 54 processes the sensory data from the physiological parameter sensing system 14 to provide data which is used by the VR generation module 58 .
  • the processed data may, for example, comprise information in relation to the intent of a user to move a particular body part or a cognitive state of a user (for example, the cognitive state in response to moving a particular body part or the perceived motion of a body part).
  • the processed data can be used to track the progress of a user, for example as part of a neural rehabilitation program and/or to provide real-time feedback to the user for enhanced adaptive treatment and recovery, as is discussed in more detail in the following.
  • the cortical activity is measured and recorded as the user performs specific body part movements/intended movements, which are instructed in the VR environment. Examples of such instructed movements are provided in the appended examples.
  • the EEG sensors 22 are used to extract event related electrical potentials and event related spectral perturbations, in response to the execution and/or observation of the movements/intended movements which can be viewed in VR as an avatar of the user.
  • slow cortical potentials which are in the range of 0.1-1.5 Hz and occur in motor areas of the brain provide data in relation to preparation for movement
  • mu-rhythm 8-12 Hz
  • beta oscillations 13-30 Hz
  • one or more of the above potentials or other suitable potentials may be monitored. Monitoring such potentials over a period of time can be used to provide information in relation to the recovery or a user.
  • EEG sensors 22 may advantageously be arranged into groups to measure motor areas in one or more areas of the brain, for example: central (C 1 -C 6 , Cz); fronto-central (FC 1 -FC 4 , FCZ); centro-pariental (CP 3 , CP 4 , CPZ).
  • contra lateral EEG sensors C 1 , C 2 , C 3 and C 4 are arranged to measure arm/hand movements.
  • the central, fronto-central and centro-pariental sensors may be used for measuring SCPs.
  • EOG sensors 25 may further be provided to measure eye movement signals. In this way the eye movement signals can be isolated and accounted for when processing the signals of other groups to avoid contamination.
  • the physiological parameter processing module 54 comprises a re-referencing unit 66 which is arranged to receive data from the physiological parameter sensing system 14 and configured to process the data to reduce the effect of external noise on the data. For example, it may process data from one or more of the EEG, EOG or EMG sensors.
  • the re-referencing unit 66 may comprise one or more re-referencing blocks: examples of suitable re-referencing blocks include mastoid electrode average reference, and common average reference. In the example embodiment a mastoid electrode average reference is applied to some of the sensors and common average reference is applied to all of the sensors.
  • suitable noise filtering techniques may be applied to various sensors and sensor groups.
  • the processed data of the re-referencing unit 66 may be output to a filtering unit 68 , however in an embodiment without re-referencing unit, the data from the physiological parameter sensing system 14 is fed directly to the filtering unit 68 .
  • the filtering unit 68 may comprise a spectral filtering module 70 which is configured to band pass filter the data for one or more of the EEG, EOG and EMG sensors.
  • the data may be band pass filtered for one or more of the sensors to obtain the activity on one or more of the bands: SCPs, theta, alpha, beta, gamma, mu, gamma, and delta.
  • the bands SCPs (0.1-1.5 Hz), alpha and mu (8-12 Hz), beta (18-30 Hz) delta (1.5-3.5 Hz), theta (3-8 Hz), low gamma (30-100 Hz) and high gamma (above 100 Hz) are filtered for all of the EEG sensors.
  • similar spectral filtering may be applied but with different spectral filtering parameters. For example, for EMG sensors spectral filtering of a 30 Hz high pass cut off may be applied.
  • the filtering unit 66 may alternatively or additionally comprise a spatial filtering module 72 .
  • a spatial filtering module 72 is applied to the SCPs band data from the EEG sensors (which is extracted by the spectral filtering module 70 ), however it may also be applied to other extracted bands.
  • a suitable form of spatial filtering is spatial smoothing which comprises weighted averaging of neighboring electrodes to reduce spatial variability of the data. Spatial filtering may also be applied to data from the EOG and EMG sensors.
  • the filtering unit 66 may alternatively or additionally comprise a Laplacian filtering module 74 , which is generally for data from the EEG sensors but may also be applied to data from the EOG and EMG sensors.
  • a Laplacian filtering module 72 is applied to each of the Alpha, Mu and Beta band data of the EEG sensors which is extracted by the spectral filtering module 70 , however it may be applied to other bands.
  • the Laplacian filtering module 72 is configured to further reduce noise and increase spatial resolution of the data.
  • the physiological parameter sensing system 14 may further comprise an event marking unit 76 .
  • the event marking unit 76 is arranged to receive processed data from either or both of these units when arranged in series (as shown in the embodiment of FIG. 3 c ).
  • the event marking unit 76 is operable to use event based makers determined by an exercise logic unit (which will be discussed in more detail in the following) to extract segments of sensory data. For example, when a specific instruction to move a body part is sent to the user from the exercise logic unit, a segment of data is extracted within a suitable time frame following the instruction.
  • the data may, in the example of an EEG sensor, comprise data from a particular cortical area to thereby measure the response of the user to the instruction.
  • an instruction may be sent to the user to move their arm and the extracted data segment may comprise the cortical activity for a period of 2 seconds following instruction.
  • Other example events may comprise: potentials in response to infrequent stimuli in the central and centro-parietal electrodes; movement related potentials that are central SCPs (slow cortical potentials) which appear slightly prior to movement and; error related potentials.
  • the event marking unit is configured to perform one or more of following operations: extract event related potential data segments from the SCP band data; extract event related spectral perturbation marker data segments from Alpha and Beta or Mu or gamma band data; extract spontaneous data segments from Beta band data.
  • spontaneous data segments correspond to EEG segments without an event marker, and are different to event related potentials, the extraction of which depends on the temporal location of the event marker.
  • the physiological parameter sensing system 14 may further comprise an artefact detection unit 78 which is arranged to receive the extracted data segments from the event marking unit 76 and is operable to further process the data segments to identify specific artefacts in the segments.
  • the identified artefacts may comprise 1) movement artefacts: the effect of a user movement on a sensor/sensor group 2) electrical interference artefacts: interference, typically 50 Hz from the mains electrical supply 3) eye movement artefacts: such artefacts can be identified by the EOG sensors 25 of the physiological parameter sensing system 14 .
  • the artefact detection unit 78 comprises an artefact detector module 80 which is configured to detect specific artefacts in the data segments.
  • an erroneous segment which requires deleting or a portion of the segment which is erroneous and requires removing from the segment.
  • the advantageous embodiment further comprises an artefact removal module 82 , which is arranged to receive the data segments from the event marking unit 76 and artefact detected from the artefact detector module 80 to perform an operation of removing the detected artefact from the data segment.
  • Such an operation may comprise a statistical method such as a regression model which is operable to remove the artefact from the data segment without loss of the segment.
  • the resulting data segment is thereafter output to the VR generation module 58 , wherein it may be processed to provide real-time VR feedback which may be based on movement intention as will be discussed in the following.
  • the data may also be stored to enable the progress of a user to be tracked.
  • the data from such sensors can be processed using one of more of the abovementioned techniques where applicable, for example: noise reduction; filtering; event marking to extract event relate data segments; artefact removal from extracted data segments.
  • the head tracking module 56 is configured to process the data from the head movement sensing unit 40 to determine the degree of head movement.
  • the processed data is sent to the VR generation module 58 , wherein it is processed to provide real-time VR feedback to recreate the associated head movement in the VR environment. For example, as the user moves their head to look to the left the displayed VR images move to the left.
  • the eye gaze tracking module 104 is configured to process the data from the eye gaze sensing unit 100 to determine a change in gaze of the user.
  • the processed data is sent to the VR generation module 58 , wherein it is processed to provide real-time VR feedback to recreate the change in gaze in the VR environment.
  • the VR generation module 58 is arranged to receive data from the skeletal tracking module 52 , physiological parameter processing module 54 , and optionally one or both of the head tracking module 56 and the eye gaze tracking module 104 , and is configured to process this data such that it is contextualized with respect to a status of an exercise logic unit (which is discussed in more detail in the following), and to generate a VR environment based on the processed data.
  • the VR generation module may be organized into several units: an exercise logic unit 84 ; a VR environment unit 86 ; a body model unit 88 ; an avatar posture generation unit 90 ; a VR content integration unit 92 ; an audio generation unit 94 ; and a feedback generation unit 96 .
  • the exercise logic unit 84 is operable to interface with a user input, such as a keyboard or other suitable input device.
  • the user input may be used to select a particular task from a library of tasks and/or set particular parameters for a task.
  • a body model unit 88 may be arranged to receive data from the exercise logic unit 84 in relation to the particular part of the body required for the selected task. For example this may comprise the entire skeletal structure of the body or a particular part of the body such as an arm. The body model unit 88 thereafter retrieves a model of the required body part, for example from a library of body parts.
  • the model may comprise a 3D point cloud model, or other suitable model.
  • the avatar posture generation unit 90 is configured to generate an avatar based on the model of the body part from the body part model 88 .
  • the VR environment unit 86 is arranged to receive data from the exercise logic unit 84 in relation to the particular objects which are required for the selected task.
  • the objects may comprise a disk or ball to be displayed to the user.
  • the VR content integration unit may be arranged to receive the avatar data from the avatar posture generation unit 90 and the environment data from the VR environment unit 86 and to integrate the data in a VR environment.
  • the integrated data is thereafter transferred to the exercise logic unit 58 and also output to the feedback generation unit 86 .
  • the feedback generation unit 86 is arranged to output the VR environment data to the display means 34 of the head set 2 .
  • the exercise logic unit 84 receives data comprising joint position information from the skeletal tracking module 64 , data comprising physiological data segments from the physiological parameter processing module 54 data from the body model unit 88 and data from the VR environment unit 86 .
  • the exercise logic unit 84 is operable to processes the joint position information data which is in turn sent to the avatar posture generation unit 90 for further processing and subsequent display.
  • the exercise logic unit 84 may optionally manipulated the data so that it may be used to provide VR feedback to the user. Examples of such processing and manipulation include amplification of erroneous movement; auto correction of movement to induce positive reinforcement; mapping of movements of one limb to another.
  • the exercise logic unit 84 may also provide audio feedback.
  • an audio generation unit (not shown) may receive audio data from the exercise logic unit, which is subsequently processed by the feedback unit 94 and output to the user, for example, by headphones (not shown) mounted to the head set 2 .
  • the audio data may be synchronised with the visual feedback, for example, to better indicate collisions with objects in the VR environment and to provide a more immersive VR environment.
  • the exercise logic unit 84 may send instructions to the physiological parameter sensing system 14 to provide feedback to the user via one or more of the sensors 20 of the physiological parameter sensing system 14 .
  • the EEG 22 and/or EMG 24 sensors may be supplied with an electrical potential that is transferred to the user.
  • such feedback may be provided during the task.
  • an electrical potential may be sent to EMG 24 sensors arranged on the arm and/or EEG sensors to attempt to stimulate the user into moving their arm.
  • such feedback may be provided before initiation of the task, for instance, a set period of time before the task, to attempt to enhance a state of memory and learning.
  • control system comprises a clock module 106 .
  • the clock module is used to assign time information to data and various stages of input and output and processing.
  • the time information can be used to ensure the data is processed correctly, for example, data from various sensors is combined at the correct time intervals. This is particularly advantageous to ensure accurate real-time processing of multimodal inputs from the various sensors and to generate real-time feedback to the user.
  • the clock module may be configured to interface with one or more modules of the control system to time stamp data.
  • the clock module 106 interfaces with the skeletal tracking module 52 to time stamp data received from the position/motion detection system 16 ; the clock module 106 interfaces with the physiological parameter processing module 54 to time stamp data received from the physiological parameter sensing system 14 ; the clock module 106 interfaces with the head tracking module 58 to time stamp data received from the head movement sensing unit 40 ; the clock module 106 interfaces with the eye gaze tracking module 104 to time stamp data received from the eye gaze sensing unit 100 .
  • Various operations on the VR generation module 58 may also interface with the clock module to time stamp data, for example data output to the display means 34 .
  • synchronization occurs at the source of the data generation (for both sensing and stimulation), thereby ensuring accurate synchronization with minimal latency and, importantly, low jitter.
  • the delay would be as small as 16.7 ms.
  • An important feature of the present invention is that it is able to combine a heterogeneous ensemble of data, synchronizing them into a dedicated system architecture at source for ensuring multimodal feedback with minimal latencies.
  • the wearable compact head-mounted device allows easy recording of physiological data from brain and other body parts.
  • Latency or Delay It is the time difference between the moment of users actual action or brain state to the moment of its corresponding feedback/stimulation. It is a positive constant in a typical application. Jitter ( ⁇ T) is the trial to trial deviation in Latency or Delay. For applications that require for instance immersive VR or AR, both latency T and jitter ⁇ T should be minimized to the least possible. Whereas in brain computer interface and offline applications, latency T can be compromised but jitter ⁇ T should be as small as possible.
  • FIGS. 1 a and 1 b two conventional prior-art system architectures are schematically illustrated. In these the synchronization may be ensured to some degree but jitter ( ⁇ T) is not fully minimized.
  • ⁇ T jitter
  • the above drawbacks are addressed to provide a system that is accurate and scalable to many different sensors and many different stimuli. This is achieved by employing a centralized clock system that supplies a time-stamp information and each sensor's samples are registered in relation to this to the time-stamp.
  • each stimulation device may advantageously be equipped with an embedded sensor whose signal is registered by a synchronization device. This way, a controller can interpret plurality of sensor data and stimulation data can be interpreted accurately for further operation of the system.
  • video content code from a display register may be read.
  • FIG. 2 a an embodiment of the invention in which the content fed to a micro-display on the headset is synchronized with brain activity signals (EEG signals) is schematically illustrated.
  • EEG signals brain activity signals
  • the visual/video content that is generated in the control system is first pushed to a display register (a final stage before the video content is activated on the display).
  • the controller sends a code to a part of the register (say N bits) corresponding to one or more pixels (not too many pixels, so that the user is not disturbed; the corner pixels in the micro display are recommended as they may not be visible to user).
  • the code will be defined by controller describing what exactly the display content is.
  • the acquisition module reads the code from the display register and attaches a time stamp and sends to next modules.
  • EEG samples are also sampled and attached with the same time stamp. This way when EEG samples and the video code samples are arrived at the controller, these samples could be interpreted accordingly.
  • the same principle may be used for an audio stimulation as illustrated in FIG. 2 b .
  • the audio stimulation can be sampled by the data sent to a digital to analog (DAC) converter.
  • DAC digital to analog
  • any kind of stimulation could be directed to the acquisition module using a sensor and an analog to digital (ADC) converter. This can also be achieved by sending the digital signals supplied to DAC as illustrated in the case of audio stimulation.
  • Plural data from an EEG, video camera data or any other sensor e.g. INS: Inertial sensor
  • each sensor or stimulation could be sampled with different sampling frequency. An important point is that the sensor or stimulation data samples are attached with the time-stamp defined with the clock module.
  • Example 1 Operation of System ( 10 ) in Exemplary “Reach an Object” Task
  • an object 110 such as a 3D disk, is displayed in a VR environment 112 to a user.
  • the user is instructed to reach to the object using a virtual arm 114 of the user.
  • the arm 114 is animated based on data from the skeletal tracking module 16 derived from the sensors of the position/motion detection system 16 .
  • the movement is based data relating to intended movement from the physiological parameter processing module 52 detected by the physiological parameter sensing system 14 , and in particular the data may be from the EEG sensors 22 and/or EMG sensors 24 .
  • FIGS. 7 and 8 a - 8 g describe the process in more detail.
  • a user such as a patient or operator, interfaces with a user input of the exercise logic unit 84 of the VR generation module 58 to select a task from a library of tasks which may be stored. In this example a ‘reach an object task’ is selected.
  • the user may be provided with the results 108 of previous like tasks, as shown in FIG. 8 a . These results may be provided to aid in the selection of the particular task or task difficulty.
  • the user may also input parameters to adjust the difficulty of the task, for example based on a level of success from the previous task.
  • the exercise logic unit 84 initializes the task. This comprises steps of the exercise logic unit 84 interfacing with the VR environment unit 86 to retrieve the parts (such as the disk 110 ) associated with the selected task from a library of parts.
  • the exercise logic unit 84 also interfaces with the body model unit 88 to retrieve, from a library of body parts, a 3D point cloud model of the body part (in this example a single arm 114 ) associated with the exercise.
  • the body part data is then supplied to the avatar posture generation unit 90 so that an avatar of the body part 114 can be created.
  • the VR content integration unit 92 receives data in relation to the avatar of the body part and parts in the VR environment and integrates them in a VR environment.
  • This data is thereafter received by the exercise logic unit 84 and is output to the display means 34 of the head set 2 as shown in FIG. 8 b .
  • the target path 118 for the user to move a hand 115 of the arm 114 along is indicated, for example, by colouring it blue.
  • the exercise logic unit 84 interrogates the skeletal tracking module 16 to determine whether any arm movement has occurred.
  • the arm movement being derived from the sensors of the position/motion detection system 16 which are worn by the user. If a negligible amount of movement (for example an amount less than a predetermined amount, which may be determined by the state of the user and location of movement) or no movement has occurred then stage 5 is executed, else stage 4 is executed.
  • stage 4 the exercise logic unit 84 processes the movement data to determine whether the movement is correct. If the user has moved their hand 115 in the correct direction, for example, towards the object 110 , along the target path 118 , then stage 4 a is executed and the colour of the target path may change, for example it is coloured green, as shown in FIG. 8 c . Else, if the user moves their hand 115 in an incorrect direction, for example, away from the object 110 , Then stage 4 b is executed and the colour of the target path may change, for example it is coloured red, as shown as FIG. 8 d.
  • stage 4 c is executed, wherein the exercise logic unit 84 determines whether the hand 115 has reached the object 110 . If the hand has reached the object, as shown in FIG. 8 e then stage 6 is executed, else stage 3 is re-executed.
  • the exercise logic unit 84 interrogates the physiological parameter processing module 52 to determine whether any physiological activity has occurred.
  • the physiological activity is derived from the sensors of the physiological parameter sensing system module 14 , which are worn by the user, for example the EEG and/or EMG sensors. EEG and EMG sensors may be combined to improve detection rates, and in the absence of a signal from one type of sensor a signal from the other type of sensor maybe used. If there is such activity, then it may be processed by the exercise logic unit 84 and correlated to a movement of the hand 115 . For example a characteristic of the event related data segment from the physiological parameter processing module 52 , such as the intensity or duration of part of the signal, may be used to calculate a magnitude of the hand movement 115 . Thereafter stage 6 is executed.
  • a reward score may be calculated, which may be based on the accuracy of the calculated trajectory of the hand 115 movement.
  • FIG. 8 e shows the feedback 116 displayed to the user. The results from the previous task may also be updated.
  • stage 6 b is executed, wherein a marker strength of the sensors of the physiological parameter sensing system module 14 , for example the EEG and EMG, sensors may be used to provide feedback 118 .
  • FIG. 8 f shows an example of the feedback 120 displayed to the user, wherein the marker strength is displayed as a percentage of a maximum value. The results from the previous task may also be updated.
  • stage 7 is executed, wherein the task is terminated.
  • stage 8 if there is no data provided by either of the sensors of the physiological parameter sensing system module 14 or the sensors of the position/motion detection system 16 with in a set period of time then time out 122 occurs, as shown in FIG. 8 g and stage 7 is executed.
  • Example 2 Hybrid Brain-Computer Interface with Virtual Reality Feedback with Head-Mounted Display, Robotic System and Functional Electrical Stimulation
  • the system could exploit Hebbian learning in associating brain's input and output areas in reintegrating the lost movement function.
  • the Hebbian principle is “Any two systems of cells in the brain that are repeatedly active at the same time will tend to become ‘associated’, so that activity in one facilitates activity in the other.”
  • the two systems of cells are the areas of the brain that are involved in sensory processing and in generating motor command.
  • association is lost due to neural injury, it could be restored or re-built via Hebbian training.
  • Hebbian training For the optimal results of this training, one must ensure near perfect synchronization of system inputs and outputs and in providing realtime multi-sensory feedback to the patient with small delay and more importantly almost negligible jitter.
  • the physical embodiment illustrated in FIG. 9 comprises a wearable system having a head-mounted display (HMD) 19 to display virtual reality 3D video content on micro-displays (e.g., in first person perspective), a stereo video camera 30 and a depth camera 28 , whose data is used for tracking the wearers own arm, objects and any second person under the field of view (motion tracking unit).
  • HMD head-mounted display
  • the EEG electrodes 22 placed over the head of the wearer 1 integrated in a head set 2 as previously described (not shown in detail in this figure for simplification), EMG electrodes 24 placed on the arm will measure electrical activity of the brain and of muscles respectively, used for inferring user's intention in making a goal directed movement.
  • IMU 29 Inertial Measurement Unit 29 that is used for tracking head movements.
  • the executed or intended movements are rendered in the virtual reality display.
  • the biological sensor data ie, EEG, EMG, and motion tracing
  • FES functional electrical stimulation
  • the feedback mechanisms shall provide appropriate stimulation tightly coupling to the intention to move to ensure the implementation of Hebbian learning mechanism.
  • a 3D visual cue 81 in this case a door knob, when displayed in the HMD could instruct the patient 1 to make a movement corresponding to opening the door.
  • the patient may attempt to make the suggested movement.
  • Sensor data EEG, EMG, IMU, motion data
  • the control system 51 then extracts the sensor data and infers user intention and a consensus is made in providing feedback to the user through a robot 41 that moves the arm, and HMD displays movement of an avatar 83 , which is animated based on the inferred data.
  • a Functional Electrical Stimulation (FES) 31 is also synchronized together with other feedbacks ensuring a congruence among them.
  • the acquisition unit acquires physiological data (i.e., EEG 22 , EMG 24 , IMU 29 and camera system 30 ).
  • the camera system data include stereo video frames and depth sensor data.
  • stimulation related data such as the moment at which a particular image frame of the video is displayed on the HMD, robot's motor data and sensors 23 and that of FES 31 stimulation data are also sampled by the acquisition unit 53 .
  • This unit associates each sensor and stimulation sample with a time stamp (TS) obtained from the clock input.
  • TS time stamp
  • the synchronized data is then processed by control system and is used in generating appropriate feedback content to the user through VR HMD display, robotic movement as well as FES stimulation.
  • Each sensor data may have different sampling frequency and whose sampling may have not initiated at exact same moment due to non-shared internal clock.
  • the sampling frequency of EEG data is 1 kHz
  • EMG data is 10 KHz
  • IMU data is 300 Hz
  • Video camera data is 120 frames per second (fps).
  • the stimulation signals have different frequencies, where the display refresh rate is at 60 Hz, robot sensors of 1 KHz, and FES data at 1 KHz.
  • the acquisition unit 53 aims at solving the issue of synchronization of inputs and outputs accurately.
  • the outputs of the system are sensed either with dedicated sensors or indirectly recorded from a stage before stimulation, for instance as follows:
  • the acquisition module reads the sensor samples and attaches a time stamp as illustrated in the FIG. 12 .
  • a sample of a sensor arrives from its ADC 37 a , its time of arrival is annotated with next immediate rising edge of the clock signal.
  • a time-stamp is associated.
  • these samples arrive at the controller, it interprets the samples according to the time stamp of arrival leading to minimized jitters across sensors and stimulations.
  • the physiological data signals EEG and EMG are noisy electrical signals and preferably are pre-processed using appropriate statistical methods. Additionally the noise can also be reduced by better synchronizing the events of stimulation and behavior with the physiological data measurements with negligible jitter.
  • FIG. 13 illustrates various stages of the pre-processing (filtering 68 , epoch extraction and feature extraction stages).
  • EEG samples from all the electrodes are first spectrally filtered in various bands (e.g., 0.1-1 Hz, for slow cortical potentials, 8-12 Hz for alpha waves and Rolandic mu rhythms, 18-30 Hz for beta band and from 30-100 Hz for gamma band).
  • bands e.g., 0.1-1 Hz, for slow cortical potentials, 8-12 Hz for alpha waves and Rolandic mu rhythms, 18-30 Hz for beta band and from 30-100 Hz for gamma band.
  • Each of these spectral bands contains different aspects of neural oscillations at different locations.
  • the signals undergo spatial filtering to improve signal-to-noise ratio additionally.
  • the spatial filters include simple processes such as common average removal to spatial convolution with Gaussian window or Laplace windows.
  • the incoming samples are segmented into temporal windows based on event markers arriving from event
  • EEG segments are then fed to feature extraction unit 69 , where temporal correction is first made.
  • temporal correction is removal of baseline or offset from the trial data from a selected spectral band data.
  • the quality of these trials may be assessed using statistical methods such as Outliers detection.
  • the trials are annotated as artefact trials.
  • features are computed from each trial that well describe the underlying neural processing. These features are then fed to a statistical unit 67 .
  • EMG electrode samples are first spectrally filtered, and applied a spatial filter.
  • the movement information is obtained from the envelope or power of the EMG signals.
  • EMG spectral data is segmented and passed to feature extraction unit 69 .
  • the output of EMG feature data is then sent to statistical unit 67 .
  • the statistical unit 67 combines various physiological signals and motion data to interpret the intention of the user in performing a goal directed movement.
  • This program unit includes mainly machine learning methods for detection, classification and regression analysis in interpretation of the features.
  • the outputs of this module are intention probabilities and related parameters which drive the logic of the exercise in the Exercise logic unit 84 .
  • This exercise logic unit 84 generates stimulation parameters which are then sent to a feedback/stimulation generation unit of the stimulation system 17 .
  • FIG. 14 illustrates event detection.
  • the events corresponding to movements and those of external objects or of a second person need to be detected.
  • the data from camera system 30 stereo cameras, and 3D point cloud from the depth sensor
  • the tracking unit module 73 to produce various tracking information such as: (i) patient's skeletal tracking data, (ii) object tracking data, and (iii) a second user tracking data. Based on the requirements of the behavioral analysis, these tracking data may be used for generating various events (e.g., the moment at which patient lifts his hand to hold door knob).
  • IMU data provides head movement information. This data is analyzed to get events such as user moving head to look at the virtual door knob.
  • the video display codes correspond to the video content (e.g., display of virtual door knob, or any visual stimulation). These codes also represent visual events. Similarly FES stimulation events, Robot movement and haptic feedback events are detected and transferred into event manager 71 .
  • Analyzer modules 75 including a movement analyzer 75 a , an IMU analyzer 75 b , an FES analyzer 75 c and a robot sensor analyzer 75 d , process the various sensor and stimulation signals for the event manager 71 .
  • the event manager 71 then sends these events for tagging the physiological data, motion tracking data etc. Additionally these events also sent to Exercise logic unit for adapting the dynamics of exercise or challenges for the patient.
  • the control system interprets the incoming motion data, intention probabilities from the physiological data and activates exercise logic unit and generates stimulation/feedback parameters.
  • the following blocks are main parts of the control system.
  • Example 3 Brain Computer Interface and Motion Data Activated Neural Stimulation with Augmented Reality Feedback
  • a system could provide precise neural stimulation in relation to the actions performed by a patient in real world, resulting in reinforcement of neural patterns for intended behaviors.
  • Actions of the user and that of a second person and objects in the scene are captured with a camera system for behavioral analysis. Additionally neural data is recorded with one of of the modalities (EEG, ECOG etc.) are synchronized with IMU data. The video captured from the camera system is interleaved with virtual objects to generate 3D augmented reality feedback and provided to the user though head-mounted display. Finally, appropriate neural stimulation parameters are generated in the control system and sent to the neural stimulation.
  • Example 2 The implementation of this example is similar to Example 2, except that the head-mounted display (HMD) displays Augmented Reality content instead of Virtual Reality (see FIG. 2 e ). Meaning, virtual objects are embedded in 3D seen captured using stereo camera and displayed on micro displays insuring first person perspective of the scene. Additionally, direct neural stimulation in implemented through such as deep brain stimulation and cortical stimulation, and non-invasive stimulations such as trans-cranial direct current stimulation (tDCS), trans-cranial alternating current stimulation (tACS), trans-cranial magnetic stimulation (TMS) and trans-cranial Ultrasonic stimulation.
  • tDCS trans-cranial direct current stimulation
  • tACS trans-cranial alternating current stimulation
  • TMS trans-cranial magnetic stimulation
  • Ultrasonic stimulation trans-cranial Ultrasonic stimulation.
  • the system can advantageously use one or more than one stimulation modalities at time to optimize the effect. This system exploits the acquisition unit described in the example 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Dermatology (AREA)
  • Neurosurgery (AREA)
  • Neurology (AREA)
  • Power Engineering (AREA)
  • Computer Hardware Design (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Manufacturing & Machinery (AREA)
  • Psychology (AREA)
  • Psychiatry (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Electromagnetism (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
US15/555,561 2015-03-02 2016-03-01 Brain activity measurement and feedback system Abandoned US20180239430A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP15157206.2A EP3064130A1 (fr) 2015-03-02 2015-03-02 Mesure d'activité cérébrale et système de rétroaction
EP15157206.2 2015-03-02
PCT/IB2016/051125 WO2016139576A2 (fr) 2015-03-02 2016-03-01 Système de mesure et de rétroaction d'activité cérébrale

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2016/051125 A-371-Of-International WO2016139576A2 (fr) 2015-03-02 2016-03-01 Système de mesure et de rétroaction d'activité cérébrale

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/000,872 Continuation US12056280B2 (en) 2015-03-02 2020-08-24 Brain activity measurement and feedback system

Publications (1)

Publication Number Publication Date
US20180239430A1 true US20180239430A1 (en) 2018-08-23

Family

ID=52598621

Family Applications (2)

Application Number Title Priority Date Filing Date
US15/555,561 Abandoned US20180239430A1 (en) 2015-03-02 2016-03-01 Brain activity measurement and feedback system
US17/000,872 Active 2037-02-06 US12056280B2 (en) 2015-03-02 2020-08-24 Brain activity measurement and feedback system

Family Applications After (1)

Application Number Title Priority Date Filing Date
US17/000,872 Active 2037-02-06 US12056280B2 (en) 2015-03-02 2020-08-24 Brain activity measurement and feedback system

Country Status (5)

Country Link
US (2) US20180239430A1 (fr)
EP (2) EP3064130A1 (fr)
CN (1) CN107405101A (fr)
ES (1) ES2910273T3 (fr)
WO (1) WO2016139576A2 (fr)

Cited By (73)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170243499A1 (en) * 2016-02-23 2017-08-24 Seiko Epson Corporation Training device, training method, and program
US20180157332A1 (en) * 2016-12-06 2018-06-07 Vuelosophy Inc. Systems and Methods for Tracking Motion and Gesture of Heads and Eyes
CN109009173A (zh) * 2018-08-30 2018-12-18 北京机械设备研究所 一种基于脑电-眼动双模态信号的疲劳检测与调控方法
US20190064924A1 (en) * 2017-08-30 2019-02-28 Disney Enterprises, Inc. Systems And Methods To Synchronize Visual Effects and Haptic Feedback For Interactive Experiences
US10317987B2 (en) * 2015-11-05 2019-06-11 Boe Technology Group Co., Ltd. Wearable device and application system
KR102020598B1 (ko) * 2018-12-11 2019-09-10 전자부품연구원 생체신호 센서 기반 정신질환 진단 및 치유를 위한 바이오피드백 시스템
US20190387995A1 (en) * 2016-12-20 2019-12-26 South China University Of Technology Brain-Computer Interface Based Robotic Arm Self-Assisting System and Method
US10564717B1 (en) * 2018-07-16 2020-02-18 Facebook Technologies, Llc Apparatus, systems, and methods for sensing biopotential signals
US10579141B2 (en) * 2017-07-17 2020-03-03 North Inc. Dynamic calibration methods for eye tracking systems of wearable heads-up displays
US10598936B1 (en) * 2018-04-23 2020-03-24 Facebook Technologies, Llc Multi-mode active pixel sensor
US20200133393A1 (en) * 2018-10-29 2020-04-30 Cognixion Brain computer interface for augmented reality
KR20200072099A (ko) * 2018-12-12 2020-06-22 광운대학교 산학협력단 3d 헤드업 디스플레이와 연동된 웨어러블 eeg 헤드셋을 구비하는 eeg 측정 및 자극 시스템 및 방법
CN111831118A (zh) * 2020-07-10 2020-10-27 清华大学 一种基于增强现实的三维脑电显示方法及其显示系统
WO2020227394A1 (fr) * 2019-05-07 2020-11-12 The Trustees Of Princeton University Système et procédé pour capteur d'oxygène exhalé par courant principal
US10950336B2 (en) 2013-05-17 2021-03-16 Vincent J. Macri System and method for pre-action training and control
CN112751882A (zh) * 2021-01-19 2021-05-04 华南理工大学 一种基于混合脑机接口的实时通讯方法
US20210128904A1 (en) * 2017-01-18 2021-05-06 Viktor S.R.L. Electrostimulation apparatus
WO2021107329A1 (fr) * 2019-11-28 2021-06-03 서울대학교산학협력단 Procédé et appareil d'évaluation d'intégration visuelle et tactile utilisant la réalité virtuelle
US11036288B2 (en) * 2018-04-23 2021-06-15 Beijing Boe Optoelectronics Technology Co., Ltd. Head-mounted virtual reality display device, method for measuring position and posture of the same and virtual reality display apparatus
WO2021168099A1 (fr) * 2020-02-19 2021-08-26 California Institute Of Technology Capteur cérébral à amortissement inductif
US11116441B2 (en) 2014-01-13 2021-09-14 Vincent John Macri Apparatus, method, and system for pre-action therapy
US20210302749A1 (en) * 2020-03-27 2021-09-30 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
CN113456080A (zh) * 2021-05-25 2021-10-01 北京机械设备研究所 一种干湿通用型传感电极及其应用方法
US11199903B1 (en) * 2021-03-26 2021-12-14 The Florida International University Board Of Trustees Systems and methods for providing haptic feedback when interacting with virtual objects
US20210392243A1 (en) * 2020-06-10 2021-12-16 Sam Salemnia Head mountable camera system
US11207491B2 (en) * 2017-09-15 2021-12-28 Christoph Guger Device and method for the learning of the deliberate control of a specified body part by a test subject
CN114010197A (zh) * 2021-11-05 2022-02-08 哈工大机器人（合肥）国际创新研究院 一种基于空间树状网络结构的干湿混合脑电电极
CN114173663A (zh) * 2020-02-07 2022-03-11 Vibraint公司 神经康复系统及神经康复方法
WO2022055948A1 (fr) * 2020-09-08 2022-03-17 Beacon Biosignals, Inc. Systèmes et méthodes de mesure de la neurotoxicité chez un sujet
WO2022061322A1 (fr) * 2020-09-18 2022-03-24 University Of Florida Research Foundation Élimination de composantes de bruit latent à partir de signaux de données
US11327566B2 (en) * 2019-03-29 2022-05-10 Facebook Technologies, Llc Methods and apparatuses for low latency body state prediction based on neuromuscular data
WO2022132670A1 (fr) * 2020-12-15 2022-06-23 Neurable, Inc. Surveillance de données biométriques afin de déterminer des états mentaux et commandes d'entrée
US20220206576A1 (en) * 2020-12-30 2022-06-30 Imagine Technologies, Inc. Wearable electroencephalography sensor and device control methods using same
US11402909B2 (en) 2017-04-26 2022-08-02 Cognixion Brain computer interface for augmented reality
US11399762B1 (en) 2021-08-05 2022-08-02 Starcat LLC Modular electroencephalograph (EEG) system
WO2022204726A1 (fr) * 2021-03-26 2022-09-29 Wave Neuroscience, Inc. Électroencéphalogramme (eeg) basé sur des thérapies de stimulation magnétique transcrânienne (tms)
US11475987B2 (en) 2018-09-12 2022-10-18 California Institute Of Technology Wearable inductive damping sensor
US11481030B2 (en) 2019-03-29 2022-10-25 Meta Platforms Technologies, Llc Methods and apparatus for gesture detection and classification
US11481031B1 (en) 2019-04-30 2022-10-25 Meta Platforms Technologies, Llc Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
US11484243B2 (en) * 2016-08-12 2022-11-01 Nuro Corp. Portable Alzheimer detector
US11493993B2 (en) 2019-09-04 2022-11-08 Meta Platforms Technologies, Llc Systems, methods, and interfaces for performing inputs based on neuromuscular control
US20220391014A1 (en) * 2021-06-08 2022-12-08 Honda Motor Co., Ltd. Operation system and operation method
US11567573B2 (en) 2018-09-20 2023-01-31 Meta Platforms Technologies, Llc Neuromuscular text entry, writing and drawing in augmented reality systems
US20230043938A1 (en) * 2021-08-05 2023-02-09 Zeto, Inc. Flexible electroencephalography headset
WO2023014357A1 (fr) * 2021-08-05 2023-02-09 Starcat LLC Système d'électroencéphalographe (eeg) modulaire
US11583231B2 (en) 2019-03-06 2023-02-21 X Development Llc Adjustable electrode headset
US11589749B2 (en) * 2018-07-23 2023-02-28 Northeastern University Optically monitoring brain activities using 3D-aware head-probe
WO2023027880A1 (fr) * 2021-08-25 2023-03-02 Battelle Memorial Institute Stimulation électrique neuromusculaire commandée par la vision artificielle
US11598967B2 (en) 2020-03-27 2023-03-07 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US20230095350A1 (en) * 2021-09-17 2023-03-30 Smart Science Technology, LLC Focus group apparatus and system
CN115919321A (zh) * 2022-12-29 2023-04-07 中电云脑(天津)科技有限公司 一种便携式无创神经电生理信号采集计算装置
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
US20230129718A1 (en) * 2021-10-21 2023-04-27 Sony Interactive Entertainment LLC Biometric feedback captured during viewing of displayed content
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
US11642081B2 (en) 2019-02-01 2023-05-09 X Development Llc Electrode headset
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
US11673042B2 (en) 2012-06-27 2023-06-13 Vincent John Macri Digital anatomical virtual extremities for pre-training physical movement
WO2023112613A1 (fr) * 2021-12-15 2023-06-22 住友ベークライト株式会社 Dispositif de mesure d'onde cérébrale et procédé de mesure d'onde cérébrale
US11686948B2 (en) 2020-03-27 2023-06-27 ResMed Pty Ltd Positioning, stabilising, and interfacing structures and system incorporating same
US20230218215A1 (en) * 2022-01-10 2023-07-13 Yewon SONG Apparatus and method for generating 1:1 emotion-tailored cognitive behavioral therapy in metaverse space through artificial intelligence control module for emotion-tailored cognitive behavioral therapy
US11701046B2 (en) * 2016-11-02 2023-07-18 Northeastern University Portable brain and vision diagnostic and therapeutic system
WO2023177946A1 (fr) * 2022-03-18 2023-09-21 Apple Inc. Structures de montage flexibles
US20230310934A1 (en) * 2022-03-31 2023-10-05 bOMDIC Inc. Movement determination method, movement determination device and computer-readable storage medium
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11804148B2 (en) 2012-06-27 2023-10-31 Vincent John Macri Methods and apparatuses for pre-action gaming
WO2023239361A1 (fr) * 2022-06-08 2023-12-14 Hewlett-Packard Development Company, L.P. Angle prévu basé sur des valeurs d'emg du muscle
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
US11904101B2 (en) 2012-06-27 2024-02-20 Vincent John Macri Digital virtual limb and body interaction
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
US11961494B1 (en) 2019-03-29 2024-04-16 Meta Platforms Technologies, Llc Electromagnetic interference reduction in extended reality environments
US20240188902A1 (en) * 2021-06-21 2024-06-13 Virtual Reality Concussion Assessment Corporation System for concussion assessment
US20240298943A1 (en) * 2019-11-15 2024-09-12 Looxid Labs Inc. Biosignal sensing device

Families Citing this family (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9747546B2 (en) 2015-05-21 2017-08-29 Google Inc. Neural network processor
US10192162B2 (en) 2015-05-21 2019-01-29 Google Llc Vector computation unit in a neural network processor
US9805303B2 (en) 2015-05-21 2017-10-31 Google Inc. Rotating data for neural network computations
US10438117B1 (en) 2015-05-21 2019-10-08 Google Llc Computing convolutions using a neural network processor
US10083395B2 (en) 2015-05-21 2018-09-25 Google Llc Batch processing in a neural network processor
US10049322B2 (en) 2015-05-21 2018-08-14 Google Llc Prefetching weights for use in a neural network processor
GB2550843A (en) * 2016-05-18 2017-12-06 Univ Ulster Headgear incorporating electrical measurement apparatus
WO2018058231A1 (fr) * 2016-09-28 2018-04-05 Archeoptix Biomedical Inc. Dispositif d'imagerie optique et capuchon
US10796246B2 (en) 2016-12-29 2020-10-06 Arizona Board Of Regents On Behalf Of Arizona State University Brain-mobile interface optimization using internet-of-things
CN110325112A (zh) 2017-01-04 2019-10-11 斯托瑞阿普股份有限公司 使用虚拟现实疗法修改生物测定活动的系统和方法
EP3571627A2 (fr) 2017-01-19 2019-11-27 Mindmaze Holding S.A. Systèmes, procédés, appareils et dispositifs pour détecter une expression faciale et pour suivre un mouvement et un emplacement y compris pour un système de réalité virtuelle et/ou de réalité augmentée
WO2018146558A2 (fr) 2017-02-07 2018-08-16 Mindmaze Holding Sa Systèmes, procédés et appareils de vision stéréo et de suivi
EP3568739A4 (fr) * 2017-03-15 2020-03-11 Samsung Electronics Co., Ltd. Systèmes et procédés permettant de déterminer des défauts dans le champ visuel d'un utilisateur
WO2018174854A1 (fr) 2017-03-21 2018-09-27 Hewlett-Packard Development Company, L.P. Estimations dans des dispositifs d'affichages
CN107362465A (zh) * 2017-07-06 2017-11-21 上海交通大学 一种用于人体经颅超声刺激与脑电记录同步的系统
EP3441030B1 (fr) 2017-08-11 2020-05-13 Stryker European Holdings I, LLC Plaque de trou fraisé crânienne déformable et instrument de déformation correspondant
US10987016B2 (en) 2017-08-23 2021-04-27 The Boeing Company Visualization system for deep brain stimulation
KR102022586B1 (ko) 2017-08-24 2019-09-18 김경모 Vr 장치 및 밴드를 이용한 뇌파 측정 시스템
US10863469B2 (en) 2017-10-11 2020-12-08 Google Llc System and method for accurate timestamping of virtual reality controller data
HRP20171555A2 (hr) 2017-10-13 2019-04-19 Umo Neuroscience J.D.O.O. Sustav i postupak za izvođenje neurološke povratne veze koji uključuje druge životinjske vrste u samom lancu povratne veze
WO2019084007A1 (fr) * 2017-10-23 2019-05-02 The Regents Of The University Of California Systèmes et procédés de suivi d'activité automatique
JP7069716B2 (ja) 2017-12-28 2022-05-18 株式会社リコー 生体機能計測解析システム、生体機能計測解析プログラム及び生体機能計測解析方法
CA3087589A1 (fr) * 2018-01-04 2019-07-11 Interaxon Inc. Dispositif informatique pouvant etre porte
CN108334195B (zh) * 2018-01-17 2019-10-18 西安交通大学 基于调制的生物运动视觉感知的脑-机接口方法
JP7108174B2 (ja) * 2018-01-30 2022-07-28 日本電信電話株式会社 ウェアラブルインタフェース、情報提供装置及び情報提供方法
US10601960B2 (en) 2018-02-14 2020-03-24 Eingot Llc Zero-knowledge environment based networking engine
IT201800005095A1 (it) * 2018-05-07 2019-11-07 Sistema e metodo per la riabilitazione di persone affette da ictus mediante realtà virtuale e gestione dello stato di affaticamento
CN108606789A (zh) * 2018-05-18 2018-10-02 东南大学 一种应用于抑郁情绪监测的脑电头箍
KR102200414B1 (ko) * 2018-08-28 2021-01-08 주식회사 룩시드랩스 생체 데이터 획득용 탈착식 기능모듈 및 이를 포함하는 헤드 마운트 디스플레이 장치
WO2019240564A1 (fr) * 2018-06-15 2019-12-19 주식회사 룩시드랩스 Module de fonction détachable pour acquérir des données biométriques et visiocasque le comprenant
CN108670241A (zh) * 2018-06-27 2018-10-19 华南理工大学 一种新型柔性印刷贴面电极阵列
EP3594925B1 (fr) * 2018-07-13 2022-09-28 Wolfgang Vogel Dispositif, système et procédé pour la synchronisation des ondes cérébrales et l'entraînement du cerveau humain
CN108922299B (zh) * 2018-07-24 2020-12-01 淮北市荣恋科技有限公司 一种消防演习虚拟训练的方法
US11315325B2 (en) * 2018-10-09 2022-04-26 Magic Leap, Inc. Systems and methods for artificial intelligence-based virtual and augmented reality
CN109924976A (zh) * 2019-04-29 2019-06-25 燕山大学 小鼠经颅超声刺激及脑肌电信号同步采集系统
US20220236801A1 (en) * 2019-06-28 2022-07-28 Sony Group Corporation Method, computer program and head-mounted device for triggering an action, method and computer program for a computing device and computing device
WO2021011391A1 (fr) * 2019-07-12 2021-01-21 The Regents Of The University Of California Systèmes et procédés relatifs à la spécificité des souvenirs faisant appel à la réalité virtuelle
TWI726821B (zh) * 2020-10-06 2021-05-01 國立臺灣科技大學 具有多通道的腦機介面裝置
TWI758993B (zh) * 2020-12-04 2022-03-21 高雄醫學大學 基於整合實境與腦機介面之下肢復健系統
CN113040790B (zh) * 2021-03-29 2022-01-28 中国科学院自动化研究所 基于内源性脑信号的闭环神经调控系统、方法和设备
WO2022212798A1 (fr) * 2021-03-31 2022-10-06 University Of Mississippi Medical Center Dispositif sensoriel immersif virtuel et procédés de détection de déficiences neurologiques
BE1029415B1 (fr) 2021-05-19 2022-12-19 P³Lab Un système et un procédé de mesure de réactions d’un sujet, un programme d’ordinateur et un support lisible par ordinateur
CN113951897B (zh) * 2021-10-09 2023-05-05 中国科学院自动化研究所 一种多模态静息脑电数据干扰消除和标记方法及装置
WO2023228131A1 (fr) * 2022-05-25 2023-11-30 Sens.Ai Inc Procédé et appareil pour dispositif à porter sur soi avec interface synchronisée par temporisation pour tests cognitifs
IL294216A (en) * 2022-06-22 2024-01-01 Mor Research Applic Ltd Devices, systems and methods for brain protection in craniectomy patients
HUP2200356A1 (hu) * 2022-09-06 2024-03-28 Kiserleti Orvostudomanyi Ki Rendszer és eljárások biológiai adatok valós idõben milliszekundumos idõbeli pontossággal történõ szinkronizációjára
CN115810138B (zh) * 2022-11-18 2024-07-23 天津大学 基于多电极阵列体外培养神经元网络的图像识别方法

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150011857A1 (en) * 2012-02-23 2015-01-08 Bio-Signal Group Corp. Shielded multi-channel eeg headset systems and methods

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4967038A (en) 1986-12-16 1990-10-30 Sam Techology Inc. Dry electrode brain wave recording system
US5038782A (en) 1986-12-16 1991-08-13 Sam Technology, Inc. Electrode system for brain wave detection
AU667199B2 (en) 1991-11-08 1996-03-14 Physiometrix, Inc. EEG headpiece with disposable electrodes and apparatus and system and method for use therewith
US6301493B1 (en) * 1999-07-10 2001-10-09 Physiometrix, Inc. Reservoir electrodes for electroencephalograph headgear appliance
KR20020068825A (ko) * 2001-02-23 2002-08-28 학교법인 한양학원 가상 현실과 바이오 피드백을 이용한 집중력 향상 시스템및 방법
KR20020069382A (ko) * 2001-02-26 2002-09-04 학교법인 한양학원 바이오피드백 센서가 부착된 가상 현실 영상 제시 장치
WO2006121469A1 (fr) * 2005-05-06 2006-11-16 The General Hospital Corporation Appareils et procedes d'apport et d'enregistrement de signaux electrophysiologiques pendant une irm
EP3087918B1 (fr) * 2007-11-06 2018-08-22 Bio-signal Group Corp. Dispositif pour réaliser une électroencéphalographie
US8948849B2 (en) * 2008-04-28 2015-02-03 The Trustees Of Dartmouth College System and method for optode and electrode positioning cap for electroencephalography, diffuse optical imaging, and functional neuroimaging
US20110054870A1 (en) 2009-09-02 2011-03-03 Honda Motor Co., Ltd. Vision Based Human Activity Recognition and Monitoring System for Guided Virtual Rehabilitation
JP5677440B2 (ja) * 2009-09-25 2015-02-25 ニューロントリックス・ソリューションズ・エルエルシーＮｅｕｒｏｎｅｔｒｉｘ Ｓｏｌｕｔｉｏｎｓ， ＬＬＣ リジッド‐フレックス回路を備えた電極システム
WO2011062591A1 (fr) * 2009-11-21 2011-05-26 Douglas Peter Magyari Dispositif de type visiocasque
SG184333A1 (en) 2010-03-31 2012-11-29 Agency Science Tech & Res Brain- computer interface system and method
US20110282231A1 (en) * 2010-05-12 2011-11-17 Neurofocus, Inc. Mechanisms for collecting electroencephalography data
US9993190B2 (en) 2011-08-16 2018-06-12 Intendu Ltd. System and method for neurocognitive training and/or neuropsychological assessment
GB2499595A (en) * 2012-02-21 2013-08-28 James Roche Infant EEG electrode system
CN103961090B (zh) * 2013-01-31 2016-03-30 香港理工大学 一种可头部安装并具有脑电图用电极的装置
US9622702B2 (en) * 2014-04-03 2017-04-18 The Nielsen Company (Us), Llc Methods and apparatus to gather and analyze electroencephalographic data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150011857A1 (en) * 2012-02-23 2015-01-08 Bio-Signal Group Corp. Shielded multi-channel eeg headset systems and methods

Cited By (107)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11673042B2 (en) 2012-06-27 2023-06-13 Vincent John Macri Digital anatomical virtual extremities for pre-training physical movement
US11904101B2 (en) 2012-06-27 2024-02-20 Vincent John Macri Digital virtual limb and body interaction
US11804148B2 (en) 2012-06-27 2023-10-31 Vincent John Macri Methods and apparatuses for pre-action gaming
US10950336B2 (en) 2013-05-17 2021-03-16 Vincent J. Macri System and method for pre-action training and control
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
US11116441B2 (en) 2014-01-13 2021-09-14 Vincent John Macri Apparatus, method, and system for pre-action therapy
US11944446B2 (en) 2014-01-13 2024-04-02 Vincent John Macri Apparatus, method, and system for pre-action therapy
US10317987B2 (en) * 2015-11-05 2019-06-11 Boe Technology Group Co., Ltd. Wearable device and application system
US11081015B2 (en) * 2016-02-23 2021-08-03 Seiko Epson Corporation Training device, training method, and program
US20170243499A1 (en) * 2016-02-23 2017-08-24 Seiko Epson Corporation Training device, training method, and program
US11484243B2 (en) * 2016-08-12 2022-11-01 Nuro Corp. Portable Alzheimer detector
US11701046B2 (en) * 2016-11-02 2023-07-18 Northeastern University Portable brain and vision diagnostic and therapeutic system
US10353482B2 (en) * 2016-12-06 2019-07-16 Vuelosophy Inc. Systems and methods for tracking motion and gesture of heads and eyes
US20180157332A1 (en) * 2016-12-06 2018-06-07 Vuelosophy Inc. Systems and Methods for Tracking Motion and Gesture of Heads and Eyes
US11602300B2 (en) * 2016-12-20 2023-03-14 South China University Of Technology Brain-computer interface based robotic arm self-assisting system and method
US20190387995A1 (en) * 2016-12-20 2019-12-26 South China University Of Technology Brain-Computer Interface Based Robotic Arm Self-Assisting System and Method
US20210128904A1 (en) * 2017-01-18 2021-05-06 Viktor S.R.L. Electrostimulation apparatus
US11872388B2 (en) * 2017-01-18 2024-01-16 Viktor S.R.L. Electrostimulation apparatus
US11402909B2 (en) 2017-04-26 2022-08-02 Cognixion Brain computer interface for augmented reality
US10579141B2 (en) * 2017-07-17 2020-03-03 North Inc. Dynamic calibration methods for eye tracking systems of wearable heads-up displays
US20190064924A1 (en) * 2017-08-30 2019-02-28 Disney Enterprises, Inc. Systems And Methods To Synchronize Visual Effects and Haptic Feedback For Interactive Experiences
US10444840B2 (en) * 2017-08-30 2019-10-15 Disney Enterprises, Inc. Systems and methods to synchronize visual effects and haptic feedback for interactive experiences
US11207491B2 (en) * 2017-09-15 2021-12-28 Christoph Guger Device and method for the learning of the deliberate control of a specified body part by a test subject
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
US11036288B2 (en) * 2018-04-23 2021-06-15 Beijing Boe Optoelectronics Technology Co., Ltd. Head-mounted virtual reality display device, method for measuring position and posture of the same and virtual reality display apparatus
US10598936B1 (en) * 2018-04-23 2020-03-24 Facebook Technologies, Llc Multi-mode active pixel sensor
US10564717B1 (en) * 2018-07-16 2020-02-18 Facebook Technologies, Llc Apparatus, systems, and methods for sensing biopotential signals
US11589749B2 (en) * 2018-07-23 2023-02-28 Northeastern University Optically monitoring brain activities using 3D-aware head-probe
CN109009173A (zh) * 2018-08-30 2018-12-18 北京机械设备研究所 一种基于脑电-眼动双模态信号的疲劳检测与调控方法
US11475987B2 (en) 2018-09-12 2022-10-18 California Institute Of Technology Wearable inductive damping sensor
US11567573B2 (en) 2018-09-20 2023-01-31 Meta Platforms Technologies, Llc Neuromuscular text entry, writing and drawing in augmented reality systems
WO2020092420A1 (fr) * 2018-10-29 2020-05-07 Cognixion Interface cerveau-ordinateur de réalité augmentée
US10990175B2 (en) * 2018-10-29 2021-04-27 Cognixion Brain computer interface for augmented reality
CN113227944A (zh) * 2018-10-29 2021-08-06 康尼克逊 用于增强现实的脑机接口
EP3874351A4 (fr) * 2018-10-29 2022-07-27 Cognixion Interface cerveau-ordinateur de réalité augmentée
US20200133393A1 (en) * 2018-10-29 2020-04-30 Cognixion Brain computer interface for augmented reality
JP2022509752A (ja) * 2018-10-29 2022-01-24 コグニクソン 拡張現実のための脳コンピュータ・インターフェース
AU2019369395B2 (en) * 2018-10-29 2024-03-28 Cognixion Brain computer interface for augmented reality
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11941176B1 (en) 2018-11-27 2024-03-26 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
KR102020598B1 (ko) * 2018-12-11 2019-09-10 전자부품연구원 생체신호 센서 기반 정신질환 진단 및 치유를 위한 바이오피드백 시스템
KR102238067B1 (ko) 2018-12-12 2021-04-08 광운대학교 산학협력단 3d 헤드업 디스플레이와 연동된 웨어러블 eeg 헤드셋을 구비하는 eeg 측정 및 자극 시스템 및 방법
KR20200072099A (ko) * 2018-12-12 2020-06-22 광운대학교 산학협력단 3d 헤드업 디스플레이와 연동된 웨어러블 eeg 헤드셋을 구비하는 eeg 측정 및 자극 시스템 및 방법
US11642081B2 (en) 2019-02-01 2023-05-09 X Development Llc Electrode headset
US11583231B2 (en) 2019-03-06 2023-02-21 X Development Llc Adjustable electrode headset
US11481030B2 (en) 2019-03-29 2022-10-25 Meta Platforms Technologies, Llc Methods and apparatus for gesture detection and classification
US11327566B2 (en) * 2019-03-29 2022-05-10 Facebook Technologies, Llc Methods and apparatuses for low latency body state prediction based on neuromuscular data
US11961494B1 (en) 2019-03-29 2024-04-16 Meta Platforms Technologies, Llc Electromagnetic interference reduction in extended reality environments
US11481031B1 (en) 2019-04-30 2022-10-25 Meta Platforms Technologies, Llc Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
WO2020227394A1 (fr) * 2019-05-07 2020-11-12 The Trustees Of Princeton University Système et procédé pour capteur d'oxygène exhalé par courant principal
US11493993B2 (en) 2019-09-04 2022-11-08 Meta Platforms Technologies, Llc Systems, methods, and interfaces for performing inputs based on neuromuscular control
US20240298943A1 (en) * 2019-11-15 2024-09-12 Looxid Labs Inc. Biosignal sensing device
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
WO2021107329A1 (fr) * 2019-11-28 2021-06-03 서울대학교산학협력단 Procédé et appareil d'évaluation d'intégration visuelle et tactile utilisant la réalité virtuelle
CN114173663A (zh) * 2020-02-07 2022-03-11 Vibraint公司 神经康复系统及神经康复方法
EP4000578A4 (fr) * 2020-02-07 2023-08-02 Vibraint Inc. Système de réhabilitation neurologique et procédé de réhabilitation neurologique
US11890097B2 (en) 2020-02-19 2024-02-06 California Institute Of Technology Inductive damping brain sensor
WO2021168099A1 (fr) * 2020-02-19 2021-08-26 California Institute Of Technology Capteur cérébral à amortissement inductif
US12029566B2 (en) 2020-02-19 2024-07-09 California Institute Of Technology Methods of using an inductive damping brain sensor
US20210302749A1 (en) * 2020-03-27 2021-09-30 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US11874473B2 (en) 2020-03-27 2024-01-16 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US11598967B2 (en) 2020-03-27 2023-03-07 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US11994685B2 (en) 2020-03-27 2024-05-28 ResMed Pty Ltd Positioning, stabilising, and interfacing structures and system incorporating same
US20210302748A1 (en) * 2020-03-27 2021-09-30 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US11243405B2 (en) * 2020-03-27 2022-02-08 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US11686948B2 (en) 2020-03-27 2023-06-27 ResMed Pty Ltd Positioning, stabilising, and interfacing structures and system incorporating same
US11262589B2 (en) * 2020-03-27 2022-03-01 ResMed Pty Ltd Positioning and stabilising structure and system incorporating same
US20210392243A1 (en) * 2020-06-10 2021-12-16 Sam Salemnia Head mountable camera system
CN111831118A (zh) * 2020-07-10 2020-10-27 清华大学 一种基于增强现实的三维脑电显示方法及其显示系统
WO2022055948A1 (fr) * 2020-09-08 2022-03-17 Beacon Biosignals, Inc. Systèmes et méthodes de mesure de la neurotoxicité chez un sujet
WO2022061322A1 (fr) * 2020-09-18 2022-03-24 University Of Florida Research Foundation Élimination de composantes de bruit latent à partir de signaux de données
US11609633B2 (en) 2020-12-15 2023-03-21 Neurable, Inc. Monitoring of biometric data to determine mental states and input commands
WO2022132670A1 (fr) * 2020-12-15 2022-06-23 Neurable, Inc. Surveillance de données biométriques afin de déterminer des états mentaux et commandes d'entrée
US20220206576A1 (en) * 2020-12-30 2022-06-30 Imagine Technologies, Inc. Wearable electroencephalography sensor and device control methods using same
US11816266B2 (en) 2020-12-30 2023-11-14 Imagine Technologies, Inc. Method of developing a database of controllable objects in an environment
US11461991B2 (en) 2020-12-30 2022-10-04 Imagine Technologies, Inc. Method of developing a database of controllable objects in an environment
US11500463B2 (en) * 2020-12-30 2022-11-15 Imagine Technologies, Inc. Wearable electroencephalography sensor and device control methods using same
CN112751882A (zh) * 2021-01-19 2021-05-04 华南理工大学 一种基于混合脑机接口的实时通讯方法
US11199903B1 (en) * 2021-03-26 2021-12-14 The Florida International University Board Of Trustees Systems and methods for providing haptic feedback when interacting with virtual objects
US11402904B1 (en) * 2021-03-26 2022-08-02 The Florida International University Board Of Trustees Systems and methods for providing haptic feedback when interacting with virtual objects
WO2022204726A1 (fr) * 2021-03-26 2022-09-29 Wave Neuroscience, Inc. Électroencéphalogramme (eeg) basé sur des thérapies de stimulation magnétique transcrânienne (tms)
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
CN113456080A (zh) * 2021-05-25 2021-10-01 北京机械设备研究所 一种干湿通用型传感电极及其应用方法
US20220391014A1 (en) * 2021-06-08 2022-12-08 Honda Motor Co., Ltd. Operation system and operation method
US11640203B2 (en) * 2021-06-08 2023-05-02 Honda Motor Co., Ltd. Operation system and operation method
US20240188902A1 (en) * 2021-06-21 2024-06-13 Virtual Reality Concussion Assessment Corporation System for concussion assessment
US11744504B2 (en) * 2021-08-05 2023-09-05 Zeto, Inc. Flexible electroencephalography headset
US20240099644A1 (en) * 2021-08-05 2024-03-28 Zeto, Inc. Flexible electroencephalography headset
US11399762B1 (en) 2021-08-05 2022-08-02 Starcat LLC Modular electroencephalograph (EEG) system
US20230043938A1 (en) * 2021-08-05 2023-02-09 Zeto, Inc. Flexible electroencephalography headset
WO2023014357A1 (fr) * 2021-08-05 2023-02-09 Starcat LLC Système d'électroencéphalographe (eeg) modulaire
WO2023027880A1 (fr) * 2021-08-25 2023-03-02 Battelle Memorial Institute Stimulation électrique neuromusculaire commandée par la vision artificielle
US20230095350A1 (en) * 2021-09-17 2023-03-30 Smart Science Technology, LLC Focus group apparatus and system
US11995233B2 (en) * 2021-10-21 2024-05-28 Sony Interactive Entertainment LLC Biometric feedback captured during viewing of displayed content
US20230129718A1 (en) * 2021-10-21 2023-04-27 Sony Interactive Entertainment LLC Biometric feedback captured during viewing of displayed content
CN114010197A (zh) * 2021-11-05 2022-02-08 哈工大机器人（合肥）国际创新研究院 一种基于空间树状网络结构的干湿混合脑电电极
WO2023112613A1 (fr) * 2021-12-15 2023-06-22 住友ベークライト株式会社 Dispositif de mesure d'onde cérébrale et procédé de mesure d'onde cérébrale
JP7327708B1 (ja) * 2021-12-15 2023-08-16 住友ベークライト株式会社 脳波測定装置および脳波測定方法
US11759136B2 (en) * 2022-01-10 2023-09-19 Yewon SONG Apparatus and method for generating 1:1 emotion-tailored cognitive behavioral therapy in meta verse space through artificial intelligence control module for emotion-tailored cognitive behavioral therapy
US20230218215A1 (en) * 2022-01-10 2023-07-13 Yewon SONG Apparatus and method for generating 1:1 emotion-tailored cognitive behavioral therapy in metaverse space through artificial intelligence control module for emotion-tailored cognitive behavioral therapy
WO2023177946A1 (fr) * 2022-03-18 2023-09-21 Apple Inc. Structures de montage flexibles
US11944870B2 (en) * 2022-03-31 2024-04-02 bOMDIC Inc. Movement determination method, movement determination device and computer-readable storage medium
US20230310934A1 (en) * 2022-03-31 2023-10-05 bOMDIC Inc. Movement determination method, movement determination device and computer-readable storage medium
WO2023239361A1 (fr) * 2022-06-08 2023-12-14 Hewlett-Packard Development Company, L.P. Angle prévu basé sur des valeurs d'emg du muscle
CN115919321A (zh) * 2022-12-29 2023-04-07 中电云脑(天津)科技有限公司 一种便携式无创神经电生理信号采集计算装置

Also Published As

Publication number Publication date
US12056280B2 (en) 2024-08-06
CN107405101A (zh) 2017-11-28
EP3264977A2 (fr) 2018-01-10
EP3064130A1 (fr) 2016-09-07
WO2016139576A2 (fr) 2016-09-09
WO2016139576A3 (fr) 2016-10-27
EP3264977B1 (fr) 2021-12-29
US20210208680A1 (en) 2021-07-08
ES2910273T3 (es) 2022-05-12

Similar Documents

Publication Publication Date Title
US12056280B2 (en) Brain activity measurement and feedback system
US20160235323A1 (en) Physiological parameter measurement and feedback system
US20190286234A1 (en) System and method for synchronized neural marketing in a virtual environment
US11534605B2 (en) Neural sleeve for neuromuscular stimulation, sensing and recording
US20240207605A1 (en) Neural sleeve for neuromuscular stimulation, sensing and recording
AU2018250348B2 (en) Modular physiologic monitoring systems, kits, and methods
Reis et al. Methodological aspects of EEG and body dynamics measurements during motion
Fifer et al. Simultaneous neural control of simple reaching and grasping with the modular prosthetic limb using intracranial EEG
US8938301B2 (en) Headgear with displaceable sensors for electrophysiology measurement and training
AU2014225626B2 (en) Form factors for the multi-modal physiological assessment of brain health
KR20190041467A (ko) 신체 조직 전기 신호의 검출 및 사용
US11543879B2 (en) System for communicating sensory information with an interactive system and methods thereof
Sethi et al. Advances in motion and electromyography based wearable technology for upper extremity function rehabilitation: A review
JP6145663B2 (ja) 生体信号計測システム、および生体信号計測方法
Scherer et al. Non-manual Control Devices: Direct Brain-Computer Interaction
Rihana Begum et al. Making Hospital Environment Friendly for People: A Concept of HMI
SIONG Training and assessment of hand-eye coordination with electroencephalography
Simanski et al. Current developments in automatic drug delivery in anesthesia
Contreras-Vidal et al. Design principles for noninvasive brain-machine interfaces
Lee et al. Biosignal-integrated robotic systems with emerging trends in visual interfaces: A systematic review
Farzan Brain-computer interface
Mishra Brain computer interface based neurorehabilitation technique using a commercially available EEG headset
Lee et al. Training and measuring the hand-eye coordination capability of mentally ill patients
Schombs Low Cost Devices for Research in Brain-Computer Interfaces

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: MINDMAZE HOLDING SA, FRANCE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BOURDAUD, NICHOLAS, MR.;REEL/FRAME:067606/0743

Effective date: 20211014