WO2024044242A1 - Procédé d'apprentissage et de quantification de compétences motrices spécifiques et de processus cognitifs chez des personnes par analyse de motifs oculomoteurs w au moyen d'un dispositif de réalité virtuelle 3d à technologie de suivi oculaire intégrée, stimuli visuels spécifiques, capteurs pour montrer le mouvement de membres - Google Patents

Procédé d'apprentissage et de quantification de compétences motrices spécifiques et de processus cognitifs chez des personnes par analyse de motifs oculomoteurs w au moyen d'un dispositif de réalité virtuelle 3d à technologie de suivi oculaire intégrée, stimuli visuels spécifiques, capteurs pour montrer le mouvement de membres Download PDF

Info

Publication number
WO2024044242A1
WO2024044242A1 PCT/US2023/030917 US2023030917W WO2024044242A1 WO 2024044242 A1 WO2024044242 A1 WO 2024044242A1 US 2023030917 W US2023030917 W US 2023030917W WO 2024044242 A1 WO2024044242 A1 WO 2024044242A1
Authority
WO
WIPO (PCT)
Prior art keywords
person
subject
eye
objects
compromise
Prior art date
Application number
PCT/US2023/030917
Other languages
English (en)
Inventor
Gerardo ABEL FERNANDEZ
Original Assignee
Viewmind, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US18/217,688 external-priority patent/US20230352175A1/en
Priority claimed from US18/227,577 external-priority patent/US20230368919A1/en
Application filed by Viewmind, Inc. filed Critical Viewmind, Inc.
Publication of WO2024044242A1 publication Critical patent/WO2024044242A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/40—Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4076—Diagnosing or monitoring particular conditions of the nervous system
    • A61B5/4082—Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/486—Bio-feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/10—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to drugs or medications, e.g. for ensuring correct administration to patients
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/30—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement

Definitions

  • Eye tracking systems have been used as a diagnostic took For instance,, co-pending U.S. Appl. Serial No. 18/217,688 [Docket No. 9901/lcl], which is incorporated by reference herein in its entirety, shows a system for detecting one or more neurological disorders in a subject by measuring eye movements. Examples of neurologicaldisorders that may be detected include Multiple sclerosis (MS), attention dcficit-hypcractivcdisorder (ADHD), Parkinson disorder (PD), Alzheimer disease (AD), etc.
  • MS Multiple sclerosis
  • ADHD attention dcficit-hypcractivcdisorder
  • PD Parkinson disorder
  • AD Alzheimer disease
  • VR virtual Reality
  • Human cognitive ability can be roughly classified into working memory, attention, perception, reasoning and judgment, decision-making and so on.
  • VR studies are mostly performed by collecting partial data related to behavioral feedback e.g., What is the number of correct responses when performing an activity? or what is the total time needed to complete a task?
  • a processor [20] configured to receive data from the eye tracker [10] while the subject [5] is reading the text [15]; and c. a display means [40] configured to display a test report [50] received from the processor [20] ;wherein the processor [20] is further configured to analyze the eyetracking data for evidence of one or more neurological disorders or general cognitive performance and to report, in the test report [50], a detection of one or more neurological disorders orameasureof cognitive performance of the subject [5],
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. count a total number of ocular fixations of a subject while reading the text; and b. if the total number of ocular fixations of a subject when reading is higher than for a control group, then report in the test report that a compromise in attentional processes isdetected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. count a number of forward ocular fixations of the subject while reading the text; and b. if the number of forward ocular fixations of the subj ect is lower than for the controlgroup; and the number of ocular fixations of a subject when reading is higher than for the control group, then report in the test report [50] that a compromise in working memory is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. count a number of words that the subj ect fixated on only once while reading the text; and b. if the number of words that the subject fixated on only once is lower than for the control group, then report in the test report that a compromise in retrieval memory is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. count a number of multiple ocular fixations of the subject while reading the text; and b. if the number of multiple ocular fixations is higher than for the control group, thenreport in the test report that a compromise in executive processes is detected.
  • a display means [40] configured to display the output of the intelligentalgorithm on a test report [50] received from the processor [20]; wherein the processor [20] is further configured to analyze and modeling the eye-trackingdata for evidence of one or more neurological disorders and from cognitive performance and to report, in the test report [50], a detection and classification of the one or more neurological disorders of the subject [5] both, between and within pathologies.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to identify and classifying eye movement features and pupil behavior during reading the text providing an output of the classifier for reporting in the test report a subject’s cognitiveperformance and/or pathological classification (i.e, the pathology that correspond to the subject because his/her eye movement features); and a value within the pathology (i.e., thelevel of cognitive, behavioral and biological compromise that the subject shows within a particular pathology).
  • cognitiveperformance and/or pathological classification i.e, the pathology that correspond to the subject because his/her eye movement features
  • a value within the pathology i.e., thelevel of cognitive, behavioral and biological compromise that the subject shows within a particular pathology.
  • the intelligent algorithm is configured to read at least one input, the input selected from agroupconsisting of: a. Index of total number of ocular fixations of a subject while reading the text. b. Index of forward ocular fixations of the subject while reading the text. c. Index of words that the subject fixated on only once while reading the text d. Index of multiple ocular fixations of the subject while reading the text e. Average saccade amplitude from one ocular fixation to a next ocular fixation f. Pupil diameter of the subject reading the text g. Index of blinks coming from the left eye, the right eye or from both eyes. h.
  • Microsaccades' Factors of Form i. HEWI: shows the micro-saccade's height/width relationship.
  • ii AREA shows the area of the rectangle in which the micro-saccade is inscribed.
  • iii.LONG is the longitude of the horizontal-vertical plane trajectory of the microsaccade.
  • iv.ANG is the sum of all the angles in the plane horizontal - vertical plane of the microsaccade.
  • v. AANG is the sum of all the absolute values of angles in radians in the plane horizontal - vertical plane of the micro-saccade.
  • MOD and THETA are the modulus and the angle of the polar coordinates of the sum of the cartesian coordinates. They give a spatial orientation of the micro-saccade relative to the median of the fixation.
  • TIME is the time duration in milliseconds of the micro-saccade.
  • VMIN and VMAX are the minimum and maximum velocities of the microsaccades in degrees per second.
  • Micro-saccade rate is the instantaneous rate in each time bin.
  • Directional congruency is the congruency between the micro-saccade direction ant the location of the stimulus. i.
  • Eye position coming from the left eye, the right eye or from both eyes i.e., abscissaand ordinate coordinate
  • Fixation sequence i.e. , ocular behavior
  • the sequence willbe available from images, from matrices, etc.
  • l. Filia information of the subject (i.e., age; years of education; sex; ethnic group; occupation; hours per week of physical activity).
  • m Total reading time (i.e., the time that the subject spent when reading the text).
  • WM effect is a measure that increases when WM demand increases. For each cue number, the WM effect is represented by the ratio between the number of errors reported by the subject through all the trials, and the number of trials); and ii.
  • the method further comprises additional steps comprising measurementsperformed during the step of presenting a stimulus image [315], during which the subjectis further requested to look at the stimulus image; the measurements comprising measuring one or more of i.an amplitude of pupillary dilatation of the subject [360]; ii a number of fixations made by the subject on the stimulus image [365]; and iii.a gaze duration by the subject on the stimulus image [370]; and k. the additional steps further comprising calculating and reporting one or more of i.a degree of compromise of subcortical processes [375], with increased the amplitude of pupillary dilatation; ii.a degree of compromise of executive processes [380], with increased the number of fixations; and in. a degree of compromise of executive processes and working memory [385], with increased the gaze duration.
  • a display means [40] configured to display a test report [50] received from the processor [20]; wherein the processor [20] is further configured to analyze the eye-tracking data for evidence of neurological and attentional disorders and to report, in the test report [50], a detection of the one or more neurological and attentional disorder of the subject [5],
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker while the subject is visualizing, recognizing, maintaining, controlling, inhibiting andsequencing targets, to: a. count a total number of ocular fixations of a subject [615] while performing the visual test; and b. if the total number of ocular fixations of a subj ect when visualizing targets is higherthan for a control group, then report in the test report that a compromise in attentional processes is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. measure the saccade average speed [620] while the subject is shifting from one target to the other; and b. if the saccade average speed [620] of the subj ect is lower than for the control group;then report in the test report [50] that a compromise in executive functions is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. count a number of correct target recognitions [625]; and b. if the number of correct target recognitions [625] that the subject is lower than for the control group, then report in the test report that a compromise in working memory is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. compute an average saccade amplitude [630]; and b. if the average saccade amplitude [630] is lower than for the control group, then report in the test report that a compromise in executive processes is detected.
  • the processor is further configured, upon receiving the eye-tracking data from the eye tracker, to: a. the total time spent to perform the visual test [635], and b. if the total time spent to perform the visual test [635] is higher than for the controlgroup, then report in the test report that a compromise in attentional processes is detected.
  • a system [100] for detecting adisorder of memory binding function of a subject comprising: a. an eye tracker [10]; b. a means for measuring pupil diameters; c. a processor [20], configured to: i. receive eye-tracking data of a subject [5] from the eye tracker [10]; ii. receive pupil diameter data of the subject [5] from the means for measuring pupildiameters; and d.
  • a display means [40] configured to display a test report [50] received from theprocessor [20] ; wherein the processor [20] is further configured to analyze the eye-tracking and pupil diameter data and to report, in the test report [50], a detection of one or more disorders of memory binding function of the subject [5],
  • the processor [20] is further configured, upon receiving the eye-tracking data from the eye tracker [10], to: a. measure one or more gaze durations of the subj ect [5] on each of one or more targetsviewed by the subject [5]; b. calculate an average gaze duration of the targets by the subject [5]; and c. report in the test report [50] that a compromise in encoding and recognition of targets is detected in the subj ect [5] , if the average gaze duration of the subj ect [5] is longerthan the average gaze duration of a control group.
  • the processor [20] is further configured, upon receiving the eye-tracking data from the eye tracker [10], to: a. count a number of ocular fixations performed by the subject [5] while viewing oneor more targets; and b. report in the test report [50] that a compromise in attentional processes is detectedin the subject [5], if the number of ocular fixations performed by the subject [5] while viewing the targets is higher than for a control group.
  • the processor [20] is further configured to applying an intelligent algorithm and to: a. receive a pupil diameter of the subject [5] from the means for measuring pupil diameter, while the subject [5] performs activities requiring lower cognitive effort; b. receive a pupil diameter of the subject [5] from the means for measuring pupil diameter, while the subject [5] performs activities requiring a stronger cognitive effort; and c. report in the test report [50] that a compromise in cognitive resources is detected inthe subject [5], if the pupil diameter of the subject [5], while performing the activities requiring the stronger cognitive effort, does not show an increase over the pupil diameter of the subject [5] while performing the activities requiring reduced/minimal cognitive effort.
  • a compromise in a target encoding and recognition process is detected in thesubject, if the average gaze duration of the subject is longer than an average gaze durationof a control group [565]; ii. reporting that a compromise in cognitive resources is detected in the subject, if the pupil diameter of the subject while performing the activities requiring a stronger cognitive effortdoes not show an increase over the pupil diameter of the subject while performing the activities requiring lower cognitive effort [570]; and iii. reporting that a compromise in attentional processes is detected in the subj ect, if thenumberof ocular fixations performed by the subject while viewing the targets is higher than for a control group [575],
  • the intelligent algorithm is configured to read at least one input, the input selected from agroupconsisting of: a. Total number of ocular fixations of a subj ect while performing each BindingTask. b. Binding Evaluation Task, i.e. “Bound Colors” of “Unbound Colors”. c. Identification Number of Binding Trial. d. The Correct Behavioral Answer of the trial (i.e., if "same” or “different”). e. Subject's Behavioral response. f. Part of the Trial i.e., encoding or retrieval. g. Pupil diameter of the subject while performing while performing the BindingEvaluation.
  • h Number of blinks coming from the left eye, the right eye or from both eyes.
  • FF Factors of Form
  • HEWI shows the microsaccade’s height / width relationship.
  • AREA shows the area of the rectangle in which the microsaccade is inscribed.
  • LONG is the longitude of the horizontal -vertical plane trajectory of the microsaccade.
  • v.ANG is the sum of all the angles in the plane horizontal - vertical plane of the microsaccade.
  • v AANG is the sum of all the absolute values of angles in radians in the plane horizontal -vertical plane of the microsaccade.
  • a display means [40] configured to display a test report [50] received from the processor[20]; wherein the processor [20] is further configured to analyze the eye-tracking and pupil diameter data and to report, in the test report [50], a detection of one or more neurologicaland attentional disorders of the subject [5],
  • the processor [20] is further configured, upon receiving the eye-tracking data from the eye tracker [10], to: a. measure one or more fixation durations of the subject [5] on each of one or more targets viewed by the subject [5]; b. calculate an average saccade amplitude from each target to the other one by the subject [5]; and c. report in the test report [50] that a compromise in visualizing, recognizing, maintaining, controlling, inhibiting and sequencing of targets is detected in the subject [5], if the average saccade amplitude of the subject [5] is shorter than the average saccade amplitude of a control group.
  • the processor [20] is further configured, upon receiving the eye-tracking data from the eye tracker [10], to: a. count a number of ocular fixations performed by the subject [5] while viewing oneor more targets; and b. report in the test report [50] that a compromise in attentional processes is detectedin the subject [5], if the number of ocular fixations performed by the subject [5] while viewing the targets is higher than for a control group.
  • the processor [20] is further configured to: a.
  • the method further comprises steps of: i. reporting that a compromise in a target visualizing, recognizing, maintaining, controlling, inhibiting and sequencing process is detected in the subject, if the average saccade amplitude of the subject is shorter than an average saccade amplitude of a control group; n. reporting that a compromise in cognitive and functional resources is detected in the subject, if the pupil diameter of the subject while performing the activities requiring a major attention does not show an increase over the pupil diameter of the subject while performingthe activities requiring minor atention; and reporting that a compromise in réelleonal processes is detected in the subj ect, if the numberof ocular fixations performed by the subject while viewing the targets is higher than for a control group, iii. reporting that a compromise in executive process is detected in the subject, if the average saccade latency (speed) of the subj ect is shorter than an average saccade latency of a controlgroup;
  • the neurological disorder is selected from the group consisting of Parkinson Disease orAtention Deficit Hyperactive Disorder.
  • the system includes a three-dimensional (3D) virtual reality device configured to establish a 3D virtual reality environment in which a plurality of virtual objects is presented to the person, the objects having at least one feature that differs from one another, the objects moving toward or away from the person with a defined speed, acceleration and direction.
  • the system also includes an eye-tracker configured to measure eye movements of the person while the person is viewing the virtual objects and performing requested tasks, the requested tasks including multiple requests requesting the person to virtually touch specified virtual objects each having one of the specified features.
  • One or more motion sensors are configured to measure limb movements of the person while the person performs the requested tasks.
  • a processor is configured to receive data from the 3D virtual reality device, the eyetracker and the one or more motion sensors while the person is performing the requested tasks and being further configured to (i) identify selected ones of the measured eye and limb movements that are related to the performance, motor skills and cognitive capabilities of the person; (ii) determine expected eye and limb movements of the person while the person is viewing the virtual objects and performing the requested tasks and comparing the expected eye and limb movements to the selected ones of the measured eye and limb movements to determine deviations therebetween; and (iii) evaluate the performance, motor skills and cognitive capabilities of the person based on the deviations.
  • Figure 1 and Figure 2 show systems for detecting one or more neurological disorders of a subject, according to some embodiments of the invention.
  • Figures 3 A and 3B show a method for evaluating compromises in neurological functions associated with MS, according to some embodiments of the invention.
  • Figures 4A and 4B show a method for detecting one or more neurological disordersof a reading subject, according to some embodiments of the invention.
  • Figures 5 shows a method for detecting a disorder of memory binding function, according to some embodiments of the invention.
  • Figure 5C shows the test results as per the evaluation method of 5A: Effect of binding task on gaze duration in control and in Alzheimer Disease (AD) patients during Encoding and Recognition moments.
  • the panel shows the partial effects of LMM (i.e., after removal of other fixed effects and variance components). Shaded areas denote 95% confidence intervals. Gaze duration is plotted on a log scale for correspondence with the LMM.
  • Figure 6A and 6B shows a method for detecting Parkinson Disorder and AttentionDeficit Hyperactive Disorder.
  • Figure 7 shows the impact of Dimethyl Fumarate on Saccade Amplitude, on a Multiple Sclerosis patient that has been taking the drug for 4 years.
  • Figure 8 is a flowchart showing a method for identifying specific alterations in subj ects with defined disease analyzing oculomotor patterns when using specific visual stimuli, where a specific drug or treatment would enhance visual processing, cognitive performance and related brain activities.
  • Figure 9 shows a conceptual illustration of a system used to evaluate a person’s performance by applying a 3-Dimension Virtual Reality (3DVR) environment in combination with an embedded eye-tracking technology (ET) and motions sensors to track the movement of limbs such as hands and feet while the person performs well-defined activities.
  • 3DVR 3-Dimension Virtual Reality
  • ET embedded eye-tracking technology
  • Figure 10 shows an example of how objects may be positioned throughout the virtual environment and how objects and the person’s hands may appear.
  • Figure 11 is a graphical representation of an example of the density of eye movements recorded from the right and left eye while the person is performing the tasks touching the requested objects, presented as a heat-map.
  • Figure 12 is a graphical representation of the density of motor movements recorded of right and left-hand movements while the person is performing the tasks touching the requested objects, presented as a heat-map.
  • FIG. 13 is flowchart describing one example of the methods described herein for evaluating performance, motor skills and cognitive capabilities of a person.
  • cognitive effort reflects the total amount of mental effort that a subjectneeds to perform a task.
  • lower cognitive effort refers to a reduction on working memory demands when performing a task.
  • the term “Microsaccades”, also known as “flicks”, are small saccades performed during the fixation periods. They are the largest and fastest of the fixational eye movements.
  • saccades are small saccades performed during the fixation periods. They are the largest and fastest of the fixational eye movements.
  • sacades relate to quick, simultaneous movement of both eyes between two or more phases of a fixation.
  • optical drift is the fixational eye movement characterized by a smoother, slower, roaming motion of the eye when fixed on an object.
  • OMTs optical micro tremors
  • stimulation image refers to a specific visual pattern ortargets presented to the subject in the display.
  • visual task or “visual test” refersto the activity that performs the subject while processing each stimulus image.
  • System [100] comprises an eye tracker [10], ameans for measuring a pupil diameter[17], a processor [20], and a display means [40],
  • Eye tracker [10] can be of any type known in the art; for example, an e e-attached tracker, an optical eye tracker, or an electrooculographic eye tracker.
  • Means for measuring pupil diameter [17] may comprise, for example, a camera configured to acquire an image of the eye and a processing unit for measuring the pupil diameter from the image.
  • means for measuring a pupil diameter [17] can comprise a display of the image with manual measurement made while viewing the display.
  • Eye tracker [10] and means for measuring a pupil diameter [17] are in communicative connection with processor [20],
  • the communicative connections can be of any form(s) known in the art, and can be either wired (e.g., USB, parallel port, or similarjor wireless (e.g. WiFi, Bluetooth, or similar).
  • Processor [20] receives and executes instructions stored in one or more memory media [60], such as RAM, CD/DVD, HDD, flash memory, and/or any suitable medium.
  • the instructions command processor [20] to: 1) receive eye-tracking data from eye trackerf 10]; 2) receive pupil diameter data from means [17] of measuring pupil diameter; 3) analyze the eye-tracking and pupil diameter data (further explained herein); 4) report in a test report 50, for display on display means [40], of a detection or non-detection of one ormore disorders of memory binding function in subj ect [5] .
  • Display means [40] can be a monitor, a screen of a mobile device such as a smartphone, a printout, or any suitable meansof displaying test report [50] .
  • Processor [20] may store in memory medium [60] any of thereceived eye-tracking data, intermediate results at any stage(s) of the analysis, and/or test report [50],
  • Neurological disorders detected by system [100] can include reading function, suchas a compromise in encoding and recognition of targets, a compromise in attentional processes, a compromise in cognitive resources, or any combination thereof.
  • the disorders detected can include Multiple sclerosis (MS), Attention deficit-hyperactive disorder (ADHD), Parkinson disorder (PD), Alzheimer disease (AD), etc.
  • MS Multiple sclerosis
  • ADHD Attention deficit-hyperactive disorder
  • PD Parkinson disorder
  • AD Alzheimer disease
  • processor [20] receives eye-tracking data from eye-tracker [10] while subject [5] views each of one or more targets [30], Processor [20] measures gaze durations of subject [5] on each target [30] viewed by subject [5], Processor [20] calculates an average gaze duration on each of the targets [30] by subject [5], If an averageof the gaze durations on targets [30] of subject [5] is longer than an average gaze duration for a control group, then processor [20] reports in test report [50] that a compromise in a target encoding and recognition process is detected in subject [5],
  • processor [20] additionally, or alternatively, counts anumberof ocular fixations performed by subject [5] while viewing each of the targets [30], If thenumber of ocular fixations performed by subject [5] while viewing the targets [30] is higherthan for a control group, then processor [20] reports in test report [50] that a compromise in the attentional processes is detected in subject [5],
  • processor [20] receives pupil diameter data from means [17] of measuring pupil diameter while subject [5] performs activities requiring lower cognitiveeffort. Processor [20] further receives pupil diameter data from means [17] of measuring pupil diameter while subject [5] performs activities requiring a stronger cognitive effort than for the activities requiring lower cognitive effort. If an average pupil diameter of subject 5 while performing the activities requiring the stronger cognitive effort does not show an increase over an average pupil diameter of subject [5] while performing the activities requiring lower cognitive effort, then processor [20] reports in test report [50] that a compromise in cognitive resources is detected in subject [5],
  • the control group may comprise a statistically representative crosssection in the same demographic sector as subject [5] (e.g., the same gender, race, national culture, age group, and/or other demographic features of subject [5]). Eyetracking data for the control group may be obtained by system [100] or otherwise gathered from previous research studies and/or clinical studies. Where the average gaze duration or number of ocular fixations of subject [5] is within a selected margin — about one standard deviation of a distribution of the corresponding figure for the control group — of the average figure for the control group, system [100] may treat the average gaze duration or number of ocular fixations of subject [5] as equal to the average corresponding figure for the control group.
  • eye tracking data received by processor [20] may be a series of eyeball positions measured by eye tracker [10], which processor [20] analyzes to find gaze durations and ocular fixations of subject [5], Alternatively, processor [20] may receivea series of pre-processed signals from eye tracker [10], each signaling a gaze duration or that an ocular fixation has occurred.
  • the signals may optionally be accompanied with metadata (e.g., eyeball position, time, and/or length of the ocular fixation).
  • Method [300] comprises steps of: a. providing a system for evaluating compromises in neurological functions associated with MS [305]; b. requesting a subject to fixate on a reference target of a chart [310]; c. for a number of repetitions, presenting a stimulus image in one of a plurality of zones on the chart to the subject [315]; the subject is requested to remember which zone each stimulus image appeared and in what order; d. presenting to the subject a cue corresponding to one of the presented stimulus images [320]; e.
  • WM effect is a measure that increases when WM demand increases. For each cue number, the WM effect is represented by the ratio between the number of errors reported by the subject through all the trials, and the number of trials); and ii.
  • the additional steps further comprising calculating and reporting one or more of: i.a degree of compromise of subcortical processes, with an unchanged amplitude on pupildilatation [375] ; ii. a degree of compromise of executive processes, with increased number of fixations [380]; and iii.a degree of compromise of executive processes and working memory , with increased gazeduration [385], [82]
  • the method employs an intelligent algorithm to analyze the subject, utilizing thefollowing variables: a. Total number of ocular fixations of a subject while performing the n- Back Task. b.
  • n-Back Task Trial (i.e. if there are 20 n-Back Tasks Trials, the 5th trial is identified with the number 5. The 20th trial is identified with thenumber 20 etc.)
  • Trial Part i.e., 1, 2 and 3.
  • Part of the Trial i.e., encoding; retrieval. e. Pupil diameter of the subject while performing n-Back Task.
  • f Number of blinks coming from the left eye, the right eye or from both eyes.
  • iii. LONG is the longitude of the horizontal- vertical plane trajectory of the microsaccade.
  • iv. ANG is the sum of all the angles in the plane horizontal - vertical planeof the microsaccade.
  • v. AANG is the sum of all the absolute values of angles in radians in the plane horizontal - vertical plane of the microsaccade.
  • VMIN and VMAX are the minimum and maximum velocities of the microsaccades indegrees per second.
  • Microsaccade rate is the instantaneous rate in each time bin.
  • Directional congruency is the congruency between the microsaccade direction and thelocation of the stimulus.
  • h Eye position coming from the left eye, the right eye or from both eyes (i.e., abscissa and ordinate coordinate) while performing the n-Back Task.
  • Fixation sequence i. e. , ocular behavior
  • Thesequence will be available from images, from matrices, etc. l.
  • m Filia information of the subject (i.e., age; years of education; sex; ethnic group; occupation; hours per week of physical activity).
  • n Fixation duration while processing targets.
  • o Gaze duration while processing targets.
  • p Number of fixations on each target.
  • q Number of fixations outside each target.
  • the measurements made while presenting the stimulus image (feature) in method[300]) provides information during encoding, which occurs while the subject identifies thelocation of the visual stimulus for the first time.
  • subjectswith MS were found to be impaired when encoding visual information (e.g., subjects mademany fixations on the display).
  • Measurements during encoding are in addition to the measurements taken during recognition, when presented with cues after the visual stimuliare presented as in the study of Fielding et al. (steps a-i in method [300]).
  • performance of the subject during both encoding and recognition can help identify additional deficiencies (namely, degrees of compromise of subcortical processes, executiveprocesses, and/or executive processes) and provide greater insight into the condition of thesubject than performance during recognition alone.
  • FIG. 4A and 4B showing a method for measuring general cognitive performance and for detecting one or more neurological disorders of a subject, by measuring eye movements and/or pupil diameter of the subject while the subjectis reading, according to some embodiments of the invention.
  • Method [400] comprises steps of providing a system for measuring general cognitive performance and for detecting the presence of one or more neurological disordersby measuring eye movements and/or pupil diameter; receiving eye-tracking data and/ or pupil diameter data of a subj ect reading a text; analyzing the eye-tracking data for evidenceof one or more neurological disorders; and displaying a report of detection of the neurological disorder(s).
  • method [400] comprises steps of counting a total number ofocular fixations of the subject while the subject is reading the text [405]; and reporting that a compromise in attentional processes is detected, if the total number of ocular fixations ofthe subject when reading the text is higher than for a control group [460],
  • method [400] further comprises steps of counting a total number of ocular fixations of the subject while reading the text [405]; counting a number of forward ocular fixations of the subject while reading the text [430] ; and reporting that acompromise in working memory is detected, if the number of forward ocular fixations of the subject is higher than for the control group and the number of total ocular fixations of the subject when reading is higher than for the control group [470],
  • a compromise in working memory is correlated with deteriorationin the frontal lobe.
  • reporting of a compromise in working memory is correlated with deteriorationin the frontal lobe.
  • [470] may be used in additional treatment. For example, if neurosurgery is indicated, method [400] may be followed by studying brain imagery of the subject’s frontal lobe.
  • method [400] comprises steps of counting numbers of ocularfixations by the subject on each word in the text while the subject is reading the text [440] ; counting a number of words that the subj ect fixated on only once [445] ; and reporting thata compromise in retrieval memory is detected, if the number of words that subject fixatedon only once is lower than for the control group [480],
  • a compromise in retrieval memory is correlated with deteriorationin the temporal lobe.
  • reporting of a compromise in retrieval memory is correlated with deteriorationin the temporal lobe.
  • [480] may be used in additional treatment. For example, if neurosurgery is indicated, method [400] may be followed by studying brain imagery of the subject’s frontal lobe.
  • method [400] comprises steps of counting a number of multiple ocular fixations of subject while reading the text [450]; and reporting that a compromise in executive processes is detected, if the number of multiple ocular fixations is higher than for the control group [490],
  • method [400] comprises steps of computing an average saccade amplitude of the subject from one ocular fixation to a next ocular fixation while reading the text [454]; and reporting that a compromise in executive processes is detected, if the average saccade amplitude is lower than for the control group [491],
  • method [400] comprises steps of tracking a pupil diameter of the subject while reading the text [456]; and reporting that a compromise in executive processes is detected, if the pupil diameter of the subject does not show a reduction as advancing in reading the text [492],
  • a compromise in executive processes is correlated with deterioration in the frontal, temporal, and/or parietal lobes.
  • reporting of a compromise in executive processes [490-491-492] may be used in additionaltreatment.
  • method [400] may be followed by studying brain imagery of the subject’s frontal, temporal, and/or parietal lobes.
  • Method comprises a step [505] of providing a system for detecting a disorder of memory binding function in a subject.
  • method [500] comprises a step [510-535] of viewing by a subject of one or more targets; a step [545] of measuring a gaze duration of the subject oneach of said targets; a step [550] of calculating an average gaze duration of the targets by the subject; and a step [565] of reporting that a compromise in a target encoding and recognition process is detected in the subject, if an average of the gaze durations of the subject is longer than an average gaze duration for a control group.
  • method [500] comprises a step [555] of measuring one or more pupil diameters of the subject while performing activities requiring lower cognitive effort (e.g., recognizing three targets or distinguishing between targets; and a step [570] ofreporting that a compromise in cognitive resources is detected in subject [5], if an averagepupil diameter of subject [5] while performing the activities requiring a stronger cognitiveeffort does not show an increase over an average pupil diameter of subject [5] while performing activities requiring lower cognitive effort.
  • a step [555] of measuring one or more pupil diameters of the subject while performing activities requiring lower cognitive effort e.g., recognizing three targets or distinguishing between targets
  • a step [570] ofreporting that a compromise in cognitive resources is detected in subject [5], if an averagepupil diameter of subject [5] while performing the activities requiring a stronger cognitiveeffort does not show an increase over an average pupil diameter of subject [5] while performing activities requiring lower cognitive effort.
  • method [500] comprises a step [560] of counting a numberof ocular fixations by subject [5] while viewing the targets [30]; and a step [575] of reporting that a compromise in attentional processes is detected in subject [5], if the numberof ocular fixations performed by subject [5] while viewing the targets [30] is higher than for the control group.
  • Parkinson Disease Parkinson Disease
  • ADHD Attentional Deficit Hyperactive Disorders
  • FIG. 6A and 6B showing a method for detecting oneor more cognitive, neurological and behavioral impairments of a person, by measuring eyemovements and/or pupil diameter of the person while the person is performing the visual test, according to some embodiments of the invention.
  • Method [600] comprises steps of providing a system for detecting the presence of one or more cognitive impairments and neurological disorders by measuring eye movements while a person is visualizing, recognizing, maintaining, controlling, inhibitingand sequencing targets; receiving eye-tracking data of a person visualizing, recognizing, maintaining, controlling, inhibiting and sequencing targets; analyzing the eye-tracking datafor evidence of one or more cognitive impairments and neurological disorders; anddisplaying a report of detection of the cognitive impairments and neurological disorder(s).
  • method [600] comprises steps of counting a total number ofocular fixations [615] of the person while the person is performing the visual test; and reporting that a compromise in attentional, executive and inhibitory processes is detected, if the number of ocular fixations of the person is higher than for a control group.
  • method [600] comprises steps for calculating the saccade average speed [620] of the subject [5] from one target to the other one, while the subj ect
  • [5] is performing the visual test; reporting that a compromise in executive functions isdetected, if the saccade average speed that person did is lower than for the control group.
  • Physiologically a slower saccade speed is correlated with deterioration in frontaleye fields, basal ganglia and superior colliculus. In some embodiments, reporting of acompromise in saccade speed may be used in additional treatment.
  • method [600] comprises steps of counting a number of correct target recognitions of person while performing the visual test [625]; and reporting that a compromise in working memory is detected, if the number of correct targetrecognitions is lower than for the control group.
  • a compromise in working memory is correlated with a deterioration in Prefrontal Cortex and in the Posterior Parietal Cortex.
  • reporting of a compromise in working memory, inhibition processes and mental flexibility may be used in additional treatment.
  • method [600] comprises steps of computing an average saccade amplitude from one ocular fixation to a next ocular fixation [630]; and reporting that a compromise in executive processes is detected, if the average saccade amplitude is lower than for the control group.
  • method [600] comprises steps of tracking a pupil diameter of the person while performing the visual test [640] ; and reporting that a compromise in attentional processes is detected, if the pupil diameter of the subject does not show an increase as advancing in performing the visual test.
  • method [600] comprises steps of computing the total time spent by the person while performing the visual trial [635]; and reporting that a compromise in attentional processes is detected, if the total time needed for performing thetrial is major that the reported for the control group.
  • a compromise in attentional and inhibitory processes and in mentalflexibility is correlated with deterioration in the prefrontal cortex, the posterior parietal cortex, the prefrontal striatal cerebellar and prefrontal striatal thalamic circuits.
  • reporting of a compromise in executive processes may be used in additional treatment.
  • method [600] comprises steps of calculating fixation durations on targets of person while performing the visual test [645]; and reporting that a compromise in working memory is detected, if the fixation duration on targets is lower than for the control group.
  • a compromise in attentional and inhibitory processes and in mentalflexibility is correlated with deterioration in the prefrontal cortex, the frontal eye fields andin the dorso-parietal cortex.
  • reporting of a compromise in executiveprocesses may be used in additional treatment.
  • the method employs an intelligent algorithm to analyze the subject, utilizing thefollowing variables: a. Total number of ocular fixations of a subject while performing the Visual Test. b. Identification Number of each target depending of its place in the labyrinth ormaze. c. Pupil diameter of the subject while performing the visual Test. d. Number of blinks coming from the left eye, the right eye or from both eyes. e. Microsaccades; Factors of Form (FF): i.HEWI: shows the microsacade’s height/width relationship. ii AREA: shows the area of the rectangle in which the microsaccade is inscribed, hi. LONG: is the longitude of the horizontal-vertical plane trajectory of the microsaccade.
  • FF Factors of Form
  • iv. ANG is the sum of all the angles in the plane horizontal - vertical planeof the microsaccade.
  • v. AANG is the sum of all the absolute values of angles in radians in the plane horizontal - vertical plane of the microsaccade.
  • VMIN and VMAX are the minimum and maximum velocities of the microsaccades indegrees per second ix.
  • Microsaccade rate is the instantaneous rate in each time bin.
  • Directional congruency is the congruency between the microsaccade direction and thelocation of the stimulus.
  • f. Eye position coming from the left eye, the right eye or from both eyes (i.e., abscissa and ordinate coordinate) while performing the visual Task.
  • Fixation sequence i.e., ocular behavior
  • Thesequence will be available from images, from matrices, etc. j .
  • k Filia information of the subject (i.e., age; years of education; sex; ethnic group;occupation; hours per week of physical activity).
  • l Fixation duration while processing targets.
  • m Number of fixations on each target.
  • n Number of fixations outside each target.
  • o Total visual Task time (i.e., how much time spent the subject for performing theentire trial).
  • the medical practitioner and/or a pharmaceutical manufacturer can better track the effectiveness of the treatment regimen on the patient and alter or supplement the regimen asnecessary based on that evaluation throughout the course of the disease.
  • drugs or other medicaments that may be evaluated include neurological and/or psychiatric drugs that have a neurological and/or psychiatric effect.
  • Some embodiments of the methods described herein may perform one or more of thcfollowing: calculating, modelling and reporting one or more effects of drugs (e.g., Dimethyl fumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta) or treatments in order to test if there is (a) a decrease on the inflammation and nerve damage that can cause symptoms of multiple sclerosis; (b) a damage on the receptor of the Sphingosine-l-phosphate modulator, which sequesters lymphocytes in the lymphocytes nodes, preventing them from contributing to an autoimmune reaction; (c) a damage on the Inmune suppressor agent that w orks on the lymphocyte’s pathway and/or (d) a therapeutic effect of Monoclonal Antibodies for inhibiting the activation of lymphocyte B on some well-defined neurological processes and related cognitive activities.
  • drugs e.g., Dimethyl fumarate, Fingolimod, Cladribine, Ofatumumab, Interferon
  • the considered dependent variable could be, for example, saccade amplitude, fixation duration, pupil behavior; and predictors could be motor scales, cognitive scales, years of diagnosis of the disease and treatments (i.e., drugs), among others.
  • predictors could be motor scales, cognitive scales, years of diagnosis of the disease and treatments (i.e., drugs), among others.
  • saccade amplitude depends on the strategy developed by the person evaluated to scan figures while performing a particular test. If the test is the n-back task, because of the nature of the test, a person performing better will do longer saccades. Longer saccades suggest that working memory is performing well, while shorted saccades imply a poor performance (as shown previously in this patent). For the saccade amplitude to be longer, in this case, the dorsolateral prefrontal cortex, basal ganglia and superior colliculus must be preserved.
  • pupil behavior varies depending on the cognitive effort performed by the patient in a particular moment.
  • the size of the pupil increases when a task is more demanding (as explained previously in this patent).
  • the pupil size must increase. This particular behavior suggests that the noradrenergic system and also the locus coeruleus are responding properly as the cognitive load increase. This statement is pupil size and cognitive load (Fernandez et al, 2021).
  • Interferon-Beta which could (b) reduces damage on the Inmune suppressor agent that works on the lymphocyte’s pathway
  • the pupil size increases as the cognitive load increases
  • the treatment has a positive impact on the amount of Working Memory resources used (Sweller et al., 2011) and in the noradrenergic system and locus coeruleus.
  • Figure 8 is a flowchart showing a method for identifying specific alterations in subjects with defined disease analyzing oculomotor patterns when using specific visual stimuli, where a specific drug or treatment would enhance visual processing, cognitive performance and related brain activities.
  • a method is presented to evaluate compromises in neurological disorders, fine-motor skills, executive processes, decision making, processing speed andcognitive capabilities associated with Multiple Sclerosis [MS], the method comprising a. providing a system for evaluating compromises in neurological disorders, fine-motor skills, executive processes, decision making, fine motor skills and cognitivecapabilities associated with MS; b. requesting a subject to fixate on a reference target of a chart, where the chartincludes multiple regions (e.g., rectangles) placed in different zones; c.
  • MS Multiple Sclerosis
  • presenting a stimulus image in one of thezones to the subject the subject being requested to remember which zone each stimulus image appeared and in what order; d. presenting to the subject the chart without including the stimulus image presented in step c, where the subject is requested to fixate in a zone that is where the stimulusimage of step c appeared; e. measuring a saccade of the subject in response to the presenting of stepd who is requested to look at the zone in which was presented the stimulus image presented in step c; f. repeating steps d and e of presenting a chart and measuring a saccade; g. repeating steps b-f for a number of trials modifying a time in which thestimulus images are shown; h.
  • a WM effect wherein the WM effect is a measure that increases when WM demand increases.
  • the WM effect is represented by theratio between the number of errors reported by the subject through all the trials, and a number of trials); and ii. an average saccadic latency, saccadic latency defined as an amount of timefor the subject to initiate a saccade to the zone; and reporting one or more of iii. a degree of compromise in working memory, with increased the WMeffect; and i. a degree of compromise in executive processes, with increased saccadiclatency; j.
  • the method further comprises additional steps comprising measurements performed during the step of presenting a stimulus image, during which the subject is further requested to look at the stimulus image; the measurements comprising measuring one or more of a. an amplitude of pupillary dilatation of the subject; b. a number of fixations made by the subject on the stimulus image; c. a gaze duration by the subject on the stimulus image; d. binocular disparity by the while visual exploring and target visualization; e. target hit by the subject fixate where the visual stimulus was present previously; f. number of consecutive target hits by the subject when considering a trial; g. Number of blinks coming from the left eye, the right eye or from both eyes; h.
  • VMIN and VMAX are the minimum andmaximum velocities of themicrosaccades in degrees per second; viii) Microsaccade rate: is the instantaneous rate in each time bin; ix) Directional congruency: is the congruency between the microsaccade directionand the locationof the stimulus; k. Obtaining eye position information coming from the left eye, the right eye or from both eyes (i.e., abscissa and ordinate coordinate) while performing visual exploration.
  • steph may be omitted.
  • step i may be omitted.
  • the additional steps may further comprisecalculating, modelling and reporting one or more effects of drugs (e.g., Dimethyl fumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta) or treatments that (a) decrease inflammation and prevent nerve damage that can cause symptoms of multiple sclerosis); (b) test the Sphingosine-l-phosphate receptor modulator, which sequesters lymphocytes in the lymphocytes nodes, preventing them from contributing to an autoimmune reaction); (c) check an inmune suppressor agent that works on the lymphocyte’s pathway) and/or (d) analyze the effect of Monoclonal Antibodies for inhibiting the activation of lymphocyte B.
  • drugs e.g., Dimethyl fumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta
  • a method for evaluatingcompromises in neurological disorders, fine-motor skills, processing speed, decision making and cognitive processes associated with Multiple Sclerosis.
  • a system and method for detecting oneor more neurological disorders and/or measuring, fine-motor skills, processing speed, decision malting, and cognitive processes in a subject by measuring eye movements, oculomotor features or pupil behaviour, the measuring of eye movements being performed while the subject is visualizing (i.e., to form a picture of something in the mind, in order to imagine or remember it), recognizing (i.e., to identify something from having encounteredit before), maintaining (i.e., to keep in an existing memory), controlling (i.e., to exercise restraint or direction over), inhibiting (i.e., to prevent or hold back from doing something), fixating (i.e., to focus the eyes on something) and analyzing targets.
  • the system may comprise: a. an eye tracker, configured to monitor eye movements of a subject while the subject is visualizing, recognizing, maintaining, controlling, fixating and analyzing targets; b. a processor configured to receive data from the eye tracker while the subjectis visualizing, recognizing, maintaining, controlling, fixating and analyzing the targets; and c. a display configured to display a test report received from the processor, whereinthe processor is further configured to analyze the eye-tracking data for evidence of one or more neurological disorders or general cognitive perfonnance and to report, in the test report, a detection of the one or more neurological disorders or a measure of cognitive performance of the subject.
  • the processor is further configured, upon receiving the eye- tracking data from the eye tracker, to perform one or more (or all) ofthe following: a. count a total number of ocular fixations of a subject while visualizing, recognizing, maintaining, controlling, fixating and analyzing targets; and b. if the total number of ocular fixations of a subject when visualizing, recognizing, maintaining, controlling, fixating and analyzing targets is higher than for a control group, thenreport in the test report that a compromise in attentional processes is detected; c. count a number of correct landing positions of the subject while visualizing, recognizing, maintaining, controlling, fixating and analyzing the targets; and d.
  • the saccade latency length (time) is higher than for the control group, then report in the test report that a compromise in speed processing is detected.
  • m. track the pupil diameter of the subject when visualizing, recognizing, maintaining, controlling, inhibiting, fixating, following and analyzing targets; and n. if the pupil diameter of the subject does not show a modulation as advancingin visualizing, recognizing, maintaining, controlling, inhibiting, fixating, following and analyzing targets, then report in the test report that that a compromise in noradrenergic system is detected.
  • o. consider length of fixation duration of the subject while trying of visualizing, recognizing, maintaining, controlling, fixating, following and analyzing targets; and p.
  • the length of fixation duration is longer than for the control group, thenreport in the testreport that a compromise in on-line processing is detected; q. consider gaze duration of the subject while trying of visualizing, recognizing, maintaining, controlling, fixating, following and analyzing targets; r. if the length of gaze duration is longer than for the control group, then reportin the test report that a compromise in on-line processing is detected; s. count number of correct target recognized while visualizing, recognizing, maintaining, controlling, inhibiting, fixating, following and analyzing targets; and t. if the number of correct target recognized is lower than for the control group, then report the inthc test report that compromises in executive and working memory processes are detected; u.
  • microsaccades' Factors of Form FF
  • HEWI shows the micro-saccade's height/width relationship
  • AREA shows the area of the rectangle in which the micro-saccade is inscribed
  • iii. LONG is the longitude of the horizontal-vertical plane trajectory of themicro-saccade
  • ANG is the sum of all the angles in the plane horizontal - vertical planeofthe micro-saccade
  • AANG is the sum of all the absolute values of angles in radians in the planehorizontal - verticalplane of the micro-saccade
  • vi. FF gives an estimation of the micro-saccadic trajectory regularity
  • vii. MOD and THETA are the modulus and the angle of the polar coordinates of the sum of the cartesian coordinates. They give a spatial orientation of the micro saccaderelative to the median of the fixation
  • TIME is the time duration in milliseconds of the micro-saccade
  • ix is the time duration in milliseconds of the micro-saccade
  • VMIN and VMAX are the minimum and maximum velocities of the microsaccades in degrees per second;
  • Micro-saccade rate is the instantaneous rate in each time bin;
  • Directional congruency is the congmency between the micro-saccade directionant the location of the stimulus;
  • w. Measure eye position coming from the left eye, the right eye or from both eyes (i.e., abscissa and ordinate coordinate) during visualizing, recognizing, maintaining, controlling, sequencing and analyzing targets;
  • x. measure total visualizing, recognizing, maintaining, controlling, fixating, following and analyzing targets time (i.e., the time that the subject spent when visualizing targets through a trial);
  • the processor may be further configured to perform additional steps that include calculating, modelling and reporting one or more effects of drugs (e.g., Dimethylfumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta) or treatments that (a) decrease inflammation and prevent nerve damage that can cause symptoms of multiple sclerosis);
  • drugs e.g., Dimethylfumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta
  • treatments that (a) decrease inflammation and prevent nerve damage that can cause symptoms of multiple sclerosis e.g., Dimethylfumarate, Fingolimod, Cladribine, Ofatumumab, Interferon-Beta
  • test Sphingosine-l-phosphate receptor modulator which sequesters lymphocytes in the lymphocytes nodes, preventing them from contributing to an autoimmune reaction
  • checkan immune suppressor agent that works on the lymphocyte’s pathway
  • VR and ET virtual reality
  • the application of VR and ET in cognitive exercises with motion sensors can improve the efficacy of the intervention and the ability to quantify cognitive and motor capabilities, enhancing the effectiveness of the training on a person.
  • the combination of VR, visual scanning and arm and leg movements can provide new information about a person’s decision-making processes and the integrity of brain circuits (e.g., what a person does when visualizing a shape and deciding to move the hand to touch something in a VR environment).
  • Such a methodology would enhance a healthcare professional’s ability to analyze, quantify and train cognitive capabilities and fine-motor skills.
  • the eye tracking system described herein may be incorporated in a conventional Head Mounted Display (HMD), where a VR world is rendered and seen by the user.
  • HMD Head Mounted Display
  • Each manufacturer of commercially available HMDs/VR devices performs eye tracking in a somewhat different way.
  • the interface provided to developers allows them to access the vector (three numerical values) which indicates the direction that the eye is looking in the 3D virtual space created by the VR application. In this way integration between the eye-tracking system and the VR application is seamless.
  • This interface is usually available in multiple languages/gaming development environments. Examples of some types of commercial VR devices that may employed include: HTC Vive Eye Pro; HP G2 Reverb Omnicept Edition; Varjo Aero and Fove 0.
  • the controllers that are provided with the VR device generally will work to track hand motions.
  • VR virtual reality
  • a VR input device serves as a conduit for transmitting hand motion data to a computer system. This information is subsequently processed and harnessed to control objects existing within the simulated world.
  • motion controllers employ accelerometers and gyroscopes to detect motion and orientation changes. They can also incorporate buttons, analog sticks, and various input mechanisms depending on the specific device.
  • Motion controllers excel in scenarios where direct interaction with the virtual surroundings is paramount, such as exploratory first-person experiences.
  • game controllers are more commonly associated with traditional gaming encounters. These controllers usually offer an array of input options, encompassing dual analog sticks and a multitude of buttons.
  • an added layer of immersion is achieved through the inclusion of haptic feedback. This augmentation enhances the virtual experience by providing tangible responses when interacting with elements within the virtual domain.
  • the dependent variable that is considered could be, for example, saccade amplitude, fixation duration, pupil behavior, hand reaction time, tracking accuracy; and independent variables could be motor measurements, cognitive measurements, years of diagnosis of the disease, treatments, among others.
  • the following illustrative examples explain how to use saccade amplitude as a dependent variable:
  • the saccade amplitude depends on the strategy developed by the person being evaluated to send the eyes to a particular shape while performing a particular test. If the test is the Go No-Go 3D, because of the nature of the test, a person performing better will perform longer saccades. Longer saccades suggest that working memory is performing well, while shorter saccades imply a poor performance (as previously discussed herein). For the saccade amplitude to be longer, in this case, the dorsolateral prefrontal cortex, basal ganglia and superior colliculus must be preserved.
  • a person’s Hand Reaction Time assesses the average time it takes for the person to initiate a manual response after visually perceiving a target. It may reflect the person's motor response time and coordination. This measurement may provide insights into the speed at which the person can translate visual information into a motor action. Hand Reaction Time may evaluate the potential efficiency of sensorimotor processing and the person's ability to initiate a manual response promptly. When performing the Go No-Go 3D, given the complexity of the test, the Hand Reaction Time will decrease. This particular behavior suggests that the primary motor cortex and the cerebellum are responding properly as the difficulty of the test increases.
  • Figure 9 shows a conceptual illustration of a system used to evaluate a person’s [5] performance by applying a 3-Dimension Virtual Reality (3DVR) environment in combination with an embedded eye-tracking technology (ET) [10] and motions sensors to track the movement of limbs such as hands and feet [110] while the person performs well-defined activities [15],
  • 3DVR 3-Dimension Virtual Reality
  • ET embedded eye-tracking technology
  • FIG. 9 shows a conceptual illustration of a system used to evaluate a person’s [5] performance by applying a 3-Dimension Virtual Reality (3DVR) environment in combination with an embedded eye-tracking technology (ET) [10] and motions sensors to track the movement of limbs such as hands and feet [110] while the person performs well-defined activities [15].
  • E embedded eye-tracking technology
  • 3D Virtual Reality and Eye-Tracking Technology in a Go No-GO Task In order to quantify specific motor skills and cognitive processes, a method is presented to evaluate the performance in both healthy or non- healthy persons [5], The method employs a 3-Dimension Virtual Reality (3DVR) environment such as described above and depicted in Figure 9, which can be used to evaluate the person's performance, fine motor skills and cognitive capabilities by applying 3DVR, ET and limb movement tracking.
  • 3DVR 3-Dimension Virtual Reality
  • a person is requested to visualize objects on the VR screen while the eye movements are registered.
  • Each object will have a defined feature such as colour, and it will move towards a person with a defined speed, acceleration and direction.
  • objects are presented in different zones to the person and the person is requested to visually observe the objects on the screen.
  • the objects presented appear to move towards the subject in the 3D virtual environment and the objects will increase (or decrease) in apparent velocity when the person effectively virtually touches the correct objects by moving their limb towards the shape. Likewise, the object will decrease (or increase) in velocity when the person virtually touches the incorrect objects by moving their limbs.
  • Figure 10 shows an example of how objects may be positioned throughout the virtual environment and how objects and the person’s hands may appear.
  • the objects may be presented in two different colors and appear to come from the background of the screen.
  • the person is asked to touch the objects having one particular color.
  • the person can use his or her right or left hand or right or left leg to touch the appropriate object, depending on whether the object is presented at the level of the hands or legs.
  • the method continues by measuring the saccades of the person in response to the presentation of the objects.
  • the person is requested to look at the objects and touch them (or not touch them) in the virtual environment, and then the saccades are measured.
  • These steps of requesting the person to visually observe the object and measuring saccades may be repeated multiple times.
  • Any of a variety of different requests may be made to the person to visualize and virtually touch (or not touch) objects on the VR screen while the eye movements are registered.
  • the person may be requested to virtually touch objects having a certain feature (e.g., the color red) and not virtually touch objects having another feature (e.g., the color green).
  • the aforementioned steps may be repeated for any number of instances of objects being presented, which may be presented at different speeds.
  • Figure 11 is a graphical representation of an example of the density of eye movements recorded from the right and left eye while the person is performing the tasks touching the requested objects, presented as a heat-map.
  • Figure 12 is a graphical representation of the density of motor movements recorded of right and left-hand movements while the person is performing the tasks touching the requested objects, presented as a heat-map.
  • any one or more of the following metrics may be calculated: i. an inhibition process error (i.e. how many times the person touch the incorrect shapes); and ii. an average saccadic latency, saccadic latency defined as an amount of time for said person to initiate a saccade to anew shape; and reporting one or more of iii. a degree of compromise in processing speed with increased changes in the speed at which successive shapes are presented to the subject; and iv. a degree of compromise in executive processes, with increased inhibition error.
  • an inhibition process error i.e. how many times the person touch the incorrect shapes
  • reporting one or more of iii. a degree of compromise in processing speed with increased changes in the speed at which successive shapes are presented to the subject and iv. a degree of compromise in executive processes, with increased inhibition error.
  • the method may include additional steps, including the step of making additional measurements during the step of presenting the objects to the person and requesting that the person look and touch (or not touch) the object.
  • additional measurements may include one or more of the following: i. an amplitude of pupillary dilatation of the person; ii. a number of fixations made by the person on said stimulus image; and iii. the gaze duration by the person on the stimulus image; iv. binocular disparity by the person while performing visual exploration and object visualization v. object touched by person and fixations of where the visual stimulus was before vi. number of consecutive object touched by person when performing a trial; vii. Number of blinks coming from the left eye, the right eye or from both eyes.
  • viii Time taken to visually detect objects (For a heat map of eye movements see Figure 3).
  • ix Time from the visualization of the object until the moment the person start to move the hands and/or feet.
  • x Time since the person start to move the hands and/or feet up the time the person touches or tries to touch the object.
  • xi Number of times the subject touch -or not- the virtual object (For a heat map of hand movements see Figure 4).
  • xii Optimal place for target visualization and places where visualizing targets is less efficient.
  • xii Tracking Accuracy in maintaining visual focus on moving objects.
  • xiii Hand-Reach Depth towards the objects during the act of touching.
  • xiv Time taken to visually detect objects
  • AREA shows the area of the rectangle in which the microsaccade is inscribed.
  • LONG is the longitude of the horizontal-vertical plane trajectory of the microsaccade.
  • ANG is the sum of all the angles in the plane horizontal - vertical plane of the microsaccade.
  • AANG is the sum of all the absolute values of angles in radians in the plane horizontal - vertical plane of the microsacaccade. These last two FF give an estimation of the microsaccadic trajectory regularity.
  • MOD and THETA are the modulus and the angle of the polar coordinates of the sum of the cartesian coordinates. They give a spatial orientation of the microsaccade relative to the median of the fixation.
  • TIME is the time duration in milliseconds of the microsaccade.
  • VMIN and VMAX are the minimum and maximum velocities of the microsaccades in degrees per second.
  • Microsaccade rate is the instantaneous rate in each time bin.
  • Directional congruency is the congruency between the microsaccade direction and the location of the stimulus.
  • Additional steps that may be taken while presenting the objects to the person and requesting that the person look and touch (or not touch) the object may include measuring the person’s eye position coming from the left eye, the right eye or from both eyes (i.e., abscissa and ordinate coordinate) while performing visual exploration.
  • the measurements may consider: l. eye movements while visualizing and while touching objects;
  • Yet another additional step that may be performed includes quantifying neurological processes and related cognitive activities when considering pupil size behavior and/or binocular disparity and/or micosaccade features and/or saccade behaviour and/or target touch rate and/or hands and feet movements and/or gazing and/or fixation duration and/or number of fixations and/or executive function performance in a 3DVR.
  • FIG. 13 is flowchart describing one example of the methods described herein for evaluating performance, motor skills and cognitive capabilities of a person.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Data Mining & Analysis (AREA)
  • Veterinary Medicine (AREA)
  • Neurology (AREA)
  • Neurosurgery (AREA)
  • Developmental Disabilities (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychiatry (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Physiology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Chemical & Material Sciences (AREA)

Abstract

L'invention porte sur des systèmes et des procédés combinant la réalité virtuelle (VR), le suivi oculaire (ET) et des capteurs de mouvement sur des membres, tels que des mains et des pieds, pour évaluer les changements de capacités cognitives et motrices à la fois chez des personnes saines et non saines à l'aide d'exercices bien définis. L'application de VR et ET dans des exercices cognitifs avec des capteurs de mouvement peut améliorer l'efficacité de l'intervention et la capacité de quantifier les capacités cognitives et motrices, améliorant l'efficacité de l'apprentissage sur une personne.
PCT/US2023/030917 2022-08-23 2023-08-23 Procédé d'apprentissage et de quantification de compétences motrices spécifiques et de processus cognitifs chez des personnes par analyse de motifs oculomoteurs w au moyen d'un dispositif de réalité virtuelle 3d à technologie de suivi oculaire intégrée, stimuli visuels spécifiques, capteurs pour montrer le mouvement de membres WO2024044242A1 (fr)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US202263373228P 2022-08-23 2022-08-23
US63/373,228 2022-08-23
US18/217,688 2023-07-03
US18/217,688 US20230352175A1 (en) 2017-11-30 2023-07-03 System and method for detecting neurological disorders and for measuring general cognitive performance
US18/227,577 2023-07-28
US18/227,577 US20230368919A1 (en) 2017-11-30 2023-07-28 Method for identifying specific alterations in subjects with defined diseases analyzing oculomotor patterns when using specific visual stimuli, where a specific drug or treatment would enhance visual processing, cognitive performance and related brain activities

Publications (1)

Publication Number Publication Date
WO2024044242A1 true WO2024044242A1 (fr) 2024-02-29

Family

ID=90013899

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2023/030917 WO2024044242A1 (fr) 2022-08-23 2023-08-23 Procédé d'apprentissage et de quantification de compétences motrices spécifiques et de processus cognitifs chez des personnes par analyse de motifs oculomoteurs w au moyen d'un dispositif de réalité virtuelle 3d à technologie de suivi oculaire intégrée, stimuli visuels spécifiques, capteurs pour montrer le mouvement de membres

Country Status (1)

Country Link
WO (1) WO2024044242A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090306741A1 (en) * 2006-10-26 2009-12-10 Wicab, Inc. Systems and methods for altering brain and body functions and for treating conditions and diseases of the same
US20180235549A1 (en) * 2013-03-04 2018-08-23 Cogneuro Solutions Llc Touch Sensitive System and Method for Cognitive and Behavioral Testing and Evaluation
US20190307384A1 (en) * 2016-11-23 2019-10-10 Cognifisense, Inc. Identifying and measuring bodily states and feedback systems background
US20200242957A1 (en) * 2017-05-18 2020-07-30 Pearson Education, Inc. Multi-level executive functioning tasks
WO2021021328A2 (fr) * 2019-06-14 2021-02-04 Quantum Interface, Llc Systèmes de formation virtuelle prédictifs, appareils, interfaces et leurs procédés de mise en oeuvre
US20210093240A1 (en) * 2019-09-30 2021-04-01 BioMech Sensor LLC Systems and methods for reaction measurement
US20210174959A1 (en) * 2017-11-30 2021-06-10 Viewmind S.A. , System and method for detecting neurological disorders and for measuring general cognitive performance

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090306741A1 (en) * 2006-10-26 2009-12-10 Wicab, Inc. Systems and methods for altering brain and body functions and for treating conditions and diseases of the same
US20180235549A1 (en) * 2013-03-04 2018-08-23 Cogneuro Solutions Llc Touch Sensitive System and Method for Cognitive and Behavioral Testing and Evaluation
US20190307384A1 (en) * 2016-11-23 2019-10-10 Cognifisense, Inc. Identifying and measuring bodily states and feedback systems background
US20200242957A1 (en) * 2017-05-18 2020-07-30 Pearson Education, Inc. Multi-level executive functioning tasks
US20210174959A1 (en) * 2017-11-30 2021-06-10 Viewmind S.A. , System and method for detecting neurological disorders and for measuring general cognitive performance
WO2021021328A2 (fr) * 2019-06-14 2021-02-04 Quantum Interface, Llc Systèmes de formation virtuelle prédictifs, appareils, interfaces et leurs procédés de mise en oeuvre
US20210093240A1 (en) * 2019-09-30 2021-04-01 BioMech Sensor LLC Systems and methods for reaction measurement

Similar Documents

Publication Publication Date Title
US20240099575A1 (en) Systems and methods for vision assessment
KR102449377B1 (ko) 적응적 응답 기한 절차들에서의 신호 검출 메트릭들을 구현하기 위한 플랫폼들
Islam et al. Cybersickness prediction from integrated hmd’s sensors: A multimodal deep fusion approach using eye-tracking and head-tracking data
KR20210060595A (ko) 사용자 인터랙션들의 고속 및 정확한 추적을 사용하는 인간-컴퓨터 인터페이스
KR20200108888A (ko) 고속, 정확, 및 직관적 사용자 상호작용들을 위한 적응들을 갖는 뇌-컴퓨터 인터페이스
US11839472B2 (en) Platforms to implement signal detection metrics in adaptive response-deadline procedures
US20230248236A1 (en) Method and apparatus for detecting ocular movement disorders
CN112535479B (zh) 一种情绪加工倾向的确定方法及相关产品
EP3716837A1 (fr) Système et procédé de détection de troubles neurologiques et de mesure des performances cognitives générales
KR20200098524A (ko) 고속, 정확도 및 직관적 사용자 상호작용을 위한 적응을 갖춘 두뇌-컴퓨터 인터페이스
Krishnan et al. Gaze exploration index (GE i)-explainable detection model for glaucoma
Iskander et al. A k-nn classification based vr user verification using eye movement and ocular biomechanics
US20240029886A1 (en) Method For Training And Quantifying Specific Motor Skills And Cognitive Processes In Persons By Analysing Oculomotor Patterns W Using A 3-D Virtual Reality Device With Embedded Eye-Tracking Technology, Specific Visual Stimuli, Sensors To Show The Movement Of Limbs
WO2024044242A1 (fr) Procédé d'apprentissage et de quantification de compétences motrices spécifiques et de processus cognitifs chez des personnes par analyse de motifs oculomoteurs w au moyen d'un dispositif de réalité virtuelle 3d à technologie de suivi oculaire intégrée, stimuli visuels spécifiques, capteurs pour montrer le mouvement de membres
Clark et al. The effect of movement direction, hand dominance, and hemispace on reaching movement kinematics in virtual reality
US20230368919A1 (en) Method for identifying specific alterations in subjects with defined diseases analyzing oculomotor patterns when using specific visual stimuli, where a specific drug or treatment would enhance visual processing, cognitive performance and related brain activities
WO2024026089A2 (fr) Procédé d'identification d'altérations spécifiques chez des sujets ayant des maladies définies analysant des motifs oculomoteurs lors de l'utilisation de stimuli visuels spécifiques, un médicament ou un traitement spécifiques améliorant le traitement visuel, les performances cognitives et les activités cérébrales associées
Chytas et al. Characterization of Upper Limb Functionality Caused by Neuromuscular Disorders using Novel Motion Features from a Specialized Gaming Platform.
Shal et al. A Review on different techniques for human-computer Communication using eye movement
Mehringer et al. Virtual reality for assessing stereopsis performance and eye characteristics in Post-COVID
US20240081706A1 (en) Platforms to implement signal detection metrics in adaptive response-deadline procedures
Hvass et al. A preliminary exploration in the correlation of cybersickness and gaze direction in VR
Papanikolaou et al. Predicting ADHD in Children Using Eye-Tracking Data and Mathematical Modeling
Laubrock et al. Continuous methods: Eye tracking, hand tracking, and grip force recording
Kubicek et al. Mouse Tracking Algorithm Based on the Multiple Model Kalman Filter Design and Implementation of Ophthalmological Corrector

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23858031

Country of ref document: EP

Kind code of ref document: A1