Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
  • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
  • A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• • G—PHYSICS
  • G08—SIGNALLING
  • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
  • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
  • G08B21/02—Alarms for ensuring the safety of persons
  • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
  • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
• • G—PHYSICS
  • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
  • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
  • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
  • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
  • A61B2503/20—Workers
  • A61B2503/22—Motor vehicles operators, e.g. drivers, pilots, captains
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
  • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
  • A61B5/02438—Detecting, measuring or recording pulse rate or heart rate with portable devices, e.g. worn by the patient
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
  • A61B5/053—Measuring electrical impedance or conductance of a portion of the body
  • A61B5/0531—Measuring skin impedance
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
  • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
  • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B5/00—Measuring for diagnostic purposes; Identification of persons
  • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
  • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B64—AIRCRAFT; AVIATION; COSMONAUTICS
  • B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
  • B64D45/00—Aircraft indicators or protectors not otherwise provided for
  • B64D45/0015—Devices specially adapted for the protection against criminal attack, e.g. anti-hijacking systems
  • B64D45/0051—Devices specially adapted for the protection against criminal attack, e.g. anti-hijacking systems by monitoring passengers or crew on aircraft

Definitions

• the invention relates to the general field of support systems for monitoring the states of individuals, and in particular provides a method and a device for providing longitudinal and contextualized individual monitoring of the cognitive states of individuals.
• the invention thus covers several fields which are psychophysics, neurophysiology and the analysis of cognitive behavior. It aims to solve at least the technical problem of comparing the psychophysical and physiological state of a healthy individual at a given moment, ie his states on a cognitive level or cognitive states covering without limitation his states of vigilance, fatigue, drowsiness. , at a given moment, to these same cognitive states obtained in similar or even similar contexts.
• the invention also addresses a related and underlying technical problem which is to build an individualized modeling of the cognitive states of a healthy individual.
• these cognitive states are not evaluated in relation to the intrinsic and personal characteristics of these participants, that is to say by intra-personal comparison.
• the cognitive states of an individual are evaluated in a general context of a mission which is / must be carried out (for example driving a car) and not in relation to an instantaneous context or an instantaneous situation of a mission (for example, in an engaged flight phase such as take-off or while driving on a motorway, etc.).
• the perceptual-motor style defines the way in which a subject interprets and combines his sensory inputs which are visual, vestibular and proprioceptive information to generate an adapted motor behavior.
• This integration of perceptual signals like mental load and alertness, has great inter-individual variability, and this is what constitutes the technical problem of constructing an effective individualized modeling of the cognitive states of a healthy individual.
• the integration of perceptual signals can be considered as stable over time under nominal conditions.
• an object of the invention is to meet the aforementioned needs and to overcome the drawbacks of existing techniques.
• the invention proposes a solution based on the longitudinal and contextualized monitoring of an operator (making it possible to acquire a personalized knowledge of the latter) thanks to the interaction of several components: an individual component called “Personal Assistant Operator” or (APO) which integrates personal data of the operator; an anonymized operator database;
• APO Personal Assistant Operator
• the personal assistant operator can be a device capable on the one hand of supporting an operator during his mission (for example in difficult or rarely encountered situations), and on the other hand of '' support the operator outside of his mission.
• An example of non-mission support can be, for example, to orient the continuing training of the operator according to the observations made during the mission, and returned by the APO.
• Another use may be to replay in a simulator the scenario of a mission by adapting it as a function of the situations and reactions occurring during said mission, and reproduced by ARO.
• the assistant can be used to optimize the well-being or "well-being" of the operator by proposing, for example, sleep / wake schedules, meals, meal contents, physical training episodes for example to manage jet lag or very long standby.
• the data of the personal assistant operators can be anonymized in order to create a knowledge database. which can be an element of comparison and reference for the personal assistant operator.
• the general principle of individual and contextualized longitudinal monitoring is that during each training session on a simulated system, and each mission on a real system, two types of data are recorded: physiological data specific to the operator (such as the heart rate) and contextual or operational data specific to the mission (instructions, tasks performed, handling of the man-machine interface, machine state variables). All of this data is then aggregated to provide real-time personal assistance, which is specific to each operator based on: variables that describe the current situation; and the operator’s past experiences, collected on the APO.
• physiological data specific to the operator such as the heart rate
• contextual or operational data specific to the mission instructions, tasks performed, handling of the man-machine interface, machine state variables
• this assistance provides the operator with operational support in case of need (decrease in vigilance, signs of onset of sleep, rise in the level of mental load, deviation of perceptual-motor style, etc.). ) by one comparison between his instantaneous cognitive state and his previous cognitive states in similar contexts.
• BCC common knowledge base
• the process implements a theoretical knowledge base (BCT) reflecting the state of the art and the constraints imposed on the operator in all similar or even identical situations, ie same conditions, same operational situation, etc., this state of the art being for example all the procedures to be followed in the situation considered.
• BCT theoretical knowledge base
• the proposed process can be described as a mise en abyme of ARO thanks to the BCC and the BCT.
• This mise en abyme makes it possible to assess the current behavior of an operator vis-à-vis his "personal envelope", a "collective envelope” and a “theoretical envelope”. If necessary, the operator can be offered decision aids, reconfigurations of a man-machine interface, etc. which are adapted to the present situation.
• the present invention will find many fields of application where there is a need to monitor and support operators of complex systems throughout their career.
• the fields of use of the invention are the fields of activity in which a complex system is controlled and implemented by one or more operators.
• These business areas include, for example, industry and aeronautical, rail and road transport, the fields of security, the fields of process control, even the medical, legal fields, etc.
• the field of use of the invention also extends to the training of operators and the possibility of measuring and monitoring their progress during their training, in order to check their progress in learning and mastery. the conduct of the process for which they are responsible, and adapt the training to their behavior.
• the field of operation of the invention is also that of the continuous training of the operator, the latter being adapted according to the needs of the operator revealed by the restitution of the personal assistant and quantified during the missions and putting it into perspective with data from the common knowledge base.
• the device of the invention in the context of aeronautics, participates in several areas of innovation: the development of the cockpit of the future, pilot monitoring (in flight and off flight), physiological monitoring and psychophysics of the latter or even flight safety and mission performance.
• the proposed device addresses the problem of how to ensure that a pilot is robust from an operational point of view. This problem is multiple and raises several technical questions, because it is indeed necessary to be able to measure several physiological parameters in the least invasive way possible, to interpret these signals, to compile them with the contextual data of the mission and to produce as output relevant information which is operationally useful for the pilot.
• the advantages of the invention are multiple and are located on several levels.
• the personal assistant allows better health monitoring. By longitudinally recording the heart rate and the respiratory rate (as well as other physiological parameters), by carrying out regular checks of the hearing, the vision and possibly the operator's nycthem, the evolution or the appearance of minor clinical signs are listed and treated before the occurrence of characterized disorders. It is therefore a preventive approach.
• the individual longitudinal monitoring of the sensorimotor system makes it possible to define the notion of operational robustness and to evaluate it at regular intervals throughout the operator's career.
• the proposed solution allows better management of problems that may arise during the accomplishment of a mission (breakdowns, multiple actions to be carried out simultaneously, etc.).
• the aggregation of the physiological and psychophysical parameters of an operator also makes it possible to model individually in real time (as opposed to a time shifted after the mission), the cognitive states of the operator (fatigue, vigilance, mental load). This allows better management of delicate situations during the operation of the complex systems involved.
• the proposed solution which benefits over time from the continuous capitalization of its implementation, makes it possible to improve the robustness of the models. detection of the physiological and cognitive state of the operator.
• a computer-implemented method is proposed to aid in monitoring the states of an individual operating on a complex system during a mission.
• the method comprises at least steps for: collecting regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
• the step of analyzing intra-personal compliance comprises a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of individual reference states for said operator, said data being associated with the characterization of the physiological state and the contextualized cognitive state of said operator.
• the interpersonal conformity analysis step comprises a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of collective reference states for a plurality of operators, said data being associated with the characterization of the physiological state and the contextualized cognitive state of the plurality of operators during missions similar to the current mission.
• the step of collecting data also consists of collecting data representative of tasks performed by the operator; and the method further comprises the step of restoring the results of the analysis, a step consisting in carrying out a theoretical conformity analysis with respect to theoretical reference states, on the basis of the contextualization parameters and of parameters representative of the tasks operator; and where said steps of analyzing intra-personal compliance, interpersonal compliance, and theoretical compliance are performed in any order.
• the registry data is structured in triads "Physiological State - Cognitive States - Contextual Situation" associated with the sequence being analyzed.
• the method further comprises, after the step of analyzing intra-personal conformity, a step consisting in determining whether the instantaneous state of the operator during the mission is in acceptable conformity with his own individual reference states or n is not in acceptable compliance.
• the method further comprises, after the step of analyzing interpersonal conformity, a step consisting in determining whether the instantaneous state of the operator during the mission is in acceptable conformity with collective reference states or is not in acceptable compliance.
• the method further comprises, after the step of theoretical compliance, a step consisting in determining whether the performance of the instantaneous tasks of the operator during the mission is in acceptable compliance with the performance of the theoretical reference actions or is not in acceptable compliance.
• This step can, depending on the case, be carried out before or after the two steps of verification of intra-personal and interpersonal compliance.
• the method further includes after each analysis step, a step of capitalizing the results of each analysis to enrich the individual and collective reference states.
• the step of determining an instantaneous state of the operator consists in implementing, on the physiological data, evaluation and prediction models associated with each physiological aspect.
• the method further comprises, after the restitution step, a step of providing the operator, via a suitable interface, with personalized services according to the results of the analysis steps.
• the method further comprises after the restitution step, a step of transmitting the results of the analysis steps to external systems.
• the invention also covers a computer program product comprising code instructions for performing the steps of the claimed method, when the program is executed on a computer.
• the invention further covers a device for assisting in monitoring the states of an individual operating on a complex system during a mission.
• the device comprises means making it possible to:
• the device of the invention further comprises means for collecting during the mission a set of data relating to the tasks performed by the operator, in particular the handling, observation and communication tasks, which make it possible to carry out a theoretical conformity analysis of the tasks which are performed by the operator, with respect to the theoretical rules of the tasks to be performed or of the procedures to be performed, which are recorded as a theoretical state of the art.
• the device of the invention comprises further other means for carrying out the steps of the method of the invention.
• FIG.1 a general context for implementing the device of the invention in one embodiment
• FIG.2 an architecture of the device of the invention according to one embodiment
• FIG.3 an architecture of the device of the invention according to an alternative embodiment
• FIG.4 a method of implementing the device of the invention according to one embodiment
• FIG.5 a method of implementing the device of the invention according to another embodiment
• FIG.6 an architecture of the device of the invention according to an alternative embodiment.
• FIG. 1 shows a general environment making it possible to implement the device of the invention 100. Although not illustrated so as not to be considered as a limitation, the description of an embodiment is made for an aeronautical environment . However, a person skilled in the art can apply the principles described to any other environment mentioned above such as industry and rail or road transport, the fields of safety, the fields of process control, etc.
• An operator 102 operates in an environment during simulation training or a real mission, illustrated by a platform 104.
• the operated platform is a complex system operated by the operator such as for example an aircraft (or a vehicle or a control post, ...) from which it is possible to retrieve context information on the current mission, on the state of the complex system and on the environmental situation (in the aeronautical context: weather conditions, air traffic conditions, etc.).
• the state of the system in the context of avionics can be information relating to the states inherent in normal operation, ie the percentage of engine power, if the fins are folded or unfolded, the condition of fuel tanks, etc., and information relating to states inherent in abnormal operation, ie the left engine is faulty, the right pedal of the co-pilot's seat is faulty, or any kind of potential fault.
• the platform can also provide information on the actions carried out by the operator through its HMIs.
• the context is represented by two distinct elements: a task to be performed by the operator.
• the physiological and cognitive states of the operator are classified in relation to the tasks to be performed by the operator.
• the tasks can be of various kinds: routine procedure, reaction to a given problem, etc.
• Each task corresponds to a physiological, psychophysical and cognitive signature, where a signature corresponds to a characteristic identified in an acquired signal and which is specific to a given operator and to a given task.
• the signature in the acquisition of the heart rate can be an acceleration of it in a stressful situation (breakdown, failure, change of route, strong wind during landing, etc.). the environment in which the complex system operates and the task is performed.
• the environment constitutes an adjustment variable for the analysis of the operator state: it makes it possible to adjust envelopes of the aforementioned signatures according to the context. For example, when one seeks to characterize the signatures of a landing, the elements that go into “environment” will be: the weather (wind, etc.), the arrival airport (or even the runway used), etc. .
• the environment of the operator (and / or the operator himself) is equipped with various sensors 106 making it possible to record physiological data and task or handling data of the operator during the operation. training or mission.
• the physiological and manipulatory data acquired are, for example, without limitation: i) the heart rate by means of a heart rate bracelet or a chest strap; ii) the respiratory rate thanks to the same chest strap or thanks to a layer of sensors located in or on the seat; iii) eye tracking data collected by means of a camera located either at a distance from the pilot on the dashboard or on glasses worn by the pilot; iv) skin resistance by means of the heart rate monitor or a dedicated bracelet, v) movement sensors worn by the operator at the level of the head, trunk, upper and lower limbs or placed at the level of its manipulandum: the speed and acceleration of the rudder pedals, the joystick and the joysticks, vi) all the observation actions performed by the operator and the areas viewed in the cockpit, vii) all the handling actions performed by the operator on the instrument panel and
• the device of the invention 100 or personal assistant operator generally comprises a contextualization component 108, an individual instantaneous operator status component 110, an operator status analysis component 112, a base or register of reference states 114, an individual and collective capitalization component (l / C) 116 and a restitution component 118.
• the reference state register 114 comprises a base of individual reference states 114-1 and a base of collective reference states 114-2. According to the variant embodiment illustrated in FIG. 5, the reference state register 114 further comprises or is coupled to a base of theoretical reference states 114-3.
• the contextualization component 108 comprises means for receiving context data from the platform 104 and for providing information, contextualization parameters.
• Context data can include flight data such as: route, altitude, speed, acceleration, attitude and envelope.
• aircraft parameters can be taken into account, such as: date of the last maintenance, in-flight events, etc.
• the contextualization component is configured to take into account external data such as weather conditions.
• the contextualization component 108 comprises means for analyzing the information received and extracting elements of interest (such as for example the force, the orientation of the wind) characteristic of a sequence during the game (a sequence being a particular moment like for example the landing) and operational elements (as for example in the case of the landing of an aircraft: how did the operator operate the action?; what is the attitude of the aircraft, its speed, inclination, what actions on the pedals, etc.) in relation to the behavior of the operator.
• the device makes it possible to segment a flight into relevant flight phases (restriction of the sequence in the time domain), and to segment the analysis of the actions of the operator (restriction in the operational domain).
• the contextualization component 108 also takes into account the oral communications made by the operator with the other actors of the mission, and identifies key moments transmitted or received by the operator, such as announcements of heading, speed reference. or altitude.
• the contextualization elements identified by the contextualization component 108 are provided to the operator state analysis component 112.
• contextualization elements are also provided to the operator individual snapshot component 110 in order to enhance the robustness of prediction models and make them less sensitive to false positives.
• the individual operator instantaneous state component 110 receives the information produced by the sensors 106 in order to provide information on the state. operator snapshot.
• the sensors are not only physiological, but they can also acquire the manual actions operated by the operator, as well as the communications and their voice analysis, etc.
• the individual operator snapshot component 110 includes means for processing different physiological aspects of the operator, as well as his observations, manipulations, communications.
• the information from the contextualization component 108 and from the operator individual instantaneous state component 110 are processed by the operator state analysis component 112 which also receives information on individual and collective reference states from the component d 'reference states 1 / C 114, in order to deliver information resulting from the analysis.
• the reference state component 114 comprises two modules: a module of individual reference states of the operator 114-1, a module of collective reference states 114-2. In an alternative embodiment illustrated in Figure 5, the reference state component 114 further comprises or is coupled to a third theoretical reference state module or register 114-3.
• the theoretical reference state register 114-3 also proceeds from a knowledge base which records and makes available a characterization of the state of the art and of the technical and operational constraints associated with the field concerned. This base of theoretical reference states also constitutes a structured memorization of the various procedures and operating envelopes authorized according to the context and the situation.
• the analysis made by the operator state analysis component 112 takes into account the individual reference states, the collective reference states and the theoretical reference states, as received. by the base of reference states 114.
• the rendering component 116 comprises means for broadcasting the analysis information to external systems 120 and / or to applications 122 dedicated to the operator 102.
• the operator state analysis component 112 further provides the I / C capitalization component 118 with information resulting from the analysis.
• the component of capitalization l / C 118 includes two modules: an individual capitalization module 118-1 and a collective capitalization module 118-2 which aim to enrich the individual and collective knowledge bases of the component of reference states l / C 114, and to continuously improve the behavioral models and the analysis and discernment capacities of these models.
• Figure 2 illustrates an architecture of the device of the invention 100 according to an embodiment according to Figure 1.
• the components of Figure 1 whose function is identical keep the same references in the various figures, variants of implementation which can be carried out by those skilled in the art while retaining the same functionalities. Thus, only the elements not described in FIG. 1 are described in detail.
• the operator individual instantaneous state component 110 comprises specific evaluation and prediction models 202, associated with each of the different physiological aspects 204. to assess, for example without limitation, sensorimotor, disability, vigilance, drowsiness, mental load.
• the assessment and prediction models are based on artificial intelligence models trained to produce instantaneous individual states (cognitive, physiological, etc.) as they should be if all goes well. (no breakdown, etc.).
• the individual operator snapshot component addresses only one physiological aspect.
• the I / C reference state component 114 includes records of two types of reference states: individual reference states of operator 114-1 and collective reference states of a plurality of reference states. operators 114-2.
• the register of individual reference states of the operator 114-1 comprises significant data associated with the characterization of the physiological state and the contextualized cognitive state of an operator, and constitutes a knowledge base of previously recorded data.
• This database constitutes a structured memorization of the operator's experiences during his previous missions.
• the structuring of the individual reference register associates the physiological state and the cognitive states of the operator with a contextual situation encountered during a recorded sequence, and presents data structured in triads "Physiological State - Cognitive States - Contextual Situation" associated with the current sequence representative of the behavior of the operator.
• the register of collective reference states operators 114-2 also comes from a knowledge base which records and makes available a characterization of the physiological state of contextualized operators, previously recorded for different operators.
• This database also constitutes a structured memorization of the experiences of a group of operators during previous missions.
• the information is structured in triads "Physiological State - Cognitive States - Contextual Situation", in a manner equivalent to that used for the individual case.
• an anonymization module allows the recorded data to be anonymized.
• the operator state analysis component 112 proceeds from the combined analysis of the instantaneous physiological state of the operator which is measured 110 and contextualized 108 by the information coming from the operated platform, and from the determination of its cognitive states according to the reference states 114.
• the purpose of this analysis is to measure the gap between the instantaneous and contextualized behavior of an operator and the previous individual and collective practice.
• the operator state analysis component 112 comprises a first comparator 212 making it possible to compare the instantaneous and contextualized state of an operator, with the individual reference states of the operator supplied by the reference state register.
• individual operator 114-1 An individual-individual, intra-personal comparison of the operator is made between his physiological state and his instantaneous individual cognitive states and his same individual states present in the knowledge base 114-1 and associated with the same contextual situation. It is a question of measuring the instantaneous behavior difference of the operator with the behaviors which he produced in the past under similar conditions.
• an individual envelope is defined from the operator's individual baseline.
• an envelope defines a field in which the states in a given situation are located: for example, if for the mental load, the appropriate model existing in the model base 202 gives a measurement between 0 and 100 (to qualify the mental load rate), in a given situation (for example: a landing at Roissy Charles de Gaulle on the alpha track with a given wind and a given aircraft in perfect working order), the individual envelope will be "mental load between 55 and 67".
• the first individual-individual comparator 212 outputs information on the instantaneous state of the operator. If all the variables are included in their respective envelopes, the comparator output indicates that there is nothing to report, and the knowledge databases are consolidated directly. If one or more variables are outside their respective envelopes, a second comparative analysis is made on the variables compared to collective envelopes.
• the operator state analysis component 112 comprises a second individual-collective comparator 214 making it possible to compare the data coming from the first comparator 212 with collective reference states provided by the collective reference state register 114-2. .
• An interpersonal comparison with other operators is made between the instantaneous individual physiological state of the operator and the individual physiological and cognitive states of different operators, present in knowledge base 114-2 and associated with the same contextual situation. This is to measure the instantaneous behavioral deviation of an operator with behaviors produced in the past by other operators under similar conditions.
• a collective envelope is defined from the operator collective reference states. The result of the comparison provides a distance from the operator state to the collective envelope, which determines the restitution provided.
• the second comparator 214 outputs the results of the comparative analysis of the individual data to the collective data. If, despite the need for this second comparison, ultimately the operator states are within the collective global envelope, the output of comparator 214 indicates that there is nothing to report, and the knowledge databases are consolidated directly.
• the device makes it possible to give a feedback in real time to the operator to signal it to him, and to indicate to him that he must rectify operational parameters (its trajectory, speed, angle of attack, engine power, tilt, etc.) or non-operational parameters (going to rest, handing over, putting on autopilot, etc.).
• the restitution component 116 coupled to the analysis module 112 disseminates the results from the comparator (s) 212, 214, to the operator himself, to offer him (via a suitable interface) services adapted and / or personalized 122, and / or to other external systems 120 which can exploit the results for performance or security purposes.
• the information disseminated is adapted to the type of reproduction envisaged (on the content and the form given to the content).
• various services are likely to be offered, such as, for example: information and indicators on the instantaneous physiological state; information on the (intra-individual) comparison of the instantaneous physiological state with the history under the same conditions; alerts on intrinsically abnormal physiological states (incapacity, hypovigilance, signs of falling asleep, etc.) or alerts on contextual situations requiring established physiological states that are not observed (vigilance, wakefulness, etc.); recommendations for action on the various aspects associated with physiological states (for example, if there are signs of falling asleep, recommendation for rest); recommendations for action on various aspects associated with both physiological states and operational situations (for example, recommendation of training or training on certain aspects or on certain operational phases); indications of actions to be carried out if a procedure which was to be initiated is not, or if the physiological state of the operator is divergent, or if the operator encounters a problem that he does not have the habit of solving (for example, reminder of the landing procedure in civil aviation, or advice
• Countermeasures can be implemented by systems external to the device to overcome these detected anomalies and ensure the safety of the mission and of its performance;
• alerts to potential co-operators on the failure by the main operator to follow the theoretical procedures to be implemented in a given situation are enriched by virtue of the collective capitalization module which recorded (anonymously) the reactions of the various operators faced with similar situations.
• the results of the combined individual-individual and individual-collective analysis from the state analysis component 112 are further used as individual and collective capitalization data in the I / C capitalization component 118.
• the I / C capitalization component 118 comprises an individual capitalization module 118-1 and a collective capitalization module 118-2.
• the individual capitalization module 118-1 uses the personal operator and context elements acquired during the execution of a mission and inserts them into the knowledge base 114-1 of the individual reference statements of the operator. It is configured to extract relevant information from individual state data and context characterization elements, and combine them into the individual knowledge base.
• the individual capitalization component is configured to filter only the significant personal and contextual information.
• the significant information can be: during take-off and landing (phases considered as critical), it is necessary to extract more parameters than during cruising phases where it is sufficient to follow only a few parameters to know if the pilot is in the nails or not.
• the collective capitalization module 118-2 is configured to perform a similar capitalization function, that is to say to extract the relevant information from the data of collective statements and from the elements of characterization of the context, and to associate them in the collective knowledge base. It is also configured to carry out an anonymization of the data, the inter-individual comparison not needing to have the identification of the operators, since it is a set of behaviors carried out in the past by a set. operators in similar situations.
• FIG. 3 illustrates an alternative embodiment of the device of the invention according to FIG. 2 in which the state analysis component 112 comprises or is coupled to a component 302 for selecting instantaneous states making it possible to select from among l 'set of instantaneous states, a subset of instantaneous states to be addressed directly to second comparator 214 in addition to the output of first comparator 212.
• the component 302 is used to develop a representation of the cognitive state resulting from the representation level models compatible with the "operator collective states and references" which may have a reduced level of detail compared to the level of detail of the individual operator reference states, this because the collective level would have led to work with an abstraction or a lower level of detail (data optimization, less sensitive individual data, etc.) than at the individual level.
• Figure 4 a method 400 for implementing the device of the invention illustrated in Figures 1 or 2, according to one embodiment.
• the method is implemented by computer to aid in monitoring the states of an individual operating on a complex system during a mission. It includes at least steps of:
• the method is implemented at the start of a mission 402 on a platform 104 for an operator 102 equipped with different sensors 106.
• a mission can be training on a simulated system or an operational mission on a real system.
• the process makes it possible to record two types of data: physiological and psychophysical data specific to the operator, and contextual or operational data specific to the current mission.
• the method makes it possible to determine, from the operational data, the contextualization parameters of the mission (step 404), and from the data specific to the operator, an individual operator instantaneous state (step 406).
• the method makes it possible, on the basis of the individual operator instantaneous state and of the contextualization parameters, to carry out an analysis of intra-personal conformity of the operator's individual state with respect to his own reference states, collected previously and recorded in a database of individual operator reference states 114-1.
• a next step 410 the method makes it possible to determine whether the results of the intra-personal analysis should or should not generate an alert signifying that the instantaneous state of the operator during the mission is not or at contrary is or remains in acceptable conformity with its own individual reference states. If there is no alert generated, the process allows 411 to capitalize on the analysis performed. If the operator states are such that an alert is to be generated, the method continues to the next step 412 to perform interpersonal analysis 412 from the individual operator state to collective, collected reference states. previously and recorded in a database of collective operator reference reports 114-2.
• a next step 414 the method makes it possible to determine whether the results of the interpersonal analysis should or should not generate an alert 410 signifying that the instantaneous state of the operator during the mission is not or at contrary is or remains in acceptable conformity with collective reference states. If there is no alert generated, the process allows 415 to capitalize on the analysis performed. If the operator's states are such that an alert should be generated, the method continues to the next step 416 to render the results back to the operator and to other rendering systems. The process continues for the duration of the mission.
• FIG. 6 illustrates the main steps for operating the method of the invention on the device of FIG. 5, where the state register comprises a module of individual reference states 114-1, a module of states of collective reference 114-2 and a module of theoretical reference states 114-3.
• the method 600 differs from the method 400 in that the analysis of the operator state 610 takes into account in addition to the analysis of intra-personal compliance and the analysis of interpersonal compliance, a theoretical compliance analysis .
• the method Before the step of restoring the results of the analysis, the method carries out a theoretical conformity analysis consisting in comparing data which are collected and characterizing instantaneous tasks performed by the operator, with theoretical reference data, said theoretical reference data being associated with the characterization of actions (manipulation, observation, communication, etc.) and their sequence (example procedure monitoring) as described in a state of the art of the management of the situation encountered (reference manuals for example).
• the method takes into account the parameters of individual instantaneous tasks performed by the operator (608) . These parameters are signatures representative of the tasks performed by the operator.
• the theoretical conformity analysis is carried out in relation to theoretical reference states or tasks (representing the state of the art and the theoretical constraints imposed for the field of application of the invention, on the analyzed situation ).
• the theoretical compliance analysis is done on the basis of the contextualization parameters and the individual instantaneous tasks of the operator which are recorded and identified 608.
• the proposed solution carries out individualized longitudinal monitoring of the operator on one or more parameters over time, with several facets: physiological monitoring and psychophysical monitoring to define the operator's perceptual-motor style.
• the operator's personal assistant is configured to provide the operator with feedback which takes into account (1) the operator's behavioral history and the various situations that have arisen in the past during the period. execution of equivalent missions, and therefore makes it possible to individualize the diagnosis on the variation in behavior detected; and (2) the behavior history of a set operators in different situations that have arisen in the past during the performance of equivalent missions, and therefore makes it possible to compare individual behavior with a set of (anonymized) behaviors and thus replace the diagnosis in relation to a general practice of execution of the mission.
• the proposed solution therefore proves to be both individualized and capable of referring to general practice.
• the proposed solution makes it possible to provide a flexible and adapted restitution:

Landscapes

• Health & Medical Sciences (AREA)
• Engineering & Computer Science (AREA)
• Life Sciences & Earth Sciences (AREA)
• Public Health (AREA)
• Medical Informatics (AREA)
• General Health & Medical Sciences (AREA)
• Pathology (AREA)
• Psychiatry (AREA)
• Physics & Mathematics (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• Developmental Disabilities (AREA)
• Primary Health Care (AREA)
• Heart & Thoracic Surgery (AREA)
• Physiology (AREA)
• Molecular Biology (AREA)
• Surgery (AREA)
• Animal Behavior & Ethology (AREA)
• Cardiology (AREA)
• Epidemiology (AREA)
• Veterinary Medicine (AREA)
• Data Mining & Analysis (AREA)
• Child & Adolescent Psychology (AREA)
• Social Psychology (AREA)
• Psychology (AREA)
• Hospice & Palliative Care (AREA)
• Educational Technology (AREA)
• Business, Economics & Management (AREA)
• Emergency Management (AREA)
• Databases & Information Systems (AREA)
• General Physics & Mathematics (AREA)
• Pulmonology (AREA)
• Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
• Debugging And Monitoring (AREA)
• Electrotherapy Devices (AREA)
• Apparatus For Radiation Diagnosis (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=72470395

Family Applications (1)

Country Status (4)

Families Citing this family (1)

• 2019
  • 2019-12-20 FR FR1915173A patent/FR3104935B1/fr active Active
• 2020
  • 2020-12-17 AU AU2020407832A patent/AU2020407832A1/en active Pending
  • 2020-12-17 EP EP20835760.8A patent/EP4078609A1/de active Pending
  • 2020-12-17 WO PCT/EP2020/086653 patent/WO2021122900A1/fr unknown

Also Published As

Similar Documents

Legal Events