FR3104935A1 - PROCEDURE AND ASSISTANCE DEVICE FOR MONITORING THE COGNITIVE CONDITIONS OF AN INDIVIDUAL - Google Patents

Abstract

A computer-implemented device and method for assisting in tracking the states of an individual operating on a complex system during a mission. The method comprises at least steps for: regularly collecting during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator; - determine from the data collected, on the one hand the contextualization parameters, and on the other hand an individual instantaneous state of the operator; - carry out an analysis of intra-personal conformity in relation to one's own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator; - carry out an analysis of interpersonal conformity in relation to previous collective reference states, based on the results of the analysis of intra-personal conformity; and - restore the results of the analyses. Figure for the abstract: Fig. 1. .

Description

METHOD AND DEVICE FOR AIDING THE MONITORING OF THE COGNITIVE STATES OF AN INDIVIDUAL

The invention relates to the general field of aid systems for monitoring the states of individuals, and in particular proposes a method and a device making it possible to carry out longitudinal and contextualized individual monitoring of the cognitive states of individuals.

The invention thus covers several fields, namely psychophysics, neurophysiology and the analysis of cognitive behavior, in order to solve at least the technical problem of comparing the psychophysical and physiological state of a healthy individual at a given moment, i.e. his states on a cognitive level covering without limitation his states of vigilance, fatigue, drowsiness, among others, to these same states obtained in close or even similar contexts. The invention also addresses a second related and underlying technical problem which is to construct an individualized model of the cognitive states of a healthy individual.

• Suivi longitudinal individuel : suivi sur le temps long d’une ou plusieurs variables associées d’un opérateur, qu’elles soient brutes (par exemple le rythme cardiaque) ou calculées (par exemple la charge mentale).

In this description, the following terms are defined:
- Operator: individual, subject, person, human in charge of operating (ie piloting, controlling, activating) a complex system. It can be a single person or a team (we then speak of a crew) of several members. The complexity of a system is also dependent on the complexity of the team (or crew) considered.

• Individual longitudinal monitoring: long-term monitoring of one or more variables associated with an operator, whether raw (eg heart rate) or calculated (eg mental load).

To date, the cognitive states of operators are evaluated on aspects that have been studied in the laboratory (for example, the case of falling asleep) and in relation to averaged results measured on a cohort of healthy and voluntary participants. However, these cognitive states are not evaluated in relation to the intrinsic and personal characteristics of these participants.

Furthermore, cognitive states are assessed within a general framework of a mission that is/must be performed (e.g. driving) and not in relation to an instantaneous context or an instantaneous situation of a mission (e.g. engaged phase of flight such as take-off or driving on the highway, etc.).

Thus, existing solutions have at least the following shortcomings:
- They do not take into account the individual aspect of longitudinal follow-up. By making comparisons only against general standards acquired on cohorts of subjects, the existing solutions do not take into account the fact that the behavior of an operator is very different from that of another operator with a fixed mission and context. but can be considered stable over time under nominal conditions.
- They do not take into account the elements of contextualization of a mission: whether it is the type of mission or a specific action to be carried out at a given moment of the mission; conversely, the existing solutions are essentially based on general results.
- They only take into account a limited base of physiological and psychophysical parameters (some solutions only use eye tracking or only the heart rate), thus not allowing a truly relevant result to be achieved.
- However, among the cognitive states that have a major interest in many fields that are interested in monitoring the behavior of the individual, there is mental load, vigilance and perceptual-motor style. The perceptual-motor style defines the way a subject interprets and combines his sensory inputs which are visual, vestibular and proprioceptive information to generate an adapted motor behavior. This integration of perceptual signals, in the same way as mental load and vigilance, has a great inter-individual variability, and this is what constitutes the technical problem of constructing an effective individualized modeling of the cognitive states of a healthy individual. On the other hand, just like behavior, the integration of perceptual signals can be considered stable over time under nominal conditions.

Also, an object of the invention is to meet the aforementioned needs and to overcome the drawbacks of existing techniques.

The invention proposes a solution based on the longitudinal and contextualized monitoring of an operator (allowing to acquire personalized knowledge of the latter) thanks to the interaction of two components:
- an individual component called “Personal Operator Assistant” or (APO); combined with
- an anonymized operator database.

Several aspects contribute to the implementation of the invention, which are:
- modeling of the perceptual-motor style;
- physiological monitoring; And
- a psychophysical follow-up.

These different aspects allow the personal operator assistant to be a device capable on the one hand of supporting an operator during his mission (for example during difficult or infrequently encountered situations), and on the other hand of accompanying the operator outside his mission. An example of off-mission support can be, for example, guiding the continuous training of the operator according to the observations made during the mission, and returned by the APO. Another use can be to replay the scenario of a mission in a simulator by adapting it according to the situations and the reactions occurring during the said mission, and returned by the APO. The assistant can be used to optimize the well-being or "well-being" of the operator by proposing, for example, sleep/wake schedules, meals, meal contents, episodes of physical training, for example to manage jet lag or very prolonged standby.

Finally, on the scale of a set of operators (at the level of a company or an organization for example), the data of the personal operator assistants can be anonymized in order to create a knowledge database which can be an element of comparison and reference for the personal operator assistant.

• des expériences passées de l’opérateur, colligées sur l’APO.

The general principle of individual and contextualized longitudinal monitoring is that during each training session on a simulated system and each mission on a real system, two types of data are recorded: physiological data specific to the operator (such as heart rate) and contextual or operational data specific to the mission (instructions, tasks performed, manipulation of the man-machine interface, machine state variables). All of this data is then aggregated to provide personal assistance in real time, which is specific to each operator depending on:
- variables that describe the current situation; And

• past experiences of the operator, collected on the APO.

For example, this assistance provides the operator with operational support in case of need (decrease in alertness, signals of onset of falling asleep, increase in the level of mental load, deviation from the perceptual-motor style, etc.) by a comparison between the operator's instantaneous cognitive state and his previous cognitive states in similar contexts. A crucial point, for the operator to establish a relationship of trust with its APO and to meet legal obligations concerning medical data, all the data collected on the APO and the results of their processing by dedicated algorithms are covered. by medical secrecy: the operator is the owner and no one other than him can consult them without having given his prior authorisation. On the other hand, he cannot transfer them to an individual or to any other third party organization.

At the same time, all data collected by the APO is anonymized and aggregated in a common database or knowledge base (BCP). This database serves as a reference for the personal operator assistant to compare the operator's situation at a given moment with all similar or even identical situations (same conditions, same operational situation, etc.). This process can be described as a mise en abyme of the APO thanks to the BCP. This mise en abyme makes it possible to evaluate the current behavior of the operator and, if necessary, to offer decision-making aids, reconfigurations of the human-machine interface, etc. adapted to the present situation.

The present invention will find many fields of application where there is a need to follow and support complex system operators throughout their career. The fields of operation of the invention are the fields of activity in which a complex system is controlled and implemented by one or more operators. These business areas include, for example, industry and aeronautical, rail, road transport, security areas, process control areas, etc.

One of the main areas sensitive to the issue of monitoring an operator's behavior, whether for safety or performance issues (i.e. reducing the number of operators for a given task, known as "Single Pilot Operation ") is the field of aeronautics with the definition of the concept of "crew monitoring system" or "Crew Monitoring System" in English. This type of system aims to monitor the behavior of each member of the crew and to warn the operators themselves and the complex system of variations in the behavior of the operator vis-à-vis the expected behavior or variations vis-à-vis -à-vis the behavior during previous similar missions. In the extreme, the mise en abyme of his behavior could record the exclusion of the operator from the control of the device (for example in the event of loss of consciousness or suicidal impulse, etc.) and replace him with a control device automatique.

The field of use of the invention also extends to the training of operators and the possibility of measuring and monitoring their progress during their training, in order to check their progress in learning and mastering driving. of the process for which they are responsible, and to adapt the training to their behaviour.

The field of operation of the invention is also that of the continuous training of the operator, the latter being adapted according to the needs of the operator revealed by the restitution of the personal assistant and quantified during the missions and its implementation. in perspective with data from the common knowledge base.

Thus, the device of the invention in the context of aeronautics, participates in several areas of innovation: the development of the cockpit of the future, the monitoring of pilots (in flight and off flight), the physiological and psychophysical monitoring of these or flight safety and mission performance.

The proposed device addresses the problem of how to ensure that a pilot is robust from an operational point of view. This problem is multiple and raises several technical questions, because it is indeed necessary to be able to measure several physiological parameters in the least invasive way possible, to interpret these signals, to compile them with the contextual data of the mission and to produce in output relevant information that is operationally useful for the pilot.

The advantages of the invention are multiple and located at several levels. At the operator level, the personal assistant allows better health monitoring. By longitudinally recording heart rate and respiratory rate (as well as other physiological parameters), performing regular checks of the operator's hearing, vision and possibly nychthemeron, the evolution or appearance of minor clinical signs are listed and treated before the onset of characterized disorders. It is therefore a preventive approach. Also, the individual longitudinal monitoring of the sensorimotor system makes it possible to define the notion of operational robustness and to evaluate it at regular intervals throughout the operator's career. Finally, the proposed solution allows better management of problems that may arise during the accomplishment of a mission (breakdowns, multiple actions to be carried out simultaneously, etc.).

Always advantageously, the aggregation of the physiological and psychophysical parameters of an operator also makes it possible to model individually in real time (as opposed to a delayed time after the mission), the cognitive states of the operator (fatigue, alertness, mental load) . This allows better management of delicate situations during the operation of the complex systems involved. In addition, the proposed solution, which benefits over time from the continuous capitalization of its implementation, makes it possible to improve the robustness of the models detection of the physiological and cognitive state of the operator.

To obtain the desired results, a computer-implemented method is proposed for assisting in monitoring the states of an individual operating on a complex system during a mission. The method comprises at least steps for:
-collect regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
-determine from the data collected, on the one hand the contextualization parameters, and on the other hand an individual instantaneous state of the operator;
- carry out an analysis of intra-personal conformity with respect to one's own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator;
- carry out an analysis of interpersonal conformity in relation to previous collective reference states, based on the results of the analysis of intra-personal conformity; And
- reporting the results of the analyses.

According to alternative or combined embodiments:
- the intra-personal compliance analysis step comprises a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of individual reference states for said operator, said data being associated with the characterization of the physiological state and the contextualized cognitive state of said operator.
- the interpersonal compliance analysis step comprises a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of collective reference states for a plurality of operators, the said data being associated with the characterization of the contextualized physiological state and cognitive state of the plurality of operators during missions similar to the current mission.
- the register data is structured in triads “Physiological State – Cognitive States – Contextual Situation” associated with the sequence being analyzed.
- the method further comprises, after the intra-personal conformity analysis step, a step consisting in determining whether the instantaneous state of the operator during the mission is in acceptable conformity with his own individual reference states or n is not in acceptable compliance.
- the method further comprises, after the interpersonal conformity analysis step, a step consisting in determining whether the instantaneous state of the operator during the mission is in acceptable conformity with reference collective states or is not in acceptable compliance.
- the method further comprises, after each analysis step, a step of capitalizing the results of each analysis to enrich the individual and collective reference states.
- the stage of determining an instantaneous state of the operator consists in implementing on the physiological data evaluation and prediction models associated with each physiological aspect.
- the evaluation and prediction models are based on artificial intelligence models trained to produce instantaneous individual states.
- the method further comprises, after the restitution step, a step consisting in providing the operator, via a suitable interface, with personalized services according to the results of the analysis steps.
- the method further comprises, after the restitution step, a step consisting in transmitting the results of the analysis steps to external systems.

The invention also covers a computer program product comprising code instructions for performing the steps of the claimed method, when the program is executed on a computer.

The invention also covers a device for assisting in monitoring the states of an individual operating on a complex system during a mission. The device comprises means making it possible to:
- collect regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
- determining from the collected data, on the one hand contextualization parameters, and on the other hand an individual instantaneous state of the operator;
- carry out an analysis of intra-personal conformity in relation to one's own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator;
- carry out an analysis of interpersonal conformity in relation to previous collective reference states, based on the results of the analysis of intra-personal conformity; And
- return the results of the analyses.

The device of the invention further comprises other means for implementing the steps of the method of the invention.

Other characteristics, details and advantages of the invention will become apparent on reading the description given with reference to the appended drawings given by way of example and which represent, respectively:

a general context making it possible to implement the device of the invention;

an architecture of the device of the invention according to one embodiment;

an architecture of the device of the invention according to a variant embodiment; And

a method for implementing the device of the invention according to one embodiment.

FIG. 1 shows a general environment making it possible to implement the device of the invention 100. Although not illustrated so as not to be considered as a limitation, the description of an embodiment is made for an aeronautical environment. However, those skilled in the art can apply the principles described to any other environment mentioned above such as industry and rail or road transport, the fields of security, the fields of process control, etc.

An operator 102 evolves in an environment during simulation training or a real mission, illustrated by a platform 104. The operated platform is a complex system operated by the operator such as for example an aircraft (or a vehicle or a control post, etc. from which it is possible to retrieve contextual information on the mission in progress, on the state of the complex system and on the situation of the environment (in the aeronautical context: weather conditions, state of the air traffic, etc.). In a non-limiting way, the state of the system in the context of avionics can be information relating to the states inherent in normal operation, i.e. the percentage of power of the engines, if the fins are folded or unfolded, the status of fuel tanks, etc., and information relating to states inherent in abnormal operation, i.e. the left engine is faulty, the right pedal of the co-pilot seat is faulty, or any sort of potential fault.

In one embodiment, the context is represented by two distinct elements:
- a task to be performed by the operator. The physiological and cognitive states of the operator are classified in relation to the tasks to be performed by the operator. The tasks can be of various natures: routine procedure, reaction to a given problem, etc. Each task corresponds to a physiological, psychophysical and cognitive signature, where a signature corresponds to a characteristic identified in an acquired signal and which is specific to a given operator and to a given task. For example, the signature in the acquisition of the heart rate can be an acceleration of the latter in a stressful situation (breakdown, failure, change of itinerary, strong wind during landing, etc.).
- the environment in which the complex system evolves and the task is carried out. The environment constitutes an adjustment variable for the analysis of the operator state: it makes it possible to adjust envelopes of the above-mentioned signatures according to the context. For example, when trying to characterize the signatures of a landing, the elements that come under "environment" will be: the weather (wind, etc.), the airport of arrival (even the runway used), etc. .

The environment of the operator (and/or the operator himself) is equipped with various sensors 106 making it possible to record physiological data of the operator during the training or the mission. The physiological data acquired are for example, without limitation: i) the heart rate by means of a heart rate bracelet or a chest strap; (ii) breathing rate through the same chest strap or through a sensor pad located in or on the seat; (iii) eye-tracking data collected by means of a camera located either at a distance from the pilot on the dashboard or on glasses worn by the pilot; iv) skin resistance using a heart rate monitor watch or a dedicated bracelet, v) movement sensors worn by the operator on the head, trunk, upper and lower limbs or placed on the its manipulandum: the speed and acceleration of the pedals, the stick and the joysticks.

The device of the invention 100 or personal operator assistant (APO) generally comprises a contextualization component 108, an individual operator instantaneous state component 110, an operator state analysis component 112, a state base or register individual and collective reference (I/C) 114, an individual and collective capitalization (I/C) component 116 and a restitution component 118.

The contextualization component 108 comprises means for receiving context data from the platform 104 and for supplying information, contextualization parameters. Context data may include flight data such as: route, altitude, speed, acceleration, attitude and envelope. In addition, aircraft parameters can be taken into account, such as: date of last maintenance, in-flight events, etc. Advantageously, the contextualization component is configured to take account of external data such as weather conditions. The contextualization component 108 comprises means for analyzing the information received and extracting elements of interest (such as for example the force, the direction of the wind) characteristic of a sequence during play (a sequence being a particular moment such as for example landing) and operational elements (as for example in the case of aircraft landing: how did the operator carry out the action?; what is the attitude of the aircraft, its speed, its inclination, what actions on the pedals, etc.) in relation to the behavior of the operator. In embodiments, the device makes it possible to segment a flight into relevant flight phases (restriction of the sequence in the time domain), and to segment the analysis of the actions of the operator (restriction in the operational domain).

The contextualization elements identified by the contextualization component are provided to the operator state analysis component 112. In an advantageous embodiment, the contextualization elements are also provided to the individual operator instantaneous state component 110 in order to reinforce the robustness of prediction models and make them less sensitive to false positives.

The individual operator instantaneous state component 110 receives the information produced by the sensors 106 in order to provide information on the instantaneous state of the operator. The individual operator instantaneous state component 110 includes means for processing various physiological aspects of the operator.

The information coming from the contextualization component 108 and from the individual operator instantaneous state component 110 is processed by the operator state analysis component 112 which also receives information on individual and collective reference states from the reference I/C 114, in order to deliver information resulting from the analysis. The I/C reference states component 114 comprises two modules: an individual operator reference states module 114-1 and a collective reference states module 114-2.

The restitution component 116 comprises means for distributing the analysis information to external systems 120 and/or to applications 122 dedicated to the operator 102.

The operator state analysis component 112 further provides the I/C capitalization component 118 with information resulting from the analysis. The I/C capitalization component 118 comprises two modules: an individual capitalization module 118-1 and a collective capitalization module 118-2 which aim to enrich the individual and collective knowledge bases of the reference states component I/C 114, and to carry out a continuous improvement of the models of behavior and the capacities of analysis and discernment of these models.

FIG. 2 illustrates an architecture of the device of the invention 100 according to one embodiment. The components of FIG. 1 whose function is identical retain the same references in the various figures, implementation variants being able to be produced by those skilled in the art while retaining the same functionalities. Thus, only the elements not described in FIG. 1 are described in detail. In particular, the individual operator instantaneous state component 110 comprises specific evaluation and prediction models 202, associated with each of the different physiological aspects 204 to evaluate, such as for example in a nonlimiting way, the sensorimotority, the incapacity, the alertness, the drowsiness, the mental load. In one embodiment, the evaluation and prediction models are based on artificial intelligence models trained to produce instantaneous individual states (cognitive, physiological states, etc.) as they should be if all goes well (no failure, etc.).

In a variant embodiment, the operator individual instantaneous state component processes only one physiological aspect.

The I/C reference states component 114 includes records of two types of reference states: individual operator reference states 114-1 and collective reference states of a plurality of operators 114- 2.

The Operator Individual Reference State Register 114-1 includes meaningful data associated with the characterization of an operator's contextualized physiological and cognitive state, and constitutes a knowledge base of previously recorded data. . This database constitutes a structured memorization of the experiences of the operator during his previous missions. The structuring of the individual reference register associates the physiological state and the cognitive states of the operator with a contextual situation encountered during a recorded sequence, and presents data structured in triads "Physiological State - Cognitive States - Contextual Situation" associated with the current sequence representative of the behavior of the operator.

The register of collective operator reference states 114-2 also proceeds from a knowledge base that records and makes available a characterization of the physiological state of contextualized operators, previously recorded for different operators. This database also constitutes a structured memorization of the experiences of a group of operators during previous missions. The structuring of the information is in triads “Physiological State – Cognitive States – Contextual Situation”, in a manner equivalent to that used for the individual case. In an advantageous embodiment, an anonymization module allows the recorded data to be anonymized.

The operator state analysis component 112 proceeds from the combined analysis of the instantaneous physiological state of the operator which is measured 110 and contextualized 108 by the information from the operated platform, and the determination of his cognitive states according to the reference states 114. The purpose of this analysis is to measure the gap between the instantaneous and contextualized behavior of an operator and previous individual and collective practice.

The operator state analysis component 112 comprises a first comparator 212 making it possible to compare the instantaneous and contextualized state of an operator, with the individual reference states of the operator provided by the register of individual reference states of the operator. operator 114-1. An individual-individual, intra-personal comparison of the operator is made between his physiological state and his instantaneous individual cognitive states and his same individual states present in the knowledge base 114-1 and associated with the same contextual situation. It is a question of measuring the difference in instantaneous behavior of the operator with the behaviors he has produced in the past under similar conditions. In one embodiment, an individual envelope is defined from the operator's individual reference state. An envelope in the context of the invention defines a field in which the states are located in a given situation: for example, if for the mental load, the appropriate model existing in the model base 202 gives a measurement between 0 and 100 (to qualify the mental load rate), in a given situation (for example: a landing at Roissy Charles de Gaulle on runway alpha with a given wind and a given aircraft in perfect working order), the individual envelope will be "load between 55 and 67”.

The first individual-individual comparator 212 outputs information on the instantaneous state of the operator. If all the variables are within their respective envelopes, the comparator output indicates that there is nothing to report, and the knowledge databases are consolidated directly. If one or more variables are outside their respective envelopes, a second comparative analysis is made on the variables with respect to collective envelopes.

The operator state analysis component 112 comprises a second individual-collective comparator 214 making it possible to compare the data coming from the first comparator 212 with collective reference states provided by the register of collective reference states 114-2. An interpersonal comparison with other operators is carried out between the instantaneous individual physiological state of the operator and the individual physiological and cognitive states of different operators, present in the knowledge base 114-2 and associated with the same contextual situation. It is a question of measuring the instantaneous behavior gap of an operator with behaviors produced in the past by other operators under similar conditions. In one embodiment, a collective envelope is defined from collective operator reference states. The result of the comparison provides a distance of the state of the operator with the collective envelope, which determines the restitution provided.

The second comparator 214 outputs the results of the comparative analysis of the individual data to the collective data. If, despite the need for this second comparison, ultimately the states of the operator are within the collective global envelope, the output of the comparator 214 indicates that there is nothing to report, and the knowledge databases are consolidated directly. If the states (one or more variables) of the operator are outside their respective collective envelopes, the device allows real-time feedback to the operator to inform him of this, and indicate to him that he must rectify operational parameters (its trajectory, its speed, its angle of attack, its engine power, its inclination, etc.) or non-operational parameters (going to rest, handing over, putting the autopilot on, etc.).

The restitution component 116 coupled to the analysis module 112 broadcasts the results from the comparator(s) 212, 214, intended for the operator himself, to offer him (via a suitable interface) suitable services and/or or personalized 122, and/or to other external systems 120 that may use the results for performance or security purposes. Advantageously, according to the embodiments, the information disseminated is adapted to the type of restitution envisaged (on the content and the form given to the content).

In the case of an exploitation of the results by an application 122 intended for an operator, various services are likely to be offered, such as for example:
- information and indicators on the instantaneous physiological state;
- information on the (intra-individual) comparison of the instantaneous physiological state with the history under the same conditions;
- alerts on intrinsically abnormal physiological states (incapacity, hypovigilance, signs of drowsiness, etc.) or alerts on contextual situations requiring established physiological states not respected (vigilance, awakening, etc.);
- recommendations for action on the various aspects associated with physiological states (for example, if there are signs of falling asleep, recommendation for rest);
- recommendations for action on different aspects associated with both physiological states and operational situations (for example, recommendation of training or practice on certain aspects or on certain operational phases);
- indications of actions to be carried out if a procedure which was to be initiated is not, or if the physiological state of the operator is divergent, or if the operator encounters a problem that he does not have. habit of solving (for example, reminder of the landing procedure in civil aviation, or advice of the actions to be taken in the event of a technical failure of one of the components of the aircraft);
- indications on the theoretical procedures which apply to a given situation, according to the state of the complex system such as measured by the sensors. In one embodiment, these indications can be enriched thanks to the collective capitalization module which has recorded (anonymously) the reactions of the various operators faced with similar situations.

The person skilled in the art understands that this list of services is not exhaustive, and that it can be enriched with other specific options dedicated to the context of exploitation of the invention.

In the case of exploitation of the results by an external system 120, various services can be offered, such as for example:
- alerts on short-term or observed states of incapacity of the operator, resulting in a safety problem for the rest of the mission, intended for other operators or control services;
- alerts on states of behavior (physiological states and cognitive states) of the operator outside the behaviors expected during the phases of the mission (vigilance, drowsiness, mental load) that can alter (endanger) the safety of the mission , or just the runtime performance of it. “Countermeasures” can be implemented by systems external to the device to remedy these detected anomalies and ensure the safety of the mission and its performance;
- alerts intended for potential co-operators on the main operator's failure to follow the theoretical procedures to be implemented in a given situation. In one embodiment, these alerts are enriched thanks to the collective capitalization module which has recorded (anonymously) the reactions of the various operators faced with similar situations.

The person skilled in the art understands that this list of services is not exhaustive, and that it can be enriched with other specific options dedicated to the context of exploitation of the invention.

The combined individual-individual and individual-collective analysis results from the state analysis component 112 are further used as individual and collective capitalization data in the I/C capitalization component 118.

In the embodiment described for FIG. 2, the I/C capitalization component 118 comprises an individual capitalization module 118-1 and a collective capitalization module 118-2.

The individual capitalization module 118-1 uses the personal operator and context elements acquired during the execution of a mission and inserts them into the knowledge base 114-1 of the operator's individual reference states. It is configured to extract relevant information from individual state data and context characterization elements, and associate them in the individual knowledge base. Advantageously, the individual capitalization component is configured to filter only the significant personal and contextual information. Thus, for example, in aeronautics, the significant information can be: during take-off and landing (phases considered to be critical), it is necessary to extract more parameters than during the cruising phases where it suffices to follow only a few parameters to know if the driver is in the nails or not.

The collective capitalization module 118-2 is configured to perform a similar capitalization function, i.e. to extract the relevant information from the collective state data and from the characterization elements of the context, and associate them in the database. collective knowledge. It is also configured to carry out an anonymization of the data, the inter-individual comparison not needing to have the identification of the operators, since it is a set of behaviors carried out in the past by a set operators in similar situations.

FIG. 3 illustrates a variant embodiment of the device of the invention according to FIG. 2 in which the state analysis component 112 comprises or is coupled to a component 302 for selecting instantaneous states making it possible to select from among the set of instantaneous states, a subset of instantaneous states to be addressed directly to the second comparator 214 in addition to the output of the first comparator 212.

The component 302 is used to develop a representation of the cognitive state resulting from the models of representation level compatible with the "states and collective references operators" which can have a reduced level of detail compared to the level of detail of the individual operator reference states, this because the collective level would have led to working with an abstraction or a lower level of detail (data optimization, less sensitive individual data, etc.) than at the individual level.

FIG. 4 a method 400 for implementing the device of the invention according to one embodiment. In general, the method is implemented by computer to aid in monitoring the states of an individual operating on a complex system during a mission. It includes at least steps of:
- collect regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
- determining from the collected data, on the one hand contextualization parameters, and on the other hand an individual instantaneous state of the operator;
- carry out an analysis of intra-personal conformity with respect to one's own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator;
- carry out an analysis of interpersonal conformity in relation to previous collective reference states, based on the results of the analysis of intra-personal conformity; And
- return the results of the analyses.

In more detail, the method is implemented at the start of a mission 402 on a platform 104 for an operator 102 equipped with various sensors 106. A mission can be training on a simulated system or an operational mission on a real system. The process makes it possible to record two types of data: physiological and psychophysical data specific to the operator, and contextual or operational data specific to the mission in progress.

The method makes it possible to determine, from the operational data, the contextualization parameters of the mission (step 404), and from the data specific to the operator, an instantaneous individual operator state (step 406).

In a next step 408, the method makes it possible, from the instantaneous individual operator state and the contextualization parameters, to perform an analysis of intra-personal conformity of the individual state of the operator with respect to his own reference states , collected previously and recorded in a base of individual operator reference states 114-1.

In a next step 410, the method makes it possible to determine whether or not the results of the intra-personal analysis must generate an alert signifying that the instantaneous state of the operator during the mission is not or on the contrary is or remains in acceptable conformity with its own individual reference states. If there is no alert generated, the process allows 411 to capitalize on the analysis performed. If the states of the operator are such that an alert must be generated, the method continues to the next step 412 to carry out an interpersonal analysis 412 of the individual state of the operator to collective reference states, collected previously and recorded in a base of collective operator reference states 114-2.

In a next step 414, the method makes it possible to determine whether or not the results of the interpersonal analysis must generate an alert 410 signifying that the instantaneous state of the operator during the mission is not or on the contrary is or remains in acceptable conformity with collective reference states. If there is no alert generated, the method allows 415 to capitalize on the analysis performed. If the states of the operator are such that an alert should be generated, the method continues to the next step 416 to render the results to the operator and to other restitution systems. The process continues throughout the duration of the mission.

A solution has thus been described to assist an operator throughout his career, whether during his training periods or during his operational missions. The proposed solution carries out an individualized longitudinal follow-up of the operator on one or more parameters over time, with several facets: a physiological follow-up and a psychophysical follow-up making it possible to define the perceptual-motor style of the operator.

The personal operator assistant is configured to provide the operator with feedback that takes into account (1) the operator's behavior history and different situations that have occurred in the past during the execution of missions equivalent, and therefore makes it possible to individualize the diagnosis on the variation in behavior detected; and (2) the behavior history of a set of operators in different situations that have arisen in the past during the execution of equivalent missions, and therefore makes it possible to compare the individual behavior to a bundle of behaviors (anonymized) and thus place the diagnosis in relation to a general practice of carrying out the mission.

The proposed solution therefore turns out to be both individualized and capable of referring to a general practice. In addition, the proposed solution makes it possible to provide a modular and adapted restitution:
- to the mission;
- the context in which the assignment is carried out;
- to the operator and his needs;
- to the crew if necessary, and therefore ensures better coordination.

This description illustrates a preferred implementation of the invention, but which is not limiting. Examples are chosen to allow a good understanding of the principles of the invention and a concrete application, but are in no way exhaustive and should allow those skilled in the art to make modifications and implementation variants while retaining the same principles.

Claims (14)

A computer-implemented method for assisting in tracking the states of an individual operating on a complex system during a mission, the method comprising at least steps for:
- collecting (402) regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
- determining from the collected data, on the one hand (404) contextualization parameters, and on the other hand (406) an individual instantaneous state of the operator;
- performing an analysis of intra-personal conformity (408) with respect to its own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator;
- performing an interpersonal compliance analysis (412) with respect to previous collective reference states, based on the results of the intrapersonal compliance analysis; And
- restoring (416) the results of the analyses. The method according to claim 1 wherein the step of intra-personal compliance analysis comprises a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of individual reference states for said operator, said data being associated with the characterization of the contextualized physiological state and cognitive state of said operator. The method according to claim 1 or 2 wherein the step of analyzing interpersonal compliance includes a step consisting in comparing data characterizing the individual instantaneous state of the operator with data recorded in a register of reference states collective for a plurality of operators, said data being associated with the characterization of the contextualized physiological state and cognitive state of the plurality of operators during missions similar to the mission in progress. The method according to claims 2 or 3 in which the register data is structured in triads "Physiological State - Cognitive States - Contextual Situation" associated with the sequence being analyzed. The method according to any one of the preceding claims further comprising, after the intra-personal compliance analysis step, a step 410 consisting in determining whether the instantaneous state of the operator during the mission is in acceptable compliance with its own individual reference states or is not in acceptable compliance. The method according to any one of the preceding claims further comprising, after the interpersonal compliance analysis step, a step 414 consisting in determining whether the instantaneous state of the operator during the mission is in acceptable compliance with states reference or is not in acceptable compliance. The method according to any one of the preceding claims further comprising, after each analysis step, a step of capitalizing the results of each analysis to enrich the individual and collective reference states. The method according to any one of the preceding claims, in which the step of determining an instantaneous state of the operator consists in implementing on the physiological data evaluation and prediction models associated with each physiological aspect. The method of claim 8 wherein the assessment and prediction models are based on artificial intelligence models trained to produce instantaneous individual states. The method according to any one of claims 1 to 9 further comprising, after the restitution step, a step consisting in providing the operator, via a suitable interface, with personalized services according to the results of the analysis steps. The method according to any one of claims 1 to 10 further comprising, after the restitution step, a step consisting in transmitting the results of the analysis steps to external systems. A device for assisting in monitoring the states of an individual operating on a complex system during a mission, the device comprising means making it possible to:
- collect regularly during the mission, data relating to the state of the complex system and its environment, data relating to the context of the mission, and physiological data relating to the operator;
- determining from the collected data, on the one hand contextualization parameters, and on the other hand an individual instantaneous state of the operator;
- carry out an analysis of intra-personal conformity in relation to one's own previous individual reference states, based on the contextualization parameters and the individual instantaneous state of the operator;
- carry out an analysis of interpersonal conformity in relation to previous collective reference states, based on the results of the analysis of intra-personal conformity; And
- return the results of the analyses. The device according to claim 12 further comprising means for carrying out the steps of the method according to any one of claims 2 to 11. Computer program comprising code instructions for executing the steps of the method for assisting in monitoring the states of an individual operating on a complex system during a mission, according to any one of Claims 1 to 11, when said program is executed by a processor.