WO2007012723A2 - Method and system for modelling an interface between a user and the environment thereof in a motor vehicle - Google Patents

Abstract

The invention relates to a method for determining a model of interface between a user and the environment thereof in a motor vehicle consisting in creating (E1) an interface model according to the first type of information items representing the vehicle interface elements and a second type of information items representing knowledge of the user about the use of the interface elements, in acquiring (E2) data items representing at least one human activity required for the interaction between the user and the interface elements, wherein said data acquisition is carried out by means of at least one data acquisition device, in analysing (E3) the thus obtainable data and in adjusting (E4) the interface model according the data analysis.

Description

 Method and system for modeling an interface between a user and his environment in a vehicle

The invention relates to a method and a system for determining a model of an interface between a pilot and his environment on board a vehicle.

In different sectors (aeronautics, automobile, maritime ...), air, land or maritime vehicles require, for their use

(piloting or driving, navigation, communication, environmental monitoring, systems management ...) instrument panels having a plurality of interface elements.

To carry out its task, the user of the vehicle in question, piloting interface elements, must be perfectly familiar with the functions performed by these interface elements, the information they deliver, as well as the procedures describing sequences of interfaces. actions (manual, visual, auditory) to be performed in relation to the interface elements.

It is thus understood that, when driving a vehicle, the interaction between the user and the interface elements disposed on the vehicle is of great importance and is therefore the subject of much attention.

It would therefore be interesting to be able to evaluate this interaction in a new and effective way in order, for example, to be able to improve existing interface elements, to design new ones, or to improve flight procedures, or to improve the arrangement of several interface elements relative to each other.

For this purpose, the subject of the present invention is a method for determining a model of an interface between a user and his environment on board a vehicle, characterized in that it comprises the following steps: - development of an interface model from, on the one hand, a first type of information representative of vehicle interface elements and, on the other hand, a second type of representative information knowledge held by a user about the use of interface elements,

- Acquisition of data representative of at least one human activity that is requested during the interaction between the user and these interface elements, the acquisition of these data being performed via at least one device. 'data acquisition,

- analysis of the data thus acquired,

- adjustment of the interface model according to the analysis of the data.

The interface model is developed on the basis of the technical user-system duality and not only on the technical information representative of the system, which makes it possible to obtain a very reliable model built on a set of structured information between them. a way that is particularly representative of the interaction between the user and his environment on board the vehicle, including the interface elements of the latter.

Thanks to the recorded data which translate the visual and / or gestural and / or vocal and / or physiological behavior of the user in relation to the interface elements and to the interpretation of these data, it is possible to enrich the model interface previously developed and therefore to adjust closer to the context it must represent.

For example, it is possible to detect operating anomalies in the interface elements, to evaluate new interface elements, to determine whether an interface element should deliver certain information or perform certain functions, or to determine that A new interface element that performs one or more given functions would be particularly useful. According to one characteristic, the two types of information, the first type of information of technical origin and the second type of information of human origin, are provided, with identical configuration, to a dynamic database having a structure symmetric user-technical system.

Insofar as the information of both types is always provided in the same single format, it results in a saving of time and efficiency in the processing of this information and thus in the development of the model.

According to one characteristic, the information of the two types is configured according to the same multi-agent cognitive model.

Such a representation of the information proves particularly suitable and effective for developing the interaction model.

According to one characteristic, the configuration of the information of the first type according to a cognitive multi-agent model comprises a step of establishing a link between the procedures for using the vehicle and the interface elements of the vehicle.

This establishes a correspondence between the various stages of the vehicle utilization procedures (for example, piloting) and the interface elements involved at each stage in order to model them.

According to one characteristic, the configuration of the information of the first type according to a multi-agent cognitive model comprises a step of identifying functional areas on each interface element considered. By defining such areas within the same interface element, it will be possible to obtain a detailed model of each interface element and thus to obtain subsequently, during the step of data acquisition, detailed information about the interaction between the user (eg the driver) and the zones, or even several areas of different interface elements. The model will thus be even more complete and therefore more reliable by acquiring, for example, eye-tracking data relating to these areas of the same interface element or of several of them. According to one characteristic, the configuration of the information of the first type according to a multi-agent cognitive model comprises for each interface element the following steps:

- determination of the tasks relating to the use of the vehicle filled by the considered interface element,

- determination of the agents of the multiagents cognitive model with respect to the determined tasks,

establishing a link between the agents of the cognitive model thus determined and the identified functional areas of the interface element in question.

The model thus established is particularly representative of the interaction of the user (eg pilot) with the considered interface element, taking into account the tasks assigned to the interface element and which are determined, for example, by the procedures for using the vehicle (eg driving).

According to one characteristic, the human activity solicited during the interaction between the user and the interface elements is selected from vision, speech, hearing, motor skills, physiological manifestations and reactions of the human body. The acquisition and analysis of data reflecting a wide variety of human activities provides very useful information for, for example, completing / modifying the interaction model.

According to one characteristic, the data acquisition apparatus is an eye-tracking apparatus recording visual data representative of the user's visual journey on the interface elements.

Such a device is particularly useful for describing the visual behavior of the user (eg pilot) when his gaze travels through different interface elements as well as the external visual, or even particular areas within one or more elements interface. This device can be coupled with another device for recording, for example, in video form the actions of the pilot, while the position of his gaze is followed by the first device. Records audio can also be very useful. As a result, there is more data to process and a wider variety of data, which makes it possible to enrich the interface model and make it even more faithful to the context it has to represent. The interface model determined as briefly described above has applications in many fields (aeronautics, space, automobile, maritime ...) and can be used in many applications:

- improvement of one or more interface elements;

- design of one or more interface elements; - evaluation of one or more new interface elements;

- modification of a procedure of use (for example, piloting) of the vehicle;

- training of users (for example, pilots) in driving the vehicle. The invention also relates to a system for determining a model of an interface between a user and his environment on board a vehicle, characterized in that it comprises:

means for developing an interface model from, on the one hand, a first type of information representative of vehicle interface elements and, on the other hand, a second type information representative of the knowledge held by a user on the use of the interface elements,

at least one apparatus for acquiring data representative of at least one human activity that is requested during the interaction between the user and these interface elements,

- means of analysis of the data thus acquired,

means for adjusting the interface model according to the analysis of the data.

This system has the same aspects and advantages as those presented above with respect to the process and will not be recalled here. Other characteristics and advantages will become apparent from the following description, given solely by way of nonlimiting example and with reference to the appended drawings, in which:

FIG. 1a generally represents an algorithm of the method for determining an interface model according to the invention

- Figure 1b schematically shows a system for implementing the method according to the invention;

FIG. 2 is a schematic representation of the process of developing the interface model according to the invention;

FIG. 3 schematically represents an algorithm detailing the steps illustrated on the algorithm of FIG. 1a;

FIG. 4 is a table illustrating the correspondence between a flight procedure and the flight instruments used at each stage of the procedure;

FIG. 5 illustrates the identification of different information zones on an on-board instrument;

FIG. 6 schematically illustrates the functions assigned to the zones defined in FIG. 5; FIG. 7 represents in tabular form the connection between the agents of the cognitive model and the functional areas of the instrument panel illustrated in FIG. 5;

FIG. 8 illustrates an example of drawing up tables 16 and 18 of FIG. 1a. The invention finds a particularly interesting application in aeronautics, particularly in the modeling of the interface elements of an aircraft cockpit.

In an aircraft cockpit, there are several types of instrumentation panels, for example, the main instrument panel IP ("main instrument pane!" In English terminology) on which are arranged several instruments playing the role of interface elements for the pilot called PF ("Pilot Flying" in English terminology) and the co-pilot called PNF ("Pilot Non Flying" in English terminology), namely, for example, the instrument called PFD or "Primary Flight Display" (Anglo-Saxon terminology) and the instrument called ND or navigation display ("Navigation Display" in English terminology). There is also a central panel CP ("central paner in English terminology), a top panel OP (" overhead paner in English terminology) and a panel below the windshield GS ("glareshield paneF 'in English terminology).

The user of the cockpit, namely the pilot, uses all the interface elements of the abovementioned instrumentation panels for piloting the aircraft, for navigation tasks, as well as for protection tasks for the aircraft. maintain the aircraft in the flight area.

In order to facilitate the execution of these tasks by the pilot and to enable him to exercise his activity with maximum safety, it has proved useful to determine a model of interface between the pilot and his environment aboard the aircraft. 'plane.

The algorithm of FIG. 1a illustrates the main steps of the method according to the invention for determining such a pilot-cockpit interface model. This algorithm is executed by a computer operating in cooperation with data storage means / information (databases, memories ...).

During a first step E1, it is planned to develop a cockpit interface model based on two types of information, a first type of information, relating to the technical system and more particularly representative of data elements. cockpit interface, and a second type of human-related information, and in particular, representative of pilot knowledge of the use of cockpit interface elements and flight procedures, as well as of his behaviors (airplane pilot experience). The pilot-cockpit interaction is interface-based, with a dynamic character, including user and technical system behaviors.

This step relies on the use of both technical and human origin information in order to take into account the user-system pair during the development of the interaction model.

As illustrated in FIG. 2, the information of the two aforementioned types is provided to a dynamic database 10 having a pilot symmetric structure (human) -technical system with respect to the interaction axis separating the part of the base relating to the human aspect 12 and that relating to the technical aspect 14.

It should be noted that the information is poured into this database in a structured way according to an identical configuration which is defined, according to each aspect (human and technical), by an input-output level detailing all the inputs and outputs used and by a level of treatment detailing the different subsystems used.

The model is developed starting with the identification of inputs and outputs on the human side and the technical system side, before moving on to the identification of subsystems in information processing. The symmetrical development of the human-technical interaction model makes it possible to apply the same methods to all the entities involved. Since the technical system as well as the human are considered as complex systems, and decomposed analogously in subsystems (ie if we consider the vocal alarms (belonging to the vocal subsystem) and the graphic alarms (belonging to the sub-system). -system-graphic) on the technical system side), it is necessary to consider the subsystems with which the human perceives, will become aware and will treat these alarms: on the human side, these subsystems are assimilated to the auditory and visual modalities, to the attention to the symbol processing system, short and long-term memory and decision-making.

The information of technical origin (first type) and of human origin (second type) are identically configured according to the same multi-agent cognitive model and we use the known language UML ("Unified Modeling Language" in English terminology) to formalize the pilot-cockpit pair.

In the multi-agent cognitive model we will define agents to describe the pilot knowledge processes in relation to the interface elements of the cockpit.

This multiagents representation is particularly adapted to the description of processes that can take place simultaneously.

Indeed, a pilot may have to analyze visual information (input from the human side and output from the technical system side) along with auditory information, such as audible alarms.

This multi-agent representation is also particularly suitable when it comes to following information in a sequential manner and which can take place between different independent interface elements. Furthermore, this representation is also useful for properly ranking and classifying information in order to facilitate the subsequent analysis of data representative of human activities occurring during the interaction between the driver and interface elements.

In cognitive agent-based and resource-based modeling, agents of the cognitive model are identified by their roles, responsibilities, resources or functions, and goals to be attained.

Following this multi-agent approach, the application domain, namely the use of interface elements of the aircraft cockpit, is analyzed in terms of needs to be met in a given context. The agents are goal oriented and allow to account for the desire relative to the constitutive schema of the pilot's belief.

For example, the pilot thinks that to change the flight level he needs a certain number of conditions to ensure the smooth running of his maneuver: visibility, engine conditions, atmospheric conditions ... The pilot therefore wishes to obtain this information to be able to accomplish its task and will therefore use the cognitive resources provided by the interface elements (instruments). He thus completes his awareness of the situation and can plan for the future and act accordingly.

All these aspects will therefore be evaluated in an experimental framework based on multi-agent cognitive modeling. These agents that contribute to cognitive processes relate to the perception, comprehension and mental representation of the interface elements of the cockpit.

Thus, each agent satisfies the goals set by means of action plans that are, for example, in aeronautics, procedures defining the use of the instruments in the operations manual of the crew called FCOM ( "Flight Crew Operating Manuat" in English terminology) and which includes the review of different check lists, landing and takeoff phases ...

On the user (pilot) side, these action plans correspond to the mental representation that the user has of the written flight procedures and which varies according to the experience.

As already mentioned above with reference to Figure 2, the cognitive architecture is based on two main levels, namely the input-output level and the level of information processing. Agents are classified by level (input-output or processing) and by type (input-output channel or processing system).

Thus, on the same level, there are several types: at the input-output level, we have visual type agents, auditory type agents, etc. and at the level of treatment, attention agents, mnemonic agents, decision-making agents, etc. are available.

As noted above, agents are characterized by one or more roles, responsibilities, and resources.

More particularly, the role of an agent is defined with respect to a task or sub-task (for example, relating to the control of the vehicle) to be performed. The responsibilities of the agent are to perform the task or subtask and the resources used allow the actual execution of the task or subtask.

Thus, for example, a scene in three dimensions can be represented by a set of agents that are each in charge of a particular characteristic of the scene, such as relief, textures. The textures correspond to the grid of the relief which can be variable or constant, according to the field databases, that is to say that one can have meshes of the same size everywhere or then meshes of different sizes according to the zones of relief represented, the colors and the symbology.

Like agents, the resources of these agents are classified by level (input-output or processing) and by type (input-output channels or processing system). Thus, for example, the relief of the aforementioned three-dimensional visual scene, which may be represented by an agent, may have varied resources that are used to detect and analyze valleys, rivers, woods, roads, buildings. .. of the visual scene.

The agents of the multiagents cognitive model are determined according to the steps of the process indicated below, which are performed iteratively in two approaches, the top-down approach

Low, qualified as "Top-Down", and the bottom-up approach, described as

"Bottom-Up".

The "Top-Down" approach is based on knowledge of pilots and their use of cockpit interface elements and facilitates classification into agents.

The bottom-up approach is based on cockpit interface elements and visual cues that are grouped together to highlight responsibilities and agents. Top Down

- 1. Identification of tasks

- 2. Identification of the subsystems used to perform each task

- 3. Identification of the agents within each subsystem

- Identification of links between agents

- Identification of the resources of each agent

- Identification of links between resources and other agents

- 3. Matching categories and agents

- 2. Grouping resources into categories

- 1. Identification of resources related to elements of the visual scene

Bottom Ùp

The modeling of the cockpit following this multi-agent cognitive model makes it possible to define the elements of the visual scene at a level of fine granularity which takes into consideration elements constituting each interface element (instrument instruments), namely the information zones. of these interface elements, not each interface element as a set (high level of granularity).

As part of this model, the resources of the agents thus defined are assigned to the processing of the interface elements.

In general, the formalization of the pilot-cockpit pair does not merely represent disparate entities, but proposes to define links between these entities, as represented in FIG. 2, by organizing the entities in the form of tables 16, 18 containing resources, agents, liaison officers and plans, both on the human side and on the technical system side. It should be noted that the liaison agents make it possible to define direct links with specific resources of another agent. Without these linkers, it would be possible to link only agents, not resources to agents.

As illustrated in FIG. 2, the modeling of the technical system is represented on the left of FIG. 2 by the modeling of the PFD interface element 20 which will be detailed below, while on the right side of this same FIG. , we have represented the architecture of the cognitive modeling of the human side 22 on the two main levels, namely inputs-outputs 24 and level 26 where the information processing takes place.

Each of these levels can be broken down into several subsystems, for example, that of vision, hearing, language and motor skills for the first and that of Attention, Long-Term Memory (LTM). for "Long Term Memory" in English terminology), Work Memory (WM for "Work Memory" in English terminology) and Decision Making ("Making Decision" in Anglo-Saxon terminology) for the second.

Since the interface model has been developed from both types of information (information representative of the interface elements and information representative of the human knowledge and behaviors on the use of the interface elements), the algorithm of Figure 1a provides a step

E2 data acquisition.

During this step, data are acquired that are representative of one or more human activities (for example, vision, speech, hearing, movement of human limbs, kinesthesia, and physiological reactions of the human body ...) that are involved in the interaction of the driver with the interface elements.

For example, at a given moment, the pilot, on the one hand, looks at an area of an interface element of the cockpit, the information or information being detected by an eye-tracking device and automatically integrated into a database. of results and, on the other hand, acts at the same time on the handle and / or other equipment, the information or the corresponding information being collected by a video recording system or other and also stored.

It will be noted that depending on the nature of the human activity concerned, a suitable data acquisition apparatus (oculometer, video recorder, electrodermal probe, etc.) is used.

After acquisition of these data, during the next step we proceed to their study (step E3), for example, by the expert pilot or pilots who were the subject of the experiment referred to in step E2. During the analysis of the acquired data, the test subject examines the results and interprets them in an attempt to determine whether an action he performed at a given point in the experiment was appropriate and whether intervened at the right time. More generally, it explains the relationship between taking information / not taking information and acting / not acting.

During the interpretation of these results, the subject of the experiment determines, for example, why his gaze has followed a given visual path on one or more consecutive interface elements and / or on one or more zones of a same interface element.

Depending on the results of this analysis and their interpretation by the subject of the experiment and, possibly, by other experts from different domains, it is possible to validate the framework of the modeling of the pilot-cockpit interface such as it has been developed or adjust it. Thus, for example, it can be seen that it lacks an interface element to allow the pilot to carry out his task of piloting, navigation or other task, or a navigation interface element (display .. .).

It can also be noted that the level of granularity retained during the development of the interface model is too fine and therefore not very representative of the real context or, on the contrary, that the level of granularity is too important and therefore does not allow obtain sufficient relevant information representative of this context.

The interpretation of the results of the experiment also makes it possible to highlight dysfunctions of the interface elements or the flight procedures.

This can, for example, be observed after experiencing significant fatigue and high stress of the subject of the experiment. It is therefore possible to improve the interaction model according to the results of step E3. This is done iteratively by performing the loop shown in FIG. 1a between step E4 and step E1 until the model is obtained. desired interaction that is as representative as possible of the aircraft's on-board environment.

Since the interface model has been determined by the method according to the invention in accordance with the goals set, then the latter (validated model) can be used, for example, for the training of future pilots in a flight simulator or for the improvement of the interfaces proposed by the system (layout, sequence of information, spatial and multimodal redundancy, etc.).

It will be noted that FIG. 1b represents a system 30 for determining a model according to the invention, representative of the interaction between the user 32 and the interface elements 34. This system comprises a computer 36 having inputs outputs to cooperate with the user 32 and the interface elements 34, as well as with a data acquisition apparatus 38 (for example an eye-tracking device) which transmits to the computer 36 the acquired data to be analyzed.

The algorithm of FIG. 3 illustrates in more detail the steps of the algorithm of FIG. 2 by highlighting the symmetrical formalization of the pilot-cockpit pair.

The development of the interface model on the technical system side begins with a first step E10, during which a link is established between the flight procedures defined in the FCOM manual and the interface elements of the cockpit (flight instruments such as PFD, ND ...) that the pilot (PF) and the co-pilot (PNF) must consult for each action described in the flight procedure concerned. These procedures include the take-off procedure, the take-off procedure, the climb procedure in English terminology, and the cruise flight procedure in terminology. anglosaxon), the procedure of descent preparation ("descent preparation" in English terminology), the descent procedure ("descent" in English terminology), the standard approach procedure ("standard approach" in English terminology), the approach procedure non-precision ("non-precision approach" in English terminology) and the landing procedure ("landing" in English terminology).

By associating the instruments concerned by each action described in the climb procedure of the Airbus A340 flight manual, the table shown in FIG. 4 is obtained, showing, for example, that the pilot must consult the instrument called FCU ("Flight Control Unif in English terminology") of the GS panel in SET-value mode and the PFD instrument of the IP main panel in CHECK mode to read the BARO reference (barometric reference). even during the climb, the pilot must consult the PFD instrument on the main panel to view the speed and altitude information, as well as the altitude of the aircraft.

Once the flight procedures have been linked to the cockpit interface elements concerned, the algorithm of FIG. 3 provides a next step E12 during which the information zones of the cockpit are identified. each interface element of the cockpit, as well as the determination of the functions performed by these zones.

By way of example, the different information zones on the primary flight display PFD interface element ("Primary Flight Display" in English terminology) are identified with reference to FIG.

This figure is broken down into two parts: on the left-hand side, the PFD interface element is represented and on the right-hand side, the different information zones of this interface element have been identified as well as their location on this latest. There are thus nine areas of information marked with the numbers 1 to 9 on the right side of Figure 5 and which will be designated later by the references Z1 to Z9.

After the identification of the zones of each interface element, the following step E14 is used to determine the roles and responsibilities (functions of the different zones in view of the tasks and sub-tasks relating to the piloting of the aircraft and in which each interface element is used). From this determination of roles and responsibilities of the zones, it will be possible to determine the agents of the multiagents cognitive model.

Thus, for example for the PFD interface element, there are three basic tasks that are the piloting of the aircraft (T1), the navigation (T2) and the protection to keep the aircraft in the flight range (T3 ).

Within each of these three tasks, it is possible to determine more specific subtasks:

- indicate parameter values of the aircraft (T11),

- indicate selected values or points (coming from FMGS: "Flight Management and Guidance System" in English terminology) (T12),

- indicate flight trends (T13),

- give the indications for radionavigation instruments and the FMGS (T21),

- make it easy to follow the indications provided by the FMGS (T22),

- present the limits of the flight envelope (T31), and - alert (T32).

Once these tasks and sub-tasks have been determined, the roles and responsibilities of the different areas of each interface element and, for example, the PFD are determined.

In Figure 6 is identified and represented in a table different functions or responsibilities previously identified areas.

Thus, the zone Z1 qualified as "FMA"("Flight Mode Annunciatof" in English terminology) from which four sub-zones can be identified which provide information on the piloting mode (for example, automatic pilot mode) and The Z2 zone, referred to as "VA", provides information on the air speed and can be broken down into two sub-zones. The zone Z3 qualified as "AA" and which can be decomposed into two sub-zones provides information on the attitude of the aircraft (pitch, trim, roll, guidance, joystick ...).

The zone Z4 qualified as "A / Vv" and which can be decomposed into three sub-zones serves as an altimeter and provides information on the vertical speed of the aircraft.

Zone Z5 qualified as "ILS-GS" (ILS for "Instrument Landing System" and GS for "Glide Slope" in English terminology) provides information on the vertical position of the ILS instrument landing system, relative to the slope GS .

Zone Z6 qualified. "ILS-Loc" provides information on the ILS horizontal position, relative to the locator ("localizer" in English terminology).

The zone Z7 qualified as "M / l" provides information on the Mach number of the aircraft and navigation information.

The zone Z8 qualified as "H / T" ("heading / track zone" in English terminology) provides information on the guidance and the heading of the aircraft.

Finally, the zone Z9 described as "Ref / Alt" provides information on the height reference. It will be noted that the name of the zones acts as a definition of the role of the agent which will be defined later.

Using the table in Figure 6 and the determination of tasks and sub-tasks, it is possible to determine in the next step E16 the cognitive agents used to build the cognitive model according to the criteria related to driving and navigation. .

To resume the modeling example of the PFD interface element, cognitive agents are determined which make it possible to describe the cognitive processes for using the different zones of the PFD interface element as shown in FIG. , agents related to vertical displacement analysis are determined

(altitude, V / S), analysis of horizontal displacement (speed and heading), analysis Attitude / KJC, tracking of FMGS orders, orientation / ILS, FMA, color code and alert.

As indicated in the column "agent responsibilities", for example, the agent A1 has the role of analyzing the vertical displacement of the aircraft by looking at the parameters of altitude and vertical speed and, to fulfill this role, it is responsible for the values of the vertical parameters and the symbols of these parameters.

To fulfill this role, the agent A1 relies on four cognitive resources related to the responsibility of the values of vertical parameters, on the one hand, and on two cognitive resources related to the responsibility of the symbology, on the other hand. This allows the agent to perform the tasks related mainly to the control of the apparatus (T11 and T12) and which are located in the zone Z4 of the PFD interface element.

During the next step E18, the inputs and outputs of the system are identified with respect to the context of use, that is to say that information provided by the system is identified (elements such as the PFD) at a given time with respect to a given use situation, such as take-off or climb.

In the next step E20 of the algorithm, it is intended to identify system information (e.g., the PFD interface element) located at the processing level.

FIG. 8 illustrates in detail the preparation of the tables 16 and 18 of FIG. 2 according to the plan structure, the link agent, the agent and the resources, on the technical system side as well as on the human side, in the context of the monitoring of the altitude relative to the PFD instrument.

Thus, on the technical system side (Table 18), it is determined in the context of a plan relating to the climb procedure ("CLIMB") illustrated in FIG. 4, that the resources implemented are zones Z4 and Z9 of FIG. PFD (FIG. 7), the agent is the agent A1 of the PFD and the binding agent is the agent A3 of the instruments "EFIS" ("Electronic Flight Instrument System" in English terminology).

Table 16 (human side) will be described later. In parallel with the description which has just been made during steps E10 to E20, an interaction model of the human side is restored with reference to steps E22 to E28 which will be described below.

Human-source information is provided, for example, through interviews with pilots or flight process experts. During these interviews describing given situations (ie use of instruments that present information in two dimensions, for example, ND, PFD, compared to the use of an instrument that would present the same information directly in three dimensions) asks the experts to indicate the actions they would consider undertaking, the controls to be carried out, the information they would need to act ...

During a first step E22, it is intended to identify, at the input-output level of the human interface interface model, the modes of interaction with the technical system, namely, for example, the input-output channels. outputs that constitute human vision, human language, hearing, kinesthesia ...

During this same step, the necessary resources are also identified to undertake the appropriate maneuver, ie to perceive (look at) the altitude information provided by the corresponding area of the PFD interface element, hear (listen) the audible alarm ("call-ouf in English terminology)" TERRAIN "(meaning that the aircraft is outside the safety zone with respect to the terrain, that is, ie too low), pull the handle, or even put the gas.

It should be noted that, optionally, one can also limit the input-output to restricted use situations such as, for example, take-off. It is a division with respect to flight phases, and therefore to specific procedures or sub-sections of the procedure. It is possible to go deep into these specific phases, studying their realization in difficult conditions - bad weather, engine failures, defects in the presentation of information, stress or fatigue of the user. During the next step E24, it is planned to identify the multimodal interactions at the input-output level of the human cognitive model, ie the interactions between the different channels (vision, hearing, etc.). . Thus, for example, we identify the interactions between the different modalities previously identified from cases identified during the interviews, such as that of perceiving the altitude information on the interface element considered and hearing the alarm. auditory and as well as to pull on the handle.

In the next step E26, it is planned to define the level of processing of the cognitive model on the human side.

To do this, it is planned, on the one hand, to identify the actions to be taken by the pilot and / or the decisions to be taken which prove to be particularly difficult or difficult to implement in relation to the modeling that is made of them. on the technical system side and, on the other hand, to make assumptions about the treatment that is made of these sensitive or difficult parts.

Thus, for example, it is assumed that the user will make the right decision with respect to the visual and auditory information that the interface elements of the technical system provide him in the event of a collision alert.

During this same step E26, the information processing is identified according to the different modalities (input-output channels) previously identified.

The representative table 16 of the human side modeling corresponding to table 18 on the technical system side is constructed, as part of the monitoring of the altitude of the aircraft with respect to the PFD instrument, from the defined plane, to know how to use the PFD and how to fly the plane.

In this table, the resources used are determined / identified at the input-output and processing levels. Thus, the visual input-outputs are identified, namely the altitude monitoring provided by the PFD and the corresponding processing, namely the Working Memory (WM) and the Long-Term Memory (LTM), as well as the Taking of Decision.

The corresponding agent is the "PFD" and the aforementioned resources are linked to the "Flight Plan Monitoring" agent.

In this way, the pilot-cockpit multi-agent cognitive model is symmetrically developed. The next step, E28, completes the human cognitive model and validates it with an expert in the field (specialist in cognitive psychology, physiology, language).

During the next step E30, it is planned to validate the representative model of the human-technical system pair (pilot-cockpit) with experts from the various fields involved, namely experts in the flight procedure, experts pilots, designers and experts in human factors (experts in vision, hearing, language, kinesthesia ...). It should be noted that, optionally, steps E28 and E30 can be combined.

Once the model is developed, proceed to step E2 previously described in which human factors analysis methods are used to collect data reflecting corresponding human activities through an experimental protocol. .

It is thus possible to use several analysis methods, as indicated above, to acquire the visual data representative of the visual journey of the pilot, over time, on one or more of the interface elements of the cockpit (more particularly, to follow the position of the pilot from one area of an interface element to another zone of another interface element) via an eye-tracking device and acquire, via the video system of the cockpit , video data representative of the movements of the driver acting, for example, on the handle, and / or auditory data by means of an audio recording apparatus. Thanks to the framework or model defined in the previous steps, it is possible to link, for example, the two types of human activity data (oculometric data and data on the motor skills of the human body) in that a common receptacle (database) was set up to collect both types of information of technical origin and of human origin.

It should be noted that the experimental protocols on which the different human factors assessments are based are derived from the model. previously developed interaction and also feed this model by the results they produce.

Furthermore, it should be noted that the use of a common intermethod and inter-evaluation receptacle ensures coherence, homogeneity and traceability of the collected data.

More particularly, the eye-tracking device 38 of FIG. 1b makes it possible to record the position of the pilot's gaze on a visual scene, thus making it possible to follow the various visual elements traversed by the pilot's gaze on the interface elements of the cockpit. , as well as on the external visual. The eye-tracking device comprises an analog device, namely the eye tracker, which records the movements of the pilot's eye. The eye tracker has three elements, namely, a camera recording the movements of the eye, an infrared source emitting an infrared ray in the eye and a camera recording the visual scene seen by the pilot. Thus, the video data acquired by the camera recording the movements of the eye and the video data acquired by the camera recording the visual scene seen by the pilot are superimposed and the position of the gaze of the pilot is represented by a pointer (for example, a circle or a reticle) that moves on the visual stage. The use of the oculometer alone, although sufficient for the external visual, does not provide sufficient precision if one wants to record particularly fine details of the visual journey of the pilot, for example, the reading of texts or taking information about specific areas of screens.

Thus, the oculometer is associated with a magnetic field generator to provide maximum accuracy.

The magnetic field generator is used as a reference in the three-dimensional space to capture the position of the pilot's head relative to the coordinates of the different surfaces and planes that make up the actual environment of the latter. In this respect, the surfaces and planes concerned are those corresponding to the cockpit screens and control panels constituting regions of interest which can be decomposed into zones and subfields of interest as seen previously for each interface element.

In order to analyze the movements of the pilot's head, a magnetic field generator and a receiver fixed to the pilot's head are thus used, and these elements, combined with the aforementioned analog device (oculometer), make it possible to obtain maximum precision the position of the user's gaze on a visual scene.

More specifically, the receiver attached to the pilot's head provides the exact position of the head in the three-dimensional model. The distance between this head receiver and the camera recording the scene, as well as the distance between the head receiver and the pilot's eyes are then introduced into the three-dimensional model. The first of the aforementioned distances is necessary to perform the calibration of the camera with respect to the scene and the second of these distances is necessary to calibrate the analog device (eye tracker).

It will be noted that the adaptation of the aforesaid eye-tracking device to the cockpit in order to provide maximum precision by combining the data provided by the position of the pilot's head and those provided by the position of his gaze takes account of the geometric study of the cockpit and the study of the pilot's posture.

In performing the geometric study of the cockpit, the Applicant realized that to implant the magnetic field generator on a support in the cockpit, it was necessary to ensure that the distance between the generator and any metal surface is large enough to minimize magnetic interference that may occur with the eye tracking device.

Moreover, during the configuration of the various elements constituting the eye-tracking device within the cockpit, the Applicant has found that the distance between the magnetic field generator and the receiver of the position of the pilot's head must be strictly less than distance between the receiver of the position of the pilot's head and any metal surface, again to minimize magnetic interference. It should be noted that the pilot's postural study makes it possible to define the limits of his movement volume and thus the distances between the head receiver and the magnetic field source.

Thanks to the aforementioned eye-tracking device, it is possible to record very precisely the eye movements (behaviors) such as the bindings, sweeps and pursuits that characterize the way the pilot looks at the specific elements of an aeronautical visual scene ( instrumentation and external visual) .The constituent elements of an eye-tracking device, namely the analog device, the magnetic field generator and a helmet carrying the head receiver, are available from the company Senso-Motric.

Instruments GmbH, Warthestrasse 21, D-14513 Teltow, Germany.

As already indicated above, during the step E3 which follows the data acquisition step, these data are analyzed with the subject or subjects of the (pilot) experiment in order to check the coherence and the reliability of the results of the experiment.

Thus, according to an example borrowed from the automotive field (the invention may indeed apply to fields other than the aeronautical field), using an eye tracker in a driving school vehicle, the instructor and the student can Once the course is over and viewing the video data recorded with the eye tracker, better understand why the student did not look in the rear view mirror before turning.

All the data collected during step E2, analyzed and interpreted during step E3, are then validated at a first intra-domain collective level with the experts of the field concerned (for example, aeronautics made up of a population of pilots) and are then validated at a collective inter-domain level with experts from different fields (experts in human factors, engineers, pilots), so that this data is shared with all concerned stakeholders.

Thus, the experiences data are explained and shared at three levels, an individual level, an intra-domain level and an inter-domain level.

This validation with the experts makes it possible to return to the definition of the framework determined during the first stages (development of the cognitive model multiagents) and to adjust and perfect the model according to the results of experiments and their interpretation by the experts.

Once the model is validated, we can deduce possible improvements to pilot-cockpit interface elements, the procedures for using these interface elements (for example, flight procedure ...), or to use of this model to teach pilots to train themselves on the interface elements of the cockpit.

By way of example, the method according to the invention makes it possible to determine when a display system placed at a height above the pilot's head ("Head Up Display" in English terminology) should be used to optimize use. The method according to the invention also makes it possible to determine whether such a display system is actually used by the pilot on a particular type of vehicle.

In another example, the method according to the invention makes it possible to realize that the pilot mentally constructs a three-dimensional visual representation of the position of his vehicle in space, and this, solely on the basis of information in two dimensions provided by aircraft instruments.

The method according to the invention can then serve as a basis for designing a new instrument providing a three-dimensional visual representation of the position of the vehicle in space.

The method is particularly advantageous for determining the really useful information that is provided by interface elements of the dashboard. Indeed, thanks in particular to the acquisition and analysis of data, for example, oculometric, the method makes it possible to separate the information essential to the user from those that are not particularly useful or that are redundant.

Claims

1. A method for determining a model of an interface between a user and his environment on board a vehicle, characterized in that it comprises the following steps:

developing (E1) an interface model from, on the one hand, a first type of information representative of vehicle interface elements and, on the other hand, a second type of information. information representative of the knowledge held by a user on the use of the interface elements,

acquisition (E2) of data representative of at least one human activity that is requested during the interaction between the user and these interface elements, the acquisition of these data being carried out via at least one a data acquisition device,

- analysis (E3) of the data thus acquired,

- adjustment (E4) of the interface model according to the analysis of the data.

2. Method according to claim 1, characterized in that the two types of information, the first type of information of technical origin and the second type of information of human origin, are provided, with identical configuration, to a dynamic database with symmetric user-system technical structure.

3. Method according to claim 1 or 2, characterized in that the information of the two types are configured according to the same multi-agent cognitive model.

4. Method according to claim 3, characterized in that the configuration of the information of the first type according to a multi-agent cognitive model comprises a step of establishing a link between the vehicle operating procedures and the vehicle interface elements. .

5. Method according to claim 3 or 4, characterized in that the configuration of the information of the first type according to a cognitive model multiagent includes a step of identifying functional areas on each interface element considered.

6. Method according to claims 3 to 5, characterized in that the configuration of the information of the first type according to a multi-agent cognitive model comprises for each interface element the following steps:

- determination of the tasks relating to the use of the vehicle filled by the considered interface element,

determination of agents of the multiagents cognitive model with respect to the determined tasks; establishment of a link between the agents of the cognitive model thus determined and the identified functional areas of the interface element in question.

7. Method according to one of claims 1 to 6, characterized in that the human activity solicited during the interaction between the user and the interface elements is selected from vision, speech, hearing, motricity, manifestations and physiological reactions of the human body.

8. Method according to one of claims 1 to 7, characterized in that the data acquisition apparatus is an eye tracking device recording visual data representative of the visual path of the user on the interface elements.

9. Method according to one of claims 1 to 8, characterized in that the interface elements are the instrument panels of an aircraft cockpit.

10. Method according to one of claims 1 to 9, characterized in that the interface elements are the instruments of a dashboard of a motor vehicle.

11. Use of the interaction model determined according to one of claims 1 to 10 for designing one or more interface elements and / or the arrangement thereof.

12. Use of the interaction model determined according to one of claims 1 to 10 to evaluate one or more new interface elements.

13. Use of the interaction model determined according to one of claims 1 to 10 to modify a procedure for using the vehicle.

14. Use of the interaction model determined according to one of claims 1 to 10 for the formation of a pilot.

15. System (30) for determining a model of an interface between a user and his environment on board a vehicle, characterized in that it comprises:

means (36) for generating an interface model from, on the one hand, a first type of information representative of vehicle interface elements and, on the other hand, a second type of information representative of the knowledge held by a user on the use of the interface elements,

at least one data acquisition apparatus (38) representative of at least one human activity that is requested during the interaction between the user and these interface elements,

means (36) for analyzing the data thus acquired,

means (36) for adjusting the interface model as a function of the analysis of the data.