RU2423294C2 - Method and system to simulate interface between user and ambient medium aboard transport facility - Google Patents

Abstract

FIELD: transport.

SUBSTANCE: invention relates to development of interface elements for perfecting flight control. Proposed system and method comprises developing interface model E1 proceeding from first set of data describing interface elements of transport facility and second set of data describing user's knowledge. Data E2 describing, at least, one kind of user activity involved in interaction between user and interface elements are collected and said collected data E3 are analyzed to correct interface model E4 proceeding from results of analysis of data sent in identical configuration into dynamic data base with symmetric "user-hardware" structure.

EFFECT: perfected interface elements.

14 cl, 8 dwg

Description

Technical field

The invention relates to a method and system for determining an interface model between a pilot and his environment on board a vehicle.

State of the art

In various areas (aviation, car driving, navigation, etc.) for the operation of air, land or sea vehicles (piloting or driving, navigation, communication, environmental control, systems management, etc.), the presence of instrument panels equipped with many interface elements.

To successfully complete the tasks it faces, the vehicle user controlling the interface elements must perfectly know the functions performed by the interface elements, understand the information provided by these elements, and also know the procedures that describe the sequence of actions (manual, visual, auditory) related to interface elements.

Thus, it is clear that during piloting the vehicle, the interaction between the user and the interface elements on board the vehicle is of great importance, and therefore much attention is paid to this interaction.

Summary of the invention

An object of the present invention is to improve existing interface elements, develop new interface elements, or improve flight procedures, or improve the location of various interface elements relative to each other.

An object of the present invention is a method for determining an interface model between a user and his environment on board a vehicle, characterized in that it comprises the following steps:

development of an interface model based on the first type of information characterizing the vehicle's interface elements, and the second type of information characterizing the knowledge the user possesses on the use of interface elements;

collecting data characterizing at least one type of human activity that is involved during the interaction between the user and the interface elements, while collecting this data is carried out using at least one data collection device;

analysis of the data thus obtained;

adjustment of the interface model depending on data analysis.

The interface model is developed on the basis of a double connection “user - technical system”, and not only on the basis of technical information about the system, which allows you to get a very reliable model built on the basis of all data structured relative to each other so that they most clearly characterize the interaction between the user and his environment on board the vehicle and, in particular, the interface elements of this vehicle.

Thanks to the recorded data that reflects visual behavior, and / or gestures, and / or voice actions, and / or the physiological behavior of the user in relation to the interface elements, and as a result of the interpretation of these data, it is possible to improve the previously developed model, bringing it as close as possible to the context, which it should display.

It is possible, for example, to detect abnormal phenomena in the operation of interface elements, evaluate new interface elements, determine an interface element that should provide certain information or provide certain functions, or determine that a new interface element that performs one or more functions is of particular interest.

According to the invention, two types of information - the first type of information of a technical order and the second type of information associated with the human factor, arrive in the same configuration in a dynamic database having a symmetric structure "user - technical system".

Since information of both types arrives all the time in a single identical format, time gain and efficiency are provided when processing this information and, therefore, when developing a model.

According to the invention, information of both types is configured according to the same cognitive multivariate model.

Such a display of information turned out to be the most preferable and effective for developing a model of interaction.

According to the invention, the configuration of the first type of information in a cognitive multivariate model comprises the step of establishing a connection between vehicle use procedures and vehicle interface elements.

Thus, a correspondence is established between the various stages of the vehicle use procedures (for example, piloting) and the interface elements, which are used at each stage to model the interface elements.

According to the invention, the configuration of the first type of information according to a multifactorial cognitive model comprises the step of identifying functional zones on each interface element in question.

By defining such zones within one interface element, it is possible to obtain a detailed model of each interface element and, therefore, to obtain detailed information about the interaction between the user (e.g., the pilot) and zones, even several zones of different interface elements, during the data collection step.

Thus, the model will be even more complete and, therefore, more reliable by collecting oculometric data related to these zones of one interface element or even several interface elements.

According to the invention, the configuration of information of the first type according to a multifactorial cognitive model contains for each interface element the following steps:

identification of tasks associated with the use of the vehicle and performed by the interface element;

identifying factors of a cognitive multifactor model relative to specific tasks;

establishing a connection between certain factors of the cognitive model and identifiable functional areas of the interface element.

The resulting model reflects the interaction of the user (for example, the pilot) with the interface element, taking into account the tasks that the interface element must perform and which are determined, for example, by the procedures for using the vehicle (for example, piloting).

According to the invention, the selected type of human activity involved during the interaction between the user and the interface elements may be visual activity, voice activity, auditory activity, motor activity, physiological manifestations and reactions of the human body.

The collection and analysis of data reflecting various types of human activity allows us to obtain very useful information, for example, to supplement / change the interaction model.

According to the distinguishing feature, the data acquisition device is an oculometric device that records visual data characterizing the visual observation of the interface elements by the user.

Such a device is most preferable for describing the visual behavior of a user (for example, a pilot) when his eyes cast a glance over various interface elements, as well as the external visual environment and even individual zones inside one or several interface elements.

This device can be connected to another device that allows you to register, for example, in the video image gestures of the pilot, while the position of his eyes is tracked by the first device. You can also use audio recordings. Thanks to this, a greater amount of data to be processed and a greater variety of data are obtained, which allows enriching the interface model and making it even more reliable in the context that it should reflect.

The interface model, as it is briefly defined above, finds its application in many areas (aviation, astronautics, car driving, navigation, etc.) and can be used for:

improving one or more interface elements;

designing one or more interface elements;

evaluating one or more new interface elements;

changing the procedure for using (e.g. piloting) a vehicle;

training users (e.g. pilots) to drive a vehicle.

The object of the present invention is also a system for determining an interface model between a user and the environment on board a vehicle, characterized in that it comprises:

means for developing a model based on the first type of information characterizing the interface elements of the vehicle and the second type of information characterizing the knowledge the user possesses on using the interface elements;

at least one device for collecting data characterizing at least one type of human activity that is involved during the interaction between the user and these interface elements;

means of analyzing the received data;

tools for adjusting the interface model depending on data analysis.

This system is characterized by the same aspects and advantages that were mentioned above for the method, and their re-listing is omitted.

Brief Description of the Drawings

Other features and advantages will be more apparent from the following description, presented solely as a non-restrictive example of implementation, with reference to the accompanying drawings, in which:

figa depicts a flowchart of a method for determining an interface model according to the invention;

fig.1b is a block diagram of a system for modeling an interface according to the invention;

figure 2 - diagram of the process of developing a model of the interface according to the invention;

figure 3 - block diagram of the operational sequence of the method with details of the steps shown in figa, according to the invention;

4 is a table of correspondence between the flight procedure and the on-board devices used at each stage of the procedure, according to the invention;

5 is a diagram of the identification of various zones of information on-board device, according to the invention;

6 is a diagram of the functions assigned to the zones defined in figure 5, according to the invention;

Fig.7 is a table of the relationship between the factors of the cognitive model and the functional areas of the on-board tool shown in Fig.5, according to the invention;

Fig - an example of the development of tables 16 and 18 figa, according to the invention;

DESCRIPTION OF PREFERRED EMBODIMENTS

The invention preferably finds application in aviation, in particular, in modeling interface elements of the cockpit.

Various types of dashboards are located in the cockpit, for example, the IP main instrument panel, on which there are airborne devices that act as interface elements for the pilot, called PF (“Pilot Flying”), and the second pilot, called PNF (“Pilot Non Flying”), such as a device called the PFD or the primary flight display (“Primary Flight Display”), or a device called the ND or the navigation display (“Navigation Display”). Here are the central CP panel (“central panel”), the upper OP panel (“overhead panel”) and the panel under the GS windshield (“glareshield panel”).

Cabin user, i.e. the pilot uses all the interface elements of the above-mentioned on-board dashboards for piloting the aircraft, performing navigation tasks, and observing safety measures to keep the aircraft in the flight area.

To facilitate the fulfillment of these tasks by the pilot and ensure his activities with maximum safety, it became necessary to determine the interface model between the pilot and his environment on board the aircraft.

Algorithm i.e. the sequence of operations (figa), represents the main steps of the method, in accordance with the present invention, designed to determine the model of the interface "pilot cockpit".

This algorithm is executed by a computer working in conjunction with data / information storage facilities (databases, storage devices, etc.).

During the first stage E1, an interface model is developed on the basis of two types of information: the first type of information related to the technical system and, in particular, characterizing the interface elements of the cockpit, and the second type of information related to the person and, in particular, characterizing the knowledge acquired by the pilot on the use of cabin interface elements and on flight procedures, as well as its behavior (pilot experience).

The cockpit interaction is based on dynamic interfaces, including types of user behavior and a technical system.

This stage is based on the use of information both of a technical order and related to the human factor in order to take into account the “user – technical system” relationship during the development of an interaction model.

The information of the two above types enters the dynamic database 10 (Fig. 2) of data having the structure “pilot (human) - technical system”, symmetric about the axis of interaction, dividing part 12 of the base associated with the human aspect, and part 14 of the base associated with technical aspect.

It should be noted that the information enters the database in a structured form, according to an identical configuration, which is determined for each aspect (human and technical) by the level of inputs and outputs that details all the inputs and outputs used, and the processing level that details the various subsystems used.

The model is developed starting from the identification of inputs and outputs by the person and by the technical system, before moving on to identifying subsystems at the level of information processing.

The symmetrical development of the model of interaction "man - a technical system" allows you to apply the same methods to all available objects. Since the technical system, as well as the person, are considered as complex systems that are similarly divided into subsystems (that is, if you assign voice signals (belonging to the voice subsystem) and graphic signals (belonging to the graphic subsystem) to the technical side), then subsystems should be considered, when of the help of which a person perceives, understands and processes these signals as a human factor, and these subsystems relate to auditory and visual activity, to attention, to the processing system and symbols, to short-term and long-term memorization and to decision-making.

Technical order information (first type) and information related to the human factor (second type) are configured identically using the same cognitive multifactor model and they use the well-known unified modeling language UML (“Unified Modeling Language” in English-Saxon terminology) for formalization pilot cockpit pairs.

In a cognitive multifactor model, factors are determined that allow us to describe the process of acquiring pilot knowledge related to the interface elements of the cockpit.

Such a multifactorial approach is most preferable for describing processes that can occur simultaneously.

Indeed, the pilot may need to analyze visual information (at the input from the human side and at the output from the technical system) simultaneously with auditory information, such as sound signals.

A multi-factor approach is preferable when it comes to tracking information received in the form of a long-time sequence and from different independent interface elements.

In addition, such a mapping is also preferable to establish a hierarchy and appropriate classification of information to facilitate further analysis of data characterizing the types of human activity used during the interaction between the pilot and the interface elements.

In cognitive modeling based on factors and resources, factors of the cognitive model are determined depending on their role, on the types of their responsibility, their resources or functions, and on the goals set.

According to this multifactorial approach, the scope, i.e. the use of interface elements of the cockpit, analyzed from the point of view of needs that must be met in this context.

Factors are determined by the goal and allow you to take into account the need associated with the scheme, which is part of the pilot's conviction.

For example, a pilot believes that to change the level of flight he needs a certain number of conditions to ensure the normal implementation of maneuver: visibility, condition of engines, atmospheric conditions, etc.

Therefore, the pilot needs to receive this information in order to be able to complete the task, and, therefore, uses the cognitive resources that interface elements (on-board devices) provide him.

Thus, he supplements his knowledge of the situation and can plan and accordingly carry out subsequent actions.

Therefore, all these aspects can be subject to experimental evaluation based on cognitive multivariate modeling.

These factors that contribute to cognitive processes relate to the perception, understanding, and mental display of cabin interface elements.

Thus, each factor contributes to the achievement of goals by using action plans, which may be in aviation procedures that determine the use of on-board devices in the crew’s instructions called the FCOM (“Flight Crew Operating Manual”), which includes, inter alia, following the instructions various control charts, landing and take-off phases, etc.

On the part of the user (pilot), these action plans correspond to the mental idea that the pilot made earlier about the recorded flight procedures and which varies depending on experience.

As mentioned with reference to FIG. 2, cognitive architecture is based on two main levels, i.e. at the level of inputs and outputs and the level of information processing.

Factors are classified by level (I / O or processing) and by type (I / O channel or processing system).

At the same level, several types of factors are encountered: at the input-output level, visual type factors, auditory type factors, etc. are distinguished; at the processing level, attention factors, mnemonic factors, decision factors, etc. are distinguished.

As indicated above, factors are characterized by one or more roles, types of responsibilities and resources.

In particular, the role of the factor is defined in relation to the task or subtask, for example, related to the piloting of the vehicle to be performed.

Types of responsibility of a factor consist in the fulfillment of a task or subtask, and the resources used provide the real fulfillment of the task or subtask.

For example, a three-dimensional scene can be represented by a combination of factors, each of which carries a separate characteristic of the scene, such as relief, structure. The structure corresponds to a grid of relief squares, which can be variable or constant depending on the terrain databases, that is, you can have cells of the same size throughout the grid or cells of different sizes according to the displayed relief zones, colors and symbols.

Like factors, the resources of these factors are classified by level (inputs / outputs or processing) and by type (input / output channels or processing system).

For example, the relief of the aforementioned three-dimensional visual scene, which can be represented by one factor, can have different resources that are used to detect and analyze valleys, rivers, forests, roads, structures, etc. visual scene.

The factors of the cognitive multifactor model are determined according to the steps of the above method, which are iteratively performed according to two approaches, one of which is a top-down approach called “Top-Down”, and the other is a bottom-up approach called “Bottom-Up”.

The “Top-Down” approach is based on knowledge about the pilots and their way of using the interface elements of the cockpit and facilitates classification by factors.

The “Bottom-Up” approach is based on the interface elements of the cab and on the types of visual displays that group together to more clearly define the types of responsibilities and factors.

↓ Top-Down

- 1. Identification of tasks

- 2. Identification of the subsystems used to complete each task

- 3. Identification of factors within each subsystem

-------------------------------------------------- ----------

- Identification of relationships between factors

- Identification of the resources of each factor

- Identification of links between resources and other factors

-------------------------------------------------- ----------

- 3. Selection by pairs of categories and factors

- 2. Grouping resources into categories

- 1. Identification of resources associated with elements of the visual scene

Bottom-up ↑

Cabin modeling using this cognitive multifactor model allows us to determine the elements of the visual scene at the level of fine crushing, which takes into account the constituent elements of each interface element (on-board devices), namely the information zones of these interface elements, and not each interface element as a whole (level of large fragmentation).

Within the framework of this model, factor resources defined in this way are used to process interface elements.

As a rule, the formalization of a pair of pilot cockpit does not end with the presentation of individual objects, but involves the determination of the relationships between these objects (Fig. 2) by organizing objects in the form of tables 16, 18 containing resources, factors, communication factors and plans, both from the person so from the technical system. It should be noted that communication factors make it possible to determine direct connections with the specific resources of another factor. Without these coupling factors, only factors can be linked, but resources cannot be combined with factors.

In Fig. 2, the modeling of the technical system is shown on the left as a simulation of the interface element PFD 20, which will be described in more detail below, while on the right in Fig. 2, the architecture of cognitive modeling of the human aspect 22 is shown at two main levels, i.e. at the level of 24 inputs and outputs and at level 26, where information is processed.

Each of these two levels can be divided into several subsystems, for example, the subsystems of vision, hearing, language and motor skills for the first level and the subsystems of attention, long-term memory (LTM from “Long Term Memory”), working memory (WM from “Work Memory” ) and decision making (“Decision Making”) for the second level.

After the development of the interface model based on two types of information (information characterizing the interface elements, and information characterizing the knowledge and behavior of a person on the use of interface elements), the algorithm in figure 1 provides for data collection step E2.

During this stage, data is collected that characterize one or more types of human activity (for example, vision, voice, arm and leg movements, kinesthesia, physiological manifestations and reactions of the human body, etc.) involved in the pilot’s interaction with the interface elements .

For example, at the moment, the pilot, on the one hand, looks at the area of the cockpit interface element, while the information is detected by the oculometric device and is automatically entered into the results database, and, on the other hand, simultaneously acts on the pen and / or other devices, however, the relevant information is collected by a video recording system or other system and is also stored.

It should be noted that, depending on the nature of the type of human activity under consideration, they use the appropriate data acquisition device (oculometric device, video camera, electrodermatograph, etc.).

After receiving these data, at the next stage they are studied (stage E3), for example, by attracting a pilot or pilots who were participants in the experiment at stage E2.

During the analysis of the obtained data, the participant of the experiment studies the results and interprets them, trying to determine whether the action performed by him at the given moment of the experiment was correct and whether it was performed at the right time.

In general, he examines in detail the relationship between obtaining information / lack of information and action / lack of action.

During the interpretation of these results, the participant in the experiment determines, for example, why his gaze followed this visual path on one or more consecutive interface elements and / or on one or more zones of one interface element.

Depending on the results of this analysis and on their interpretation by a participant in the experience and, if necessary, other experts in different fields, it is possible to confirm the framework for modeling the pilot-cockpit interface as it was modeled, or to adjust the model of this interface.

So, for example, it can be stated that one interface element is missing so that the pilot can complete his piloting, navigation or other task, or there is not enough navigation interface element (display, etc.).

It can also be established that the level of fragmentation used in the development of the interface model was too small and, therefore, could not adequately reflect the real context, or, conversely, the level of fragmentation was too large and, therefore, does not allow to obtain a sufficient amount of necessary information characterizing this context.

Interpretation of the results of the experiment also allows identifying errors in the operation of interface elements or in flight procedures.

This can be seen after stating significant fatigue and increased stressful state of the participant in the experiment. Thus, it is possible to improve the interaction model depending on the results of stage E3.

Thus, the iterative method is used, returning, as shown in FIG. 1a, from step E4 to step E1, to obtain the desired interaction model that will maximize the display of the environment on board the aircraft.

After determining the interface model using the proposed method, in accordance with the goals, the confirmed model can be used to train future pilots on a flight simulator or to improve interfaces simulated by the system (location, sequence of information collection, spatial redundancy and diversity, etc. .).

1b shows a model determination system 30, in accordance with the present invention, characterizing the interaction between user 32 and interface elements 34. This system includes a computer 36 having inputs / outputs for interacting with user 32 and interface elements 34, as well as with the device 38 data collection (for example, oculometric device), which transmits to the computer 36 collected data intended for analysis.

The algorithm, that is, the sequence of operations in figure 3, shows in more detail the steps of the algorithm in figure 2 and more clearly identifies the symmetrical formalization of the pair of pilot cockpit.

The development of the interface model from the side of the technical system begins with the first stage E10, where the connection is established between the flight procedures defined in the FCOM instruction and the cabin interface elements (on-board devices such as PFD, ND ...) with which the pilot (PF) or the second the pilot (PNF) must be verified for each action described in the appropriate flight procedure.

Among these procedures, you can specify a take-off procedure, a cruise flight procedure, a descent preparation procedure, a descent procedure, a rough approach procedure ( “Non-precision approach”) and the landing procedure.

By associating the corresponding on-board instruments with each action described in the “climb” flight procedure for the Airbus A340 piloting instructions, a table (Fig. 4) is shown showing that the pilot must check the readings of an instrument called FCU (“Flight Control” Unit ”), the GS panel in SET mode and the PFD of the main IP panel in CHECK mode to read the BARO ref. (barometric control).

Similarly, during climb, the pilot must check the PFD on the main panel for speed and altitude information as well as aircraft position.

After establishing a connection between the flight procedures and the corresponding cabin interface elements, the algorithm in FIG. 3 provides for the next step E12, during which the information zones of each cabin interface element are identified and the functions performed by these zones are determined.

For example, various information zones are identified on the PFD (“Primary Flight Display”) main display interface element shown in FIG.

Figure 5 is divided into two parts: the left part shows the PFD interface element, and the right part identifies the information zones of this interface element, as well as their location on this element.

In the right part of Fig. 5, nine information zones are shown, indicated by the numbers 1-9, which will be further denoted by the positions Z1-Z9.

After identifying the zones of each interface element, at the next stage E14, roles and responsibilities are determined (functions of different zones taking into account the tasks and subtasks associated with piloting the aircraft in which each interface element is used). Based on the definition of the roles and types of responsibility of the zones, it is possible to determine the factors of the cognitive multifactor model.

For example, for the PFD interface element, three fundamental tasks are distinguished, which are piloting an airplane (T1), navigation (T2), and adhering to security measures to keep the airplane in the flight area (T3).

Within each of these tasks, more precise subtasks can be defined:

indicate the values of the parameters of the aircraft (T11);

indicate values or selected points (from FMGS: “Flight Management and Guidance System”) (T12);

indicate flight trends (T13);

give testimony of radio navigation devices and FMGS (T21);

to provide easy tracking of the indications given by the FMGS device (T22);

imagine the boundaries of the flight area (T31);

turn on the alarm (T32).

After determining these tasks and subtasks, the role and types of responsibility of different zones of each interface element and, for example, PFD, are determined.

6, the table identifies and shows various functions or responsibilities of previously identified zones.

6 shows a Z1 zone called “FMA” (“Flight Mode Annunciator”), based on which four subzones can be identified that provide information about the pilot mode, for example, automatic pilot mode, and radio navigation.

Zone Z2, called “VA”, provides air speed information and can be divided into two subzones.

Zone Z3, called “AA”, which can be divided into two subzones, provides information about the behavior of the aircraft (pitch, position of the aircraft in the air, roll, control, handle ...).

Zone Z4, called “A / Vv”, which can be divided into three subzones, serves as an altimeter and provides information about the vertical speed of the aircraft.

The Z5 zone, called “ILS-GS” (ILS from “Instrument Landing System” and GS from “Glide Slope”), provides information on the vertical position of the landing system for ILS instruments in relation to the angle of the GS glide path.

Zone Z6, called “ILS-Loc,” provides information on the horizontal position of the ILS in relation to the landing beacon (“localizer”).

Zone Z7, called “M / I,” provides aircraft Mach information and navigation information.

The Z8 zone, called “heading / track zone”, provides direction and heading information.

Finally, zone Z9, called “Ref / Alt,” provides altimeter orientation information.

It should be noted that the name of the zone replaces the definition of the role of the factor, which will be determined in the future.

Using the table in FIG. 6 and defining tasks and subtasks during the next step E16, it is possible to determine cognitive factors that make it possible to construct a cognitive model in accordance with the criteria associated with piloting and navigation.

If we return to the example of modeling the PFD interface element, then cognitive factors are determined that allow us to describe the cognitive processes of using different zones of the PFD interface element (Fig. 7).

Factors associated with the analysis of vertical movement (altitude, V / S), with the analysis of horizontal movement (speed and course), with the analysis of the position of the aircraft A / C, with tracking commands FMGS, with orientation / ILS, with FMA, with a color code are determined and with an alarm.

As indicated in the column “Types of Responsibility Factor”, for example, the role of factor A1 is the analysis of the vertical movement of the aircraft using the altitude and vertical speed parameters, and for this role it is responsible for the values of the vertical parameters and for the symbols of these parameters.

To fulfill this role, factor A1 is based on four cognitive resources related to responsibility for the values of vertical parameters, on the one hand, and two cognitive resources related to responsibility for symbolism, on the other hand. This allows the factor to perform tasks mainly related to piloting the aircraft (T11 and T12), which are located in zone Z4 of the PFD interface element.

In the next step E18, the system inputs and outputs are identified with respect to the context of use, that is, the information supplied by the system (interface elements such as PFD) at the moment is identified with respect to the current situation of use, such as take-off or climb.

During the next step E20 of the algorithm, system information (e.g., PFD interface element) is identified at the processing level.

Fig. 8 shows in detail the receipt of Tables 16 and 18 in Fig. 2, according to the structure "plan, communication factor, factor and resources", both from the technical system and from the human side, as part of the height control with respect to the PFD device .

So, on the part of the technical system (Table 18), within the framework of the plan related to the climb procedure (“CLIMB”) (Fig. 4), it is determined that the resources used are zones Z4 and Z9 of the PFD device (Fig. 7), the factor is factor A1 of the PFD instrument, and the communication factor is factor A3 of the on-board instruments “EFIS” (“Electric Flight Instrument System”).

Table 16 (human side) will be described below.

In parallel with the actions that were described during steps E10-E20, a model of human interaction is established, according to steps E22-E28, which will be described below.

Information on the human aspect is obtained, for example, during interviews with experts on piloting or on flight procedures. During these conversations describing given situations (that is, the use of devices that provide information in two dimensions, for example ND, PFD, in relation to the use of a device that could give the same information directly in three dimensions), experts are asked to indicate the actions, which they could undertake, prospective checks, information that they might need, etc.

During the first stage E22 at the input-output level of the model of the interface on the human side, identification of the features of interaction with the technical system is provided, for example, input-output channels, which are human vision, human voice, hearing, kinesthesia, etc.

During the same stage, the resources necessary to carry out the corresponding action are also identified, that is, for example, perceiving (viewing) information about the position of the aircraft supplied by the corresponding zone of the PFD interface element, listening to the sound signal (“call-out”) “EARTH” , meaning that the aircraft is outside the safety zone with respect to the ground, that is, too low, the impact on the handle or an increase in gas.

It should be noted that, optionally, you can also limit the level of inputs and outputs in limited use situations, such as take-off. This is a sample in relation to the phases of the flight, that is, in relation to specific procedures or sections of procedures. You can delve into these specific phases by examining their implementation in harsh conditions: bad weather, engine malfunction, failure to receive information, stress or fatigue of the user.

During the next stage E24, identification of various interactions is provided at the level of inputs and outputs of the cognitive human model, that is, interactions between different channels (vision, hearing ...).

For example, interactions between various previously identified features are identified on the basis of cases described during conversations, such as the case in which information about the position of the aircraft is received on the interface element in question, an audible alarm is heard and the pen is applied to itself.

During the next step E26, the level of processing of the cognitive model by the person is determined.

For this, on the one hand, it is necessary to identify actions taken by the pilot and / or decisions made that are especially difficult or difficult to implement in relation to the modeling performed by the technical system, and, on the other hand, put forward hypotheses about the processing these sensitive or complex parts.

For example, it is assumed that the user will make the right decision regarding the visual or audio information that is provided to him by the interface elements of the technical system in the event of a collision.

During the same stage E26, information processing is identified by various previously identified features (input-output channels).

To control the flight altitude of the aircraft relative to the PFD, a table 16 has been generated that describes the modeling on the human side, corresponding to table 18 from the technical system, based on a specific plan, namely the use of PFD and piloting the aircraft.

This table defines / identifies the resources used at the I / O and processing level.

Visual inputs and outputs are identified, namely monitoring the flight altitude information supplied by the PFD and the corresponding processing, namely working memorization (WM) and long-term memorization (LTM), as well as decision making.

The relevant factor is “PFD”, the indicated resources are related to the “Compliance with the flight plan” factor.

Thus, a cognitive multi-factor model of a pilot cockpit is symmetrically obtained.

The next stage E28 allows you to complement the cognitive human model and confirm it with the help of an expert in this field (specialist in cognitive psychology, physiology, language).

During the next stage of E30, a model is confirmed that characterizes the “man-technical system” pair (pilot cockpit), with the help of experts in various areas involved, i.e. flight operation experts, expert pilots, developers and human factors experts (vision, hearing, language, kinesthesia experts ...).

It should be noted that in an optional embodiment, steps E28 and E30 can be combined.

After the development of the model, they proceed to the previously described stage E2, during which they use methods for analyzing human factors to collect data reflecting the corresponding types of human activity using the protocol of the experiment.

So, you can use several analysis methods that were already mentioned above to obtain visual data characterizing the path of the pilot’s gaze in time on one or more interface elements of the cockpit (in particular, to track the position of the pilot’s gaze from one zone of the interface element to another zone of another interface element), using an oculometric device and receiving, using the on-board video system, video data characterizing the movements of the pilot, which acts on the handle, and / or sound data, record data using an audio recorder.

Thanks to the framework or model defined during the previous stages, for example, two types of data on human activity (oculometric data and data related to the motility of the human body) can be linked, provided that there is a common medium (database) for collecting two types of technical information and human aspect.

It should be noted that the experimental protocols on which various estimates of human factors are based are derived from the previously developed interaction model and also supplement this model with the results contained in them.

In addition, it should be noted that the use of a common medium for the interconnection of methods and estimates provides coherence, uniformity, and the ability to track received data.

In particular, the oculometric device 38 (Fig. 1b) allows you to register the position of the pilot’s gaze in the visual scene, as well as to track various visual elements covered by the pilot’s gaze on the interface elements of the cockpit, as well as on their visual environment.

The oculometric device contains an analog device, i.e. an oculometer that records the movements of the pilot’s eye. The oculometer contains three elements: a camera that records eye movements, a source of infrared radiation that emits an infrared ray into the eye, and a camera that records a visual scene observed by the pilot.

Thus, the video data obtained using the camera recording the eye movements and the video data obtained using the camera recording the visual scene observed by the pilot are superimposed on each other, and the position of the pilot’s gaze is displayed as a pointer (for example, a circle or a cross), which moves around the visual scene.

The use of only one oculometer, although sufficient for an external visual environment, does not provide sufficient accuracy if it is necessary to register particularly small details in the way of the pilot's eye, for example, when reading texts or reading information on specific areas of screens.

Therefore, the oculometer is connected to a magnetic field generator to ensure maximum accuracy.

A magnetic field generator is used as a coordinate system in three-dimensional space to capture the position of the pilot’s head relative to the coordinates of various surfaces or planes that form its real environment. The considered surfaces and planes correspond to screens and control panels of the cabin, which are considered areas that can be divided into the above zones and subzones for each interface element.

To analyze the movements of the pilot’s head, a magnetic field generator is used, as well as a receiver mounted on the pilot’s head, and these elements, in combination with the above-mentioned analog device (oculometer), allow to achieve maximum accuracy in displaying the user's gaze position on the visual scene.

In particular, the receiving device mounted on the pilot’s head allows you to get the exact position of the head on a three-dimensional model.

The distance between this receiver on the head and the camera recording the scene, as well as the distance between the receiver on the head and the pilot’s eyes, is then introduced into the three-dimensional model. The first of the above distances is necessary for adjusting the camera relative to the scene, and the second distance is necessary for adjusting the analog device (oculometer).

It should be noted that the adaptation of the aforementioned oculometric device to the cockpit to ensure maximum accuracy in combination with a combination of data on the position of the pilot’s head and data on the position of his gaze takes into account the geometric study of the cockpit and the study of the position of the pilot’s body.

Performing a geometric study of the cabin, it was found that to install the magnetic field generator on the bracket in the cabin, it is necessary to make sure that the distance between the generator and the entire metal surface is large enough to minimize magnetic interference that can affect the oculometer.

In addition, during the configuration of the various elements forming the oculometric device inside the cockpit, it was found that the distance between the magnetic field generator and the pilot head position receiver should be strictly less than the distance between the pilot head position receiver and the entire metal surface to minimize magnetic interference.

It should be noted that the study of the position of the pilot’s body allows one to determine the boundaries of the volume of its movement and, consequently, the distance between the receiver on the head and the source of the magnetic field.

Using the indicated oculometric device, it is possible to very accurately record eye movements (behavior), such as fixing, scanning, and tracking, which characterize how the pilot looks at specific elements of the visual scene (airborne instruments and the visual environment). Components of the oculometric device, namely, an analog device, a magnetic field generator, and a helmet with a receiver for mounting on the head, are supplied by Senso-Motric Instruments GmbH, Wartenstrasse 21, D-14513 Telt, Germany.

As mentioned above, during stage E3, which follows the stage of data collection, this data is analyzed with the participation of one or more participants in the experiment (pilot) to verify the coherence and reliability of the results of the experiment.

According to an example taken from the field of driving a car (the present invention can also be applied in other fields except aviation), using an oculometer in a driving school vehicle, an instructor and a student after completing the course of study, watching the video data recorded with the help of an oculometer, can better understand why the student did not look in the rearview mirror before turning.

All data obtained during stage E2, analyzed and interpreted during stage E3, are then confirmed at the first collective interdepartmental level by experts in this field (for example, piloting experts in the field of aviation) and then confirmed at a collective interdepartmental level by experts in different areas ( human factor experts, engineers, pilots) so that all involved participants can confirm this data.

Thus, the experience data are refined and confirmed at three levels: the individual level, the interdepartmental level and the interdepartmental level.

This confirmation by experts allows us to return to the definition of the framework defined in the first stages (development of a cognitive multivariate model), and to adjust and improve the model depending on the results of experiments and on their interpretation by experts.

After confirming the model, you can identify possible improvements to the pilot cockpit interface elements, procedures for using these elements (for example, flight procedures), or use this model to train the pilot in the use of cockpit interface elements.

For example, the method in accordance with the present invention makes it possible to determine at what point the display system should be used, set in height above the pilot’s head (“Head Up Display”), so that this use is optimal. The method, in accordance with the present invention, also allows you to determine whether such a display system is actually used by the pilot on this type of vehicle.

In another example, the method in accordance with the present invention allows us to establish that the pilot in his mind constructs a three-dimensional visual display of the position of his vehicle in space, and only on the basis of information in two dimensions from on-board devices.

In this case, the method in accordance with the present invention can serve as the basis for the development of a new device that produces a three-dimensional visual display of the position of the vehicle in space.

The method is of particular interest for determining truly useful information that is supplied by the interface elements of the dashboard.

Due, in particular, to the collection and analysis of data, for example, oculometric data, the method allows you to separate the information necessary for the user from information that is not sufficiently useful or which is redundant.

Claims (14)

1. A method for determining an interface model between a user and the environment on board a vehicle, comprising the steps of: developing (E1) an interface model based on the first type of information characterizing the vehicle's interface elements and the second type of information characterizing the knowledge the user has , on the use of interface elements, collect data (E2) characterizing at least one type of human activity that is involved during the inter actions between the user and the interface elements, while the collection of this data is carried out using at least one data acquisition device; analyze (E3) the received data, adjust (E4) the model of the interface depending on the data analysis, characterized in that two types of information: the first type of information of a technical order and the second type of information associated with the human factor, arrive in the same configuration in a dynamic database having a symmetric structure "user - technical system". 2. The method according to claim 1, characterized in that the information of both types is configured according to the same cognitive multifactor model. 3. The method according to claim 2, characterized in that for the configuration of the first type of information using a cognitive multifactor model, a connection is established between the procedures for using the vehicle and the interface elements of the vehicle. 4. The method according to claim 2, characterized in that for the configuration of information of the first type according to a multifactorial cognitive model, functional zones are identified on each interface element under consideration. 5. The method according to any one of claims 2 to 4, characterized in that at the stage of configuration of the first type of information on a multi-factor cognitive model for each interface element: determine the tasks associated with the use of the vehicle and performed by the interface element, determine the factors of the cognitive multi-factor model relative to certain tasks, establish a relationship between certain factors of the cognitive model and identifiable functional areas of the interface element. 6. The method according to any one of claims 1 to 4, characterized in that, as the selected type of human activity involved during the interaction between the user and the interface elements, visual activity, voice activity, auditory activity, motility, physiological manifestations and human reactions are selected body. 7. The method according to any one of claims 1 to 4, characterized in that an oculometric device that records visual data characterizing visual observation of interface elements by a user is used as a data acquisition device. 8. The method according to any one of claims 1 to 4, characterized in that the interface elements are on-board devices of the aircraft cabin. 9. The method according to any one of claims 1 to 4, characterized in that the interface elements are automobile dashboard devices. 10. Using the interaction model in a certain way according to any one of claims 1 to 9 for designing one or more interface elements or their location. 11. Using the interaction model in a certain way according to any one of claims 1 to 9 for evaluating one or more interface elements. 12. Using the interaction model in a certain way according to any one of claims 1 to 9 to change the procedure for using the vehicle. 13. The use of the interaction model in a certain way according to any one of claims 1 to 9 for pilot training. 14. The system (30) for determining the model of the interface between the user and his environment on board the vehicle, characterized in that it contains: means (36) for developing a model based on the first type of information characterizing the interface elements of the vehicle and the second type of information characterizing the knowledge that the user possesses on the use of interface elements, while there are two types of information: the first type of information of a technical order and the second type of information related to the human factor, at least one device (38) for collecting data characterizing at least one type of human activity that is involved in the interaction between the user is received in the same configuration in a dynamic database having a symmetric structure "user - technical system" and interface elements, means (36) for analyzing the received data, means (36) for adjusting the interface model depending on the data analysis.

Images