FR3067848B1 - METHOD FOR THE PRACTICAL TRAINING OF A TRACK AGENT USING A VIRTUAL ENVIRONMENT AND INSTALLATION FOR ITS IMPLEMENTATION - Google Patents

Abstract

The invention particularly relates to a method for the practical training of a track agent (1) implementing a virtual environment (E2), said virtual environment (E2) comprising an airport zone, a mobile element, able to be moved in a said airport area and an avatar. The method is implemented in an installation comprising a virtual reality headset (3), at least one gesture recognition sensor (4), a computer (6) implementing a data processing and generation system (7). said virtual environment (E2) simultaneously in the virtual reality headset (3) and on a screen (8) that it comprises. The method comprises steps in which (i) the system (7) receives and processes gesture information (D1) from the gesture recognition sensor (4) and then generates in the virtual environment (E2) a first motion of the mobile element, and (ii) the system (7) receives and processes information (F1-F13, Id) transmitted to it by a trainer (2) via said computer (6), and then generates in the virtual environment ( E2) a second movement of the movable element.

Description

The invention relates to a method for forming runway agents at airports, for example those which guide airplanes after a taxiway operator using a VIRTUAL ENVIRONMENT and installation for its implementation. their landing phase to place them at their point of parking in contact with the bridge or remote.

The track agents are now trained for this work by following a theoretical part and a practical part. Currently, the practical part is performed directly on the track with devices in operation. However, because of the existing operational constraints in the airports, it is very difficult to put in sufficient condition the runway agent to be trained. The aim of the invention is to provide new practical training tools in a virtual environment in order to confront the runway agent with different scenarios more quickly and in greater detail: the implementation of a virtual tool makes it possible to create artificially and on request operational constraints that are difficult to obtain in a real life situation.

In other words, the invention aims to provide a best practice for the track agents and to confront them to complex scenarios quickly (exceptional weather conditions, various mechanical problems ...).

Thus, the method according to the invention will provide a training which will include a theoretical part and a practical part, the practical part comprising a part made in virtual reality simulation and a part performed in real situation.

The method according to the invention allows two people to interact at the same time on the virtual environment so that the track agent to be trained is aided or troubled by a trainer.

To do this, the invention relates to a method for the practical training of a track agent implementing a virtual environment, said virtual environment comprising: i. An airport zone ii. A movable element, able to be moved in said airport zone iii. An avatar, said method being implemented in an installation comprising: a virtual reality headset capable of being worn by said track agent; at least one gesture recognition sensor of said track agent; a computer capable of putting implement said method, - said computer implementing a system for processing data and generating a virtual environment simultaneously in the virtual reality headset and on a screen that it comprises,

The method includes steps of: a) The system receives and processes gesture information from the gesture recognition sensor, then generates in the virtual environment a first movement of the moving element, and b) the system receives and processes information transmitted to it by a trainer via said computer and then generates in the virtual environment a second movement of the mobile element.

Thus, the method implemented allows dual access to the virtual environment since the virtual environment is generated in the virtual reality headset worn by the agent to be trained and on the screen of the computer of the trainer who monitors the session and who can also act on the virtual environment. The method therefore allows training in a virtual environment whose events are controlled by two people: the sessions are therefore random, original and adapted to what the instructor wants the runway agent and the level of the track agent to form.

According to an advantageous embodiment, said second movement generated in step b) acts on the first movement generated in step a). In this way, the trainer can correct in real time the effects of gestural information on the virtual environment and in particular on the mobile element. It can help the trainer in training or put him in difficulty.

The method according to the invention may also comprise the following characteristics, taken separately or in combination: the received gestural information is identified by comparison with typical gestural information stored in a database, the movement of said movable element is controlled according to said gestural information identified; the mobile element to be moved in said airport zone is an airplane and the standard gestural information defines actions codified and standardized by the World Air Transport Association; the system comprises an identification interface for accessing said virtual environment and the method comprises the following steps: Transmission by said trainer to said system of an identifier of said track agent via the identification interface. Recognition of the identifier, o Loading said virtual environment associated with said identifier. the system comprises a management interface for meteorological conditions, day and night, and the method comprises the following steps: Transmission by said trainer to said system of meteorological information, day and night via the interface of management, o Loading said virtual environment reflecting said meteorological information and day or night. the system communicates with a Cloud hosting platform and the method comprises the following steps: The system transmits to the hosting platform the data generated by the system during a training session, where the hosting platform records said data. the step of recognizing the identifier is carried out by said hosting platform and said recorded data are associated with said identifier. The method implements at least one second virtual gesture training environment (E1) and the implementation of said virtual environment (E2) comprising the airport zone is conditioned by authorization information, said method comprising the following steps; : o Generation of said second virtual training virtual environment (E1), o Recording of a succession of gestural information o Processing of said succession of information o And generation of said authorization information. The installation comprises a second gesture recognition sensor, capable of collecting information relating to a position and orientation of the virtual reality helmet in the space, and said method comprising the following steps: The system receives and processes the information position and orientation of the virtual reality helmet worn by said agent, o The system modifies the virtual environment so as to display a viewing angle specific to said position and to said orientation of the virtual reality helmet in space. The invention also relates to an installation for implementing the method defined above which is remarkable in that it comprises: a virtual reality helmet adapted to be worn by said track agent, at least one sensor of gesture recognition of said track agent, - a computer capable of implementing said method, said computer comprising a system for processing data and generating said virtual environment simultaneously in the virtual reality headset and on a screen that it comprises, and said system being adapted to receive and process gestural information from the gesture recognition sensor, as well as information transmitted to it by a trainer via said computer, for respectively generating a first and a second movement of the movable member in the virtual environment.

To be able to be executed, the invention is explained in a sufficiently clear and complete manner in the following description which is, in addition, accompanied by drawings in which: FIG. 1 is a schematic representation of an installation allowing the implementation of the method according to the invention, a trainer and a track agent, - Figure 2 is a representation of the type "block diagram" which illustrates the various actors ensuring the implementation of the method according to the invention - Figure 3 illustrates a first virtual environment in which the apprentice trains to move an object by performing a series of predefined gestures, - And Figure 4 illustrates a second virtual environment in which the apprentice trains to achieve the predefined gestures,

In the following description, the terms "lower", "upper", "upper", "lower", etc. are used with reference to the drawings for greater ease of understanding. They should not be understood as limitations of the scope of the invention.

The method according to the invention, which will now be described, implements two training modules: a first module M1 to teach a track 1 agent predefined gestures, codified, which he must reproduce in order to exercise his job, and a second module M2 to put in the situation of the track agent 1 in a virtual environment E2, in which it reproduces the gestures learned to, in the example described, guide an aircraft in its displacement, so as to position at a predetermined stopping point.

It should be understood that the invention is not limited to the presence of these two modules: the invention can be implemented with other modules for moving a movable element in an airport area.

FIG. 1 illustrates a track agent 1 to be formed and a trainer 2, the track agent 1 and the trainer 2 using an installation that makes it possible to implement the method according to the invention. The installation comprises a virtual reality helmet 3, capable of being worn by the track agent 1, two gesture recognition sensors 4 and 5 (the installation therefore comprises at least one) and a computer 6 capable of the process according to the invention. The installation that implements the method according to the invention is associated with a hosting platform 9, of the "cloud" type, which communicates with the computer 6 via a network R1.

According to the invention, the computer 6 implements a data processing system 7, the data being received from either the track agent 1 or the trainer 2 as will be explained below.

The system implements the two modules M1 and M2, mentioned above, which are respectively a training module with predefined gestures (see FIG. 4), or a simulation module for the track agent 1 in the virtual environment (see Figure 3).

Each training module M1 and M2 generates a virtual environment E1 and E2, respectively, which are generated both in the virtual reality headset 3 and on the screen 8 of the computer: what the track agent 1 sees in the helmet 3 is seen by the trainer 2 on the screen 8 of the computer 6. However, in the environment E2, the trainer can choose a different point of view than the one displayed in the helmet 3 in order to show another view of the scene taking place

It will be understood that the two virtual environments E1 and E2 are not generated both at the same time: only a virtual environment E1 or E2 is generated both on the screen 8 of the computer 6 and in the headset. virtual reality 3. The choice of the environment E1 or E2 is determined by the trainer 2 or the system 7 itself.

Each training module M1 or M2 also comprises one or more groups of parameters Pn (N being the number of groups) accessible by the trainer 2 via the computer 6 to transmit information F 1 to Fn to the data processing system 7. The information Fi to Fn are processed by the system and generate in the virtual environment E1 or E2 a movement of the mobile element that it comprises.

In our example, the gesture formation module M1 comprises eight groups of parameters P1 to P8 and the virtual situation module M2 of the track agent 1 comprises five groups of parameters P9 to P13.

It should be understood that the parameters Pn which comprises (the) (or) module (s) of the system (implemented by the method of the invention) could be more or less numerous without departing from the scope of the invention . Indeed, the number of parameters Pn is a function of what is useful for the learning of the track agent 1 and the formation itself.

The data processing system 7 also comprises a database 10 which includes (or stores) standard gestural information. This type of gestural information defines gestures codified and standardized by the International Air Transport Association (IATA).

The data processing system 7 also comprises a data comparator 11 which compares information transmitted to it with the information of the database 10.

The information transmitted to the comparator is gestural information D1 which is generated by the gesture recognition sensor 4. More precisely, the sensor 4 is a device capable of capturing the actions made by the track agent 1 to be formed. In other words: when the track agent makes a movement, it is detected by the sensor 4. The sensor 4 generates a gesture information D1 according to the movement that it has captured and transmits it to the data comparator 11 .

The data comparator 11 compares the received gestural information D1 to all the standard gestural information of the database. The gestural information D1 is identified (D10) if the comparator finds that it corresponds to a standard gestural information. The identified gestural information D10 is then transmitted to the module M1 (or M2 depending on the module loaded) so that it is processed and that the virtual environment E1 or E2 is modified.

It is thus understood how the data processing system 7 is able to receive gestural information D1 from the gesture recognition sensor 4 and to generate a response in the virtual environment E1 or E2.

The data processing system 7 furthermore comprises a processing device 12 intended to process information C1 relating to the position and the orientation of the virtual reality helmet 3 in space, this information C1 being generated by the sensor of Gesture Recognition 5 when the Track Agent 1 moves his head. In other words, when the track agent 1 moves his head (wearing the helmet 3), the gesture recognition sensor 5 detects it and transmits to the processing device 12 C1 information relating to this head movement.

The processing device 12 processes the information C1 and transmits to the module M1 (or M2, depending on the module loaded) commands C10 to change the environment E1 (or E2, depending on the module loaded) so that the environment E1 displayed (or E2, depending on the module loaded) is representative of the position and orientation of the virtual reality helmet 3 worn by the track agent 1.

Finally, as can be seen in FIG. 2, the data processing system 7 comprises an identification interface 13. The identification interface 13 makes it possible to associate a track agent 1 to form one or more sessions recorded on the hosting platform 9 for example. The identification interface 13 also makes it possible to have information specific to the track agent 1 to be formed and to give it access to all the modules M1 or M2, or to only one of them if the track agent did not obtain the authorization for the other. Thus, by identifying the track agent 1, the data processing system can give it access to a virtual environment E1 or E2, or both, and can give the trainer 2 access to previous sessions recorded and associated with the agent to train.

When the trainer 2 wants to start a session with a track agent to be formed, he communicates to the system 7 an identifier Id to the identification interface 13. The identification interface 13 communicates the identifier Id to the platform of accommodation 9.

The hosting platform 9 then responds to the data processing system 7 by giving access (arrow bearing the reference A in FIG. 2) to the module M1, or to the module M2, or to the two modules M1 and M2.

The data processing system 7 then loads the module required by the trainer 2.

When the training session is completed or during the training session, the data processing system 7 transmits to the hosting platform 9 the data S relating to said session and the platform 9 records its data by associating them with the identifier with which the data processing system is connected.

It can also be provided two degrees of identification: the identification interface can also deal with the identifier of a trainer at the opening of a session. Once the trainer's identifier is recognized, the system 7 gives the trainer access to all the identifiers of the track agents he trains and which are associated sessions recorded in the hosting platform.

In addition, it can be expected that access to a module is conditioned by the success of the program proposed by the first module M1. In this case, the processing of the gestural information recorded during the session with the module M1 leads to the generation of authorization information.

When the track agent 1 to be trained has acquired this authorization, the data processing system gives him access to the module M2 and communicates this authorization information to the hosting platform 9 which, in turn, records this information. authorization by associating it with the identifier. In this way, during a next session when the trainer will re-enter the identifier of the track agent, he will have access to both modules M1 and M2 without having to replay the program of the module M1.

In addition, with all the data received and recorded, the hosting platform can produce a log of the statistics of the use of the method which is searchable by the trainer 2 via the computer 6.

Reference will now be made to the implementation of the method according to the invention implementing the installation and the elements that comprise it and which have been presented above.

The trainer 2 is installed at his station in front of the computer 6 and connects to the hosting platform. It communicates the identifier of the track agent 1. The data processing system 7 receives the authorization A of the hosting platform for the identified track agent 1.

The trainer 2 loads the simulation module M1 to learn the codified and standardized gesture. The track agent 1 is equipped in the same room or in a separate room (symbolized schematically by the reference 14 in Figure 1). The invention is not limited to an implementation of the method in two separate rooms: this is an example of non-limiting implementation. The virtual environment E1 associated with the module M1 is shown in FIG. 4: it is seen both by the track agent 1 in the virtual reality helmet 3 and by the trainer 2 on the screen 8 of the computer 6 The virtual environment 1 is a training chamber and comprises only virtual floor 16, a virtual sky 17 and an avatar 15. The avatar 15 indicates to the track agent 1 the gestures to be made. The gestures made by the track agent are transformed into gestural information D1 transmitted by the gesture recognition sensor. The comparator 11 transmits the gestural information identified D10 to the module M1.

In the context of our airplane placement example, each coded gesture corresponds to a parameter P1 to P8. The list of parameters P1 to P8 is not exhaustive, others will arrive.

The parameter P1 corresponds to the gesture ordering the moving element to move straight.

Parameter P2 corresponds to the gesture ordering the moving element to turn to the left.

The parameter P3 corresponds to the gesture ordering the mobile element to put on hold.

The parameter P4 corresponds to the gesture ordering the mobile element to slow down.

The parameter P5 corresponds to the gesture ordering the mobile element to brake.

Parameter P6 corresponds to the gesture ordering the moving element to turn to the right.

Parameter P7 corresponds to the gesture ordering the moving element to stop.

And the parameter P8 corresponds to the emergency stop gesture.

When the gesture requested to the lane 1 agent is successful, the system tells the lane 1 officer the success of his gesture and proceeds to the next gesture to learn.

However, the trainer 2 can, via the computer 6, trigger gestures to learn or repeat randomly by sending the module M1 training instructions F1 to F8 generated by activating the parameters P1 to P8.

Once all the gestures have been successfully learned, the trainer 2 loads the M2 module for setting the situation of the track agent in virtual environment E2. The trainer switches the track agent 1 into the virtual environment E2 that is displayed in the virtual reality headset 3 and on the trainer 2 screen. The virtual environment E2 is shown in FIG. 3. The virtual environment E2 is an airport that includes one or more virtual aircraft 18 (one of which must be guided in its movement by the flight attendant 1 by performing the gestures learned), virtual equipment 19 usually present in airport areas (lighting, tractors , baggage carts, lifting platforms, power packs, air conditioning units, motor starter groups, push-backs, push bars, step ladders, mobile ramps, belt conveyor belts, containers and container ships, cargo pallets and door palettes etc.), virtual human personnel (avatar 15), virtual floor 20 and virtual sky 21.

The M2 module proposes to the trainer to act on various parameters of the environment: for example the parameter P9 concerns the meteorological conditions of the virtual environment E2 (good weather, bad weather, rain snow, wind etc.) and day or by night. Parameter P9 directs the trainer to a management interface that allows him to transmit to the data processing system the weather and day or night conditions of the virtual environment.

Parameter P10 concerns the possible interactions with the mobile element. The parameter P10 leads to an interface that enables it to act on the behavior of the mobile element 18 (here the aircraft 18 whose movement must be guided). For example, the trainer can act on the angle of rotation of the mobile aircraft, on its speed of rotation, on its speed in a straight line, on the reaction of the pilot of the aircraft ... Thanks to the parameter P10, the trainer generates a movement of the movable element or acts on the movement of the movable element. Thus, when the runway agent 1 makes the aircraft advance in a straight line and accelerates the aircraft by performing the appropriate gestures learned, the trainer 2 can simultaneously act on the speed of the aircraft and help, or put in difficulty , the track agent 1.

Thus, the movement generated by the trainer 2 on the mobile element 18 can act on the movement generated by the track agent on the same mobile element 18.

Parameter P11 concerns the trainer triggering sudden, unforeseen events: for example, the runway agent 1 moves the aircraft forward and the trainer decides to bring a vehicle into the virtual environment that crosses the trajectory of the aircraft. aircraft, forcing the lane 1 agent to react by performing other urgent actions to brak the aircraft 18.

Parameter P12 concerns camera management, showing different angles of view in the virtual environment.

The parameter P13 concerns the management of the statistics, results and recording videos of the track agent 1 identified.

In our example, the module M2 aims to lead the track agent 1 to perform a series of gestures learned in the module M1 to move the aircraft 18 in the virtual environment and place it precisely at a predetermined point in the airport area : on the stop bar of the parking point on which the avatar 15 is located.

At the beginning of the session, the plane 18 is on the left, on the right or coming in front of the avatar 15.

To advance the aircraft 18, to make it turn, accelerate, brake etc., the track agent 1 performs a series of gestures that are captured by the gesture recognition sensor 4 and transformed into gesture information D1 transmitted to the comparator 11, transmitted in gesture information identified D10 to the module M2. In response to this information, the moving element 18 (the aircraft in the example), advance, rotate, accelerate, brake, etc. The servo of the speed and the direction of the aircraft with the gestures of the agent of track 1 is realized by means of the sensor 4 which captures each movement and position of the shoulders, arms and forearms which, according to their relative speed, will affect the steering of the speed of the aircraft and the rotation of the nose wheel (front landing gear).

In parallel, the trainer 2 outside the scene visualizes it on the screen 8 of the computer and can act concurrently with that of the track agent 1 to help him or to put him in difficulty.

Once the placement of the aircraft 18 is completed, a last act of the track agent 1 can cut the engines and ensure a total stop of the aircraft 18.

The data processing system 7 displays a success message in the virtual environment E2. The lane 1 agent can remove his helmet 3.

The trainer can then move to the next learner by selecting his name in the interface dedicated to the management of learners or debrief the captured video (s) if necessary.

It will be understood from the foregoing how the method according to the invention makes it possible to train track agents to perform codified gestures that make it possible to communicate and have consequences on the behavior of the other players in the airport areas. The example given above relates to the placement of an aircraft at its parking point. However, it should be understood that the invention is not limited to learning gestures to guide the aircraft and that the method could implement modules on different subjects:

For example, the method could implement a module designed to learn the gesture for the repoussage of aircraft, for towing aircraft, for de-icing aircraft, for loading cargo, handling containers luggage loads etc. The invention extends to the implementation of any equivalent means.

Claims (11)

A method for the practical training of a track agent (1) implementing a virtual environment (E2), said virtual environment (E2) comprising: i. An airport zone (19, 20, 21) ii. A movable member (18) adapted to be moved in said airport area (19, 20, 21) iii. An avatar (15), said method being implemented in an installation comprising: - a virtual reality helmet (3) adapted to be worn by said track agent (1), - at least one gesture recognition sensor (4) said track agent (1), - a computer (6) able to implement said method, said computer (6) implementing a data processing system (7) (D1, C1, A, F1-F13, Id) and generating said virtual environment (E2) simultaneously in the virtual reality headset (3) and on a screen (8) that it comprises, said method comprising steps according to which: c) The system (7) receives and processes the gestural information (D1) from the gesture recognition sensor (4), then generates in the virtual environment (E2) a first movement of the mobile element (18), d) The system (7) receives and processes information (F1-F13, Id) transmitted to it by a trainer (2) via said computer (6), then generates in the virtual environment (E2) a second movement of the movable element (18). 2. Method according to claim 1, characterized in that said second movement generated in step b) acts on the first movement generated in step a). 3. Method according to claim 1 or 2, characterized in that step a) comprises the following substeps: the gestural information (D1) received is identified (D10) in comparison with standard gestural information stored in a base of data (10), and - the movement of said movable member (18) is controlled according to said identified gesture information (D10). 4. Method according to claim 3, characterized in that the movable element (18) to be moved in said airport zone (19, 20, 21) is an airplane (18), in that the standard gestural information that comprises the base of data (10) define actions codified and standardized by the World Air Transport Association. 5. Method according to any one of the preceding claims, characterized in that the system (7) comprises an identification interface (13) for accessing said virtual environment (E2) and in that the method comprises the following steps: Transmission, by said trainer (2) to said system (7), of an identifier (Id) associated with said track agent (1) via the identification interface (13), - Recognition of the identifier, - Loading of said virtual environment (E2) associated with said identifier (Id). 6. Method according to any one of the preceding claims, characterized in that the system (7) comprises a weather management interface, day and night, and in that the method comprises the following steps: - Transmission, by said trainer (2) to said system (7), meteorological information, day and night via the management interface, - Loading said virtual environment (E2) reflecting said meteorological and transmitted day or night information. 7. Method according to any one of the preceding claims, characterized in that: - The system (7) communicates with a hosting platform (9) of the Cloud type, - The system (7) transmits to said hosting platform (9) the data (S) generated by the system (7) during a training session, - the hosting platform (9) stores said data (S). 8. Method according to claims 5 and 7, characterized in that the recognition step of the identifier (Id) is performed by said hosting platform (9) and in that said data (S) recorded are associated with said identifier (Id). 9. Method according to any one of the preceding claims, characterized in that it implements at least a second virtual environment (E1) gesture training and in that the implementation of said virtual environment (E2) comprising the airport zone is conditioned by authorization information (A), said method comprising the following steps: - Generation of said second virtual training 3D environment (E1), - Recording of a succession of gestural information - Processing of said succession of information - and generation of said authorization information (A). 10. Method according to any one of the preceding claims, said installation comprising a second gesture recognition sensor (5) capable of collecting information (C1) relating to a position and an orientation of the virtual reality helmet (3) in the space, said method comprising the following steps: the system (7) receives and processes the information (C1) of position and orientation of the virtual reality headset carried by said agent, - the system (7) modifies the environment virtual (E2) so as to display a viewing angle specific to said position and to said orientation of the virtual reality helmet (3) in space. 11. Installation for implementing the method according to any one of the preceding claims, characterized in that it comprises: - a virtual reality helmet (3) adapted to be worn by said track agent (1), - At least one gesture recognition sensor (4) of said track agent (1), - A computer (6), capable of implementing said method, said computer (6) comprising a data processing system (7) (D1 , C1, A, F1-F13, Id) and of generating said virtual environment (E2) simultaneously in the virtual reality headset (3) and on a screen (8) that it comprises, and said system being able to receive and processing gestural information (D1) from the gesture recognition sensor (4), as well as information (F1-F13, Id) transmitted to it by a trainer (2) via said computer (6), for respectively generating a first and second movement of the movable member (18) in the environment virtual reality (E2).