RU2819843C2 - Method for determining the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event - Google Patents

Abstract

FIELD: training means.

SUBSTANCE: invention describes a method for determining the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event. Disclosed is a method in which stimulus material is generated in the form of a video sequence. Video sequence includes an event having a beginning at a first moment, an end at a second moment in time and a critical moment between the first and second moments in time. For the event, control zones of the direction of gaze of the user and the range of correct user inputs are determined. User is shown a video sequence with an event. Data on the direction of the user's gaze is obtained from the user state control unit, and data on the user input are obtained from the user input control unit. Video sequence is changed in response to user input. Moment of concentration of the user’s attention in response to the event is determined by comparing data on the direction of gaze of the user and the reference zone and/or by means of user input. Moment of time of user input is recorded. Assessing the level of formation of the skill of responding to an event based on comparison of moment of concentration of attention with time of beginning of event and/or critical moment and/or second moment of time, comparing the moment of time of the user input with the time of the beginning of the event and/or the critical moment and/or with the second moment of time and/or with the moment of concentration of attention, or the absence of user input, as well as comparing user input with at least one correct user input.

EFFECT: invention provides effective training of the skill of identifying a potentially dangerous situation and the skill of reacting to an event (regardless of their experience and driving experience) based on immersion in simulated traffic conditions, allowing to anticipate potential emergency situations in conditions as close as possible to real ones, using a virtual reality helmet, as well as differentiation of reasons for unformed skills of identifying a potentially dangerous situation and the skill of reacting to an event when driving vehicles (failure to recognize potential danger, incorrect decisions or low reaction rate) and the ability to form skills by focusing on training the ability to identify potential danger or correction of decisions.

7 cl, 3 dwg

Description

Field of technology to which the invention relates

The present invention relates to the psychology and psychophysiology of work and is intended for training and measuring the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event, in particular, the invention is used to measure the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle using computer technologies using eye tracker and virtual reality technology. Training and measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle allows you to reduce the risks of accidents and injuries.

State of the art

Currently, there are no reliable automated methods, techniques and devices for teaching and measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle.

To drive safely, the driver must be able to observe the road and use his visual abilities wisely. The ability to “read” the road allows you not only to detect danger in a timely manner, but also to avoid it with minimal losses. At the same time, the driver must have enough space and time to perform the necessary maneuvers. A characteristic feature of human vision is the inability of a person to assess the situation on the road at a glance: in order to obtain a more or less clear picture of what is happening, he must focus on the most essential elements from the point of view of traffic safety:

- traffic signs,

- road markings,

- traffic light signals (traffic controller),

- other technical means of traffic management,

- current road conditions,

- the presence of other traffic participants on the road and their behavior.

The driver perceives information about the road situation as a whole, receiving it from several sources at once, and processing and analyzing it accordingly, and doing this as quickly as possible. General principles of the traffic monitoring process:

- concentration of attention in the area of the center of the vehicle’s path;

- accurate identification of key objects on the road at a specific point in time;

- competent alternation of quick inspections of the situation on the road with a more careful and prolonged examination of key objects;

- monitoring the situation behind and on the sides of the vehicle (Gromakovsky A.A. “Self-instruction manual for safe driving. What they don’t teach in driving schools”: St. Petersburg; St. Petersburg; 2009, ISBN 978-5-388-00481-9).

The UK Driver and Vehicle Standards Agency highlights the concept of 'hazard perception'. Hazard perception is the ability to recognize potentially dangerous situations and respond appropriately. To diagnose the maturity of the hazard perception skill, a hazard perception test (Hazard Perception Test) was developed. The Hazard Perception Test aims to assess students' ability to predict, scan, and recognize hazards. It measures a person's ability to notice and react to things happening on a road or footpath. It is important that the driver notices and responds to these hazards before emergency action must be taken to avoid them. Typically, experienced drivers eliminate hazards by detecting and responding to them as early as possible, rather than continuing to drive, causing a dangerous situation that leaves them no choice but to aggressively brake or swerve. (The Official Highway Code. Driver & Vehicle Standards Agency; The Official DVSA Guide to Learning to Drive. Driver & Vehicle Standards Agency, https://hazardperceptiontest.net/about-hazardperceptiontest-net/).

According to the Order of the Ministry of Education of the Russian Federation dated November 8, 2021 No. 808 “On approval of exemplary professional training programs for drivers of vehicles of relevant categories and subcategories” (with amendments and additions), as a result of mastering the educational program, students should be able to:

- predict the occurrence of dangerous road transport situations during the management process and take actions to prevent them;

- make the right decisions in a timely manner and act confidently in difficult and dangerous road situations.

Accordingly, the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle include:

- the ability to predict the occurrence of dangerous road transport situations during the management process and take actions to prevent them;

- the ability to make the right decisions in a timely manner and act confidently in difficult and dangerous road situations.

Experts in this field of technology have proposed various ways to teach and measure the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle. The following are the most famous, authoritative and closest to the present invention.

Known for HazardPerceptionTest - “Hazard Perception Test” in Great Britain. The HazardPerceptionTest is part of a theory test run by the DVSA (Driver and Vehicle Standards Agency). It consists of 14 videos without sound, about a minute long. Each video contains several potential hazards and at least one developing hazard situation. An evolving hazard is a situation that has the potential to develop into a real hazard and affect other road users because it is potentially dangerous even before it fully develops and leads to an accident. During the test, it is necessary to note the moment when a developing danger is demonstrated. It is explained to the test taker that this is a situation that may require the driver to change speed or direction to avoid danger. Evolving hazards change over time and may begin as non-hazardous situations that, over time, risk escalating into a situation requiring the driver to take action to avoid a collision. The test's authors state that the video must show an evolving danger for ten seconds or more. They believe that experienced drivers always assess the situation around them, namely, determine where the greatest threat comes from and take the measures necessary to eliminate or limit this threat. It is understood that potential actions that the driver may take in the video shown may include slowing down, changing gear, braking, signaling, taking a different route, slowing down or accelerating to maintain a safe distance from the hazard. To get the highest score, the test taker needs to note the situation as early as possible, since the time period is divided into five equal parts. The faster the test taker reacts, the higher his score. This test has some limitations. The authors themselves note that a number of experienced drivers and even instructors have had problems due to the fact that their experience allows them to recognize a developing danger before it becomes an obviously visible danger. This can lead to an incorrect assessment of hazard anticipation skill, since experienced drivers often respond by pressing a button before the window for recording the reaction begins. The disadvantages of the simulator for the purpose of measuring the level of development of the skill of foreseeing danger and the timeliness of correct/correct decisions made by drivers when driving vehicles are: 1) the stimulus material is formed from a video sequence in 2D format, which does not allow for maximum immersion in the depicted driving situation, and also excludes influence of external distracting factors; 2) video stimuli do not always depict part of the steering wheel and the dashboard, which reduces the feeling of real and independent control of the vehicle; 3) during training and testing, the features of foveal and peripheral vision are not taken into account, which in turn excludes the possibility of an objective assessment of the skill of foreseeing danger outside the zone of obvious visibility; 4) registration of anticipation of danger occurs by pressing a mouse button or touch screen, which does not allow assessing whether the driver really noticed the danger or pressed the mouse button by accident; 5) the density and speed of traffic on demonstration videos does not always correspond to traffic in a metropolis; 6) testing is carried out in a dynamic test mode, but without recording the time of recognition of a potentially dangerous situation and the speed of reaction (as a rule, these reactions are combined into one parameter - the reaction window), which does not allow identifying the objective level of formation of the skill to anticipate danger (Hazard Perception Test [ Electronic resource] // HazardPerceptionTest.net. URL: https://hazardperceptiontest.net / (access date: 03/20/2022).

Known Australian patent publication AU 2016201442 (A1) “HAZARD PERCEPTION ASSESSMENT SYSTEM AND METHOD”, IPC: G09B 9/04; G09B 9/042; G09B19/16; publ. 09/22/2016). The South Australian Government Hazard Foresight Test is designed as a touch screen computer program. The test measures the ability to recognize potentially dangerous driving situations and respond appropriately to those situations. Test structure: welcome page; instructions; two test tasks with feedback; 15 direct test tasks; test completion page. The test consists of 15 video clips of real driving situations, to which you need to react by touching the touch screen. The test taker sees part of what the driver sees in a real driving situation - the speedometer on the dashboard and the operation of the turn indicators. The monitor screen indicates the content of the maneuver that the driver needs to perform (start braking, change lanes or overtake, turn right, turn left, turn around or drive straight). To perform the maneuver, you must touch the screen at a time when it is safe to perform the indicated action. The test involves demonstrating situations where maneuvering is unsafe. In this case, if the test taker assesses the road situation as unsafe, then he does not need to take any action - he does not need to touch the screen. The disadvantages of the simulator for the purpose of measuring the level of development of the skill of foreseeing danger and the timeliness of correct/correct decisions made by drivers when driving vehicles are: 1) the stimulus material is formed from a video sequence in 2D format, which does not allow for maximum immersion in the depicted driving situation, and also excludes influence of external distracting factors; 2) registration of anticipation of danger occurs by pressing the touch screen, which does not allow assessing whether the driver really noticed the danger or performed the action by accident; 3) the density and speed of traffic on demonstration videos does not always correspond to traffic in a metropolis; 4) during training and testing, the features of foveal and peripheral vision are not taken into account, which in turn excludes the possibility of an objective assessment of the skill of foreseeing danger outside the zone of obvious visibility; 5) testing is carried out in a dynamic test mode, but without recording the time of recognition of a potentially dangerous situation and the speed of reaction (as a rule, these reactions are combined into one parameter - the reaction window), which does not allow identifying the objective level of formation of the skill to anticipate danger (How the Hazard perception test works [Electronic resource] // Government of South Australia. Department for Infrastructure and Transport. URL: https://mylicence.sa.gov.au/the-hazard-perception-test/how-it-works (access date: 03/20/2023)).

There is a well-known test for foreseeing danger, used in the Australian state of Victoria. It consists of 28 video clips of real traffic situations. The advantage of this test is that, in addition to the speedometer on the dashboard, turn indicators, there are right and rear view mirrors. In this case, data on the road situation from the rear and right-view mirrors are not taken into account when the test person makes a decision about the necessary maneuver. The test includes sound that simulates the surrounding traffic situation. The monitor screen indicates the content of the maneuver that the driver must perform or not perform if the situation is not unsafe (start braking, change lanes or overtake, turn right, turn left, turn around or drive straight). (Hazard perception practice test [Electronic resource] // VicRoads. URL: https://www.vicroads.vic.gov.au/licences/your-ps/get-your-ps/hazard-perception-practice-test (date access: 03/20/2023)).

In Russia today, training and measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle is implemented in two variations.

1. Consolidation of accumulated knowledge regarding the most common road accidents, reflected in books with recommendations, for example “How to avoid getting into an accident: a book that will save your life while driving” A.V. Achkasov, A.E. Finkel. and “Typical road traffic situations that preceded incidents” Lochinsky B.F.

2. Since the beginning of the 2000s, defensive driving methods have begun to actively develop. The main goal of defensive driving courses is to reduce accident rates by developing the level of responsibility and awareness among drivers regarding the decisions they make when driving a vehicle. The programs are based on global experience in defensive driving and knowledge of Russian reality. Trainings include theoretical (2-4 hours) and practical (1.5-3 hours) blocks. Within the framework of the practical block, an individual approach is a priority. Most drivers who already have experience are allowed to take part in the training. At the same time, an objective assessment of the level of skill development after completing the courses is not possible.

The main general disadvantages of using the above-described simulators for the purpose of training and measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle are:

- the stimulus material is formed from a video sequence in 2D format, which does not allow for maximum immersion in the depicted driving situation, as well as to exclude the influence of external distracting factors;

- focus on the general preparation of the candidate for driving a vehicle, mainly for teaching actions in road situations in accordance with the rules of the road, without taking into account that the real road situation is often different than prescribed by the rules of the road, which may be consciously or unconsciously violated by other road users ( these systems teach how to drive a car according to the rules, assuming that other road users will also drive according to the rules, which in fact does not correspond to modern realities of road traffic);

- in the video stimuli, part of the steering wheel and the dashboard are not depicted, which reduces the feeling of real and independent control of the vehicle;

- the density and speed of traffic on demonstration videos does not correspond to traffic in a metropolis;

- during training and testing, the features of foveal and peripheral vision are not taken into account, which in turn excludes the possibility of an objective assessment of the skill of foreseeing danger outside the zone of obvious visibility;

- registration of anticipation of danger occurs by pressing a mouse button or touch screen, which does not allow assessing whether the driver really noticed the danger or pressed the mouse button by accident;

- testing is carried out in a dynamic test mode, but without recording the time of recognition of a potentially dangerous situation and the speed of reaction (as a rule, these reactions are combined into one parameter - the reaction window), which does not allow identifying the objective level of formation of the skill to anticipate danger;

- combining reactions (time to recognize a potentially dangerous situation and reaction speed) into one parameter does not allow identifying the cause of the error (inability to recognize a potential danger, incorrect decision or reaction speed);

- the reaction window specified by the program does not correspond to the real reaction speed of professional drivers; accordingly, the software assessment is not objective.

The closest equivalent to this solution is the UK Hazard Perception Test conducted by the DVSA (Driver and Vehicle Standards Agency).

Despite the availability of methods for teaching and measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, a method has not yet been developed that allows one to objectively diagnose the individual components of this skill: the ability to recognize a potential danger, the correctness of decisions made and the speed of reaction.

Disclosure of the invention

The present invention describes a method for determining the level of development of the skill of identifying a potentially dangerous situation and the skill of responding to an event, which may include:

- generation of stimulus material in the form of at least a video sequence;

- inclusion in the video sequence of at least one event that has a beginning at the first time point, an end at the second time point and a critical moment between the first and second time points;

- determining for each event at least one control zone of the user's gaze direction and at least one correct user input;

- demonstrating to the user a video sequence that includes at least one event;

- receiving from the user state control unit at least data about the direction of the user's gaze and from the user input control unit data about the user input;

- changing the video sequence in response to the presence and/or absence of user input;

- determining the moment in time of the user's attention in response to an event by comparing data about the direction of the user's gaze and the control zone and/or through user input;

- recording the moment in time of user input or absence of user input;

- assessment of the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event for at least one event based on comparison of the moment of concentration of attention with at least the moment of the beginning of the event and/or the critical moment and/or the second moment of time, comparison of the moment of time of the user input at least with the start of the event and/or the critical moment and/or with the second time and/or with the time of concentration, or the absence of user input, comparing the user input with at least one correct user input.

In some embodiments, the user's reaction speed is determined as the difference between the time of the event and/or the time of attention and the time of user input.

Additionally, the speed of identifying a potentially dangerous situation can be determined as a comparison of the first moment in time and the moment in time of concentration of attention.

The method may also include determining the correctness of the response by comparing the user input with at least one correct user input.

In particular implementation cases, a video sequence simulating the movement of road transport is used as stimulus material, and at least one event is used as a vehicle event selected from at least the following series: overtaking, turning around, turning, crossing the road, repair work on the road, ejection gravel, splashes and/or sand from under the wheels, wheel explosion, loss of vehicle integrity, appearance of foreign objects on the road or roadside, appearance of vehicles, people, animals on the road and/or side of the road, decreased visibility on the road and/or side of the road , and the video sequence shown to the user includes at least one specified event.

In some embodiments, an audio sequence that simulates at least the sound of a moving vehicle, sounds inside the vehicle, and/or the environment is added to the stimulus material.

In some embodiments, the inventions can generate a report on at least one of: an assessment of the level of development of skill in responding to at least one event, the correctness of the reaction, the speed of determining the identification of a potentially dangerous situation, the speed of the user's reaction; direction of the user's gaze, and the report or part of the report can be shown to the user and/or saved for transmission to the developer.

The technical result of this solution is:

- effective training in the skill of identifying a potentially dangerous situation and the skill of reacting to an event (regardless of their experience and driving experience) based on immersion in simulated traffic conditions, allowing one to anticipate potential emergency situations in conditions as close as possible to real ones using a virtual reality helmet;

- ensuring maximum immersion of consciousness into the simulated reality during the learning process, excluding the influence of external interference;

- testing skills in complex dynamic road situations, allowing, on the one hand, to obtain a full view using a virtual reality helmet (front, back, side), and on the other hand, to track the direction of the driver’s gaze using an eye tracker to diagnose the causes of errors, take into account level of development of the skill of identifying a potentially dangerous situation by fixing saccades and thermal contours formed on their basis;

- the ability to create situations of varying degrees of complexity and road conditions, taking into account the class of vehicle (passenger car, truck, bus, car with trailer), which makes it possible to teach the skill of identifying a potentially dangerous situation and the skill of reacting to an event for both novice motorists and/or recertification, changing the category of professional drivers;

- formation of certain reactions in practical activities based on anticipation of potentially dangerous emergency situations;

- increasing the reaction window time in order to prevent diagnostic errors when testing professional drivers;

- expanded diagnostics of all parameters of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle (recognition of a potential danger (saw, saw mistakenly, did not see), the correctness of the decision made (how correct the choice of maneuver was - braked, accelerated, stopped, etc. .), speed reaction);

- diagnostics of risky and dangerous driving skills;

- differentiation of the reasons for the lack of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle (inability to recognize a potential danger, incorrect decisions or low reaction speed) and the opportunity to develop skills through an emphasis on training the ability to identify a potential danger or correct decisions;

- the possibility of providing the results of measuring the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event of the driver to the employer for making management decisions during employment and periodic certification of drivers;

- prompt automatic processing of results and provision of a comprehensive summary of the level of development of the skill of identifying a potentially dangerous situation and the skill of responding to an event based on clear assessment criteria;

- lack of habituation to the stimulus material of tests in simulated reality;

- intuitive interface of the device (the instructions are simple and understandable to a person with primary education);

- the possibility of non-contact training in the skill of identifying a potentially dangerous situation and the skill of reacting to an event, ensuring the safety and efficiency of the method in terms of resources: time, gasoline consumption, wear of spare parts, attracting a driver-instructor; as well as the safety of all road users, which is impossible in a real situation, including defensive driving training;

- automation of the process (ensures the objectivity of the result and the absence of the opportunity to “agree” with the supervisor, the absence of bias), the absence of the need to attract a qualified specialist to accompany training or testing, as well as interpretation of the results.

Brief description of drawings

This solution is illustrated by the following figures:

Fig. 1-3 - time lines with marked key points are shown: 1 - the beginning of the event; 2 - end of event; 3 - critical moment; 4 - moment of concentration; 5 - user input.

The given figures are not limiting for the present invention and are provided only for a better understanding of the essence of the invention.

Implementation of the invention

The present invention includes several main stages of a method for teaching and measuring the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, which are shown in Fig. 1 in block diagram form. In particular, identification of the subject, communication of at least one message associated with the stimulus material; demonstrating at least one type of stimulus material; recording at least one response of the subject; checking the correctness of at least one answer of the subject; recording the time spent by the subject on at least one answer; measuring the level of development of the skill of identifying potentially dangerous situations based on the timeliness of at least one answer of the subject and the correctness of at least one answer of the subject; measuring the level of development of the skill to make correct/correct decisions based on the timeliness of at least one answer of the subject and the correctness of at least one answer of the subject; measuring reaction speed based on the time taken by the subject to make at least one answer and the correctness of at least one answer by the subject; determination by the formula of an integral assessment of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle; determining the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle based on indicators: 1) the level of formation of the skill of identifying potentially dangerous situations, 2) the level of formation of the skill to make the right decisions on driving a vehicle, taking into account the identified danger, 3) reaction speed.

Within the scope of the present invention, it should be understood that a potentially dangerous situation is generated by an event. An event includes all stages of a potentially dangerous situation, i.e. the occurrence of the prerequisites for the event (for example, the appearance of a car moving at high speed), the development of the event (the car approaching at high speed, making maneuvers, etc.), the outcome of the event (an accident or a positive outcome, for example, it was possible to avoid a potentially dangerous car) .

The present invention uses a helmet and virtual reality technology to immerse in specially simulated complex dynamic traffic conditions, close to real ones and allowing, on the one hand, to obtain a complete overview of the traffic situation using a virtual reality helmet (in front, behind, on the sides of traffic), and on the other hand, eliminate the influence of external interference. The virtual reality technology in our invention is represented by the cross-platform computer game development environment Unity, which works with Microsoft Windows, macOS and Linux operating systems. The main advantages of Unity are the presence of a visual development environment, cross-platform support and a modular component system. Using this technology, software is assembled that creates complete immersion of attention, thanks to the advantages of Unity, namely the modeling of the laws of physics, the physical properties of objects and other capabilities of virtual reality technologies, which increases the efficiency of learning and the objectivity of diagnosing the formation of a skill. The programming language used is C#, C++.

Eye tracking, or oculography, is a technology that allows you to record the movements of a person’s gaze. Testing with modern eye trackers takes place unnoticed by a person, without causing inconvenience or distracting from the task. An eye tracker is a special video camera that continuously records the direction of gaze. The direction of gaze is determined contactlessly by the relationship between the center of the pupil and the reflection in the cornea of the infrared illumination built into the eye tracker. This ratio changes when the direction of gaze changes, but remains unchanged when the head moves. Machine learning algorithms process the stream of images generated by the camera and determine exactly where the eye focuses. As a rule, analytical studies use data from “fixations” and “saccades”. Fixation is the moment when a participant's pupils linger on a UI element long enough to process it. A saccade is the moment when the participant's eye moves and wanders around the interface without fixating on anything.

In the present invention, an eye tracker built into a virtual reality helmet is used to analyze the visual search strategy and solve the problem: how the driver “reads” the road situation and anticipates its dangerous development, concentrating on the sources of danger and ways to avoid it. The eye tracker allows you to identify gaze fixations and saccades, thereby determining the direction and movement of the gaze in relation to the search and identification of potentially dangerous actions of road users / road situations that can lead to an accident, identifying the reasons for distracting attention from potentially dangerous actions of road users and committing errors when maneuvering.

In the present invention, a joystick is used to simulate driving a vehicle.

The process of a method for integral assessment of the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, which includes several stages: identification of the subject, familiarization testing (including maneuvering training), receiving feedback, testing the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle.

Login to the simulator is carried out on a computer.

After entering the simulator, windows appear for entering personal data (last name, first name, patronymic, date of birth). Below is the “Continue” button. This button is activated after entering data in all proposed windows.

After clicking the “Continue” button, a window opens with a view of the image from the camera for verification and the “Take photo” button. After clicking the “Take Photo” button, a photo and two buttons are displayed on the screen: “Retake” or “Done”. If you select the “Reshoot” button, the previous step is repeated. In one embodiment, a photo can be processed and used as an identifier, for example, similar algorithms are used in FaceID technologies (for example, on cell phones).

After clicking the “Finish” button, the instruction appears: “Place the joystick in front of you and put on the virtual reality helmet.”

After the test taker puts on the helmet, he sees the following instructions: “Before you begin testing the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, you are asked to undergo an introductory test in order to obtain information about the testing procedure. You will be presented with a game in which you control a car using a joystick. Your task is to get to your destination as quickly as possible (up to 3 minutes) without violating traffic rules and traffic accidents, despite the dangerous situations created by other road users or the characteristics of the road surface/condition. The movement process will be accompanied by sound navigation. The monitor will show the estimated time of arrival at your destination. The driving speed must correspond to the average speed of the traffic flow. You can determine the average speed of the flow by the number of cars overtaking you that are moving at the prescribed (permissible) speed.

Maneuvering is carried out using buttons located on the joystick. Carefully familiarize yourself with the purpose of each joystick button.

Now, to pass the familiarization test, you will be asked to perform certain vehicle maneuvers using the control buttons located on the joystick. To start maneuvering training, click “Maneuvering Training.” To repeat the instructions, click “Repeat instructions.” If you select “Repeat instructions,” the previous step is repeated.

If you select “Maneuvering training,” the game starts in virtual reality with a view corresponding to the view from the driver’s seat of a car moving along the road, and a visual display of the car’s dashboard, rear-view mirrors (right, left, center) and sound corresponding to the road situation and driver actions. During the game, the following voice commands are presented sequentially: “Start moving”, “Move into the far left lane”, “Turn around”, “Move into the far right lane”, “Turn right”, “Slow down before the intersection”, “Turn left” " To make it easier to select maneuvering buttons on the joystick, when a command is initially presented in the game, hints appear indicating the required button; when a command is presented again, hints appear only if the test taker does not execute the command for more than 5 seconds. At the end of the game, you are offered a choice of two alternatives:

“I confirm that I have memorized the purpose of the maneuvering buttons and am ready to begin introductory testing of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle” or “I want to repeat the maneuvering training.”

If Maneuvering Training is selected, the previous step is repeated.

In case of confirmation of readiness for introductory testing of the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, the following instructions are presented: “Now you will undergo an introductory test of the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle. You will be presented with a game in which you drive a car. Your task is to get to your destination as quickly as possible (up to 3 minutes) without violating traffic rules and traffic accidents, despite the dangerous situations created by other road users or the characteristics of the road surface/condition. The movement process will be accompanied by sound navigation. Please note that in this game you will independently choose the optimal strategy for moving to your destination, according to sound navigation. Consequently, you independently make the decision to perform the most optimal maneuver in your opinion. The monitor will show the estimated time of arrival at your destination. The driving speed must correspond to the average speed of the traffic flow. If you drive at a speed higher than prescribed by the traffic rules, you will be deducted points. If the speed is too low, you will not be able to reach your destination in the time allotted for the game, and, therefore, will not pass the test. You can determine the average speed of the flow by the number of cars overtaking you that are moving at the prescribed (permissible) speed. You are driving too slowly if you are overtaken by more than five vehicles."

Below the instructions is the “Start” button. When you press the “Start” button, the game starts in virtual reality with a view corresponding to the view from the driver’s seat of a car moving along the road, and a visual display of the car’s dashboard, rear-view mirrors (right, left, center) and sound corresponding to the road situation and driver actions.

Upon completion of the game, it is proposed to familiarize yourself with the results of introductory testing of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle.

After reviewing the results of the introductory testing of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, a choice of two alternatives is offered:

“I confirm that I am ready to begin testing the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle” or “I want to repeat the introductory testing of the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle.”

If you choose to repeat the “Familiarization Test”, the previous stage is repeated.

In case of confirmation of readiness for testing the level of formation of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle, the following instructions are presented:

“Now you will undergo testing of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle. You will be presented with a game in which you drive a car.

The game scenario includes various road conditions: driving on highways, in city traffic, driving in various weather conditions (snow, rain, fog, night), etc.

Your task is to get to your destination as quickly as possible (up to 15 minutes) without violating traffic rules or traffic accidents, for which points will be reduced. You need to score as many points as possible.

During the game you will be presented with 10 of the most typical potentially dangerous road situations that occur on certain sections of the road.

Be prepared for unforeseen situations, such as violation of traffic rules by other road users (turning without turning on the turn signal, exceeding the speed limit, sudden braking of a car with a faulty brake light, a cyclist (scooter) entering a pedestrian crossing at speed, crossing the roadway by a pedestrian at a prohibiting traffic light signal, a pedestrian crossing the roadway in the wrong place), as well as to the peculiarities of the road situation (potholes on the roadway, road repairs). Be prepared for potentially dangerous situations that are more likely to occur in the traffic area in which you are located. With this in mind, you will be ready to notice and react in time by choosing the safest driving strategy.

The movement process will be accompanied by sound navigation. Please note that in this game you will independently choose the optimal strategy for moving to your destination, according to sound navigation. Consequently, you independently make the decision to perform the most optimal maneuver in your opinion.

The monitor will show the estimated time of arrival at your destination.

Your driving speed should match the average speed of traffic. If you drive at a speed higher than prescribed by the traffic rules, you will be deducted points. If the speed is too low, you will not be able to reach your destination in the time allotted for the game, and, therefore, will not pass the test. You can determine the average speed of the flow by the number of cars overtaking you that are moving at the prescribed (permissible) speed. You are driving too slowly if you are overtaken by more than five vehicles.

Focus on the game and maneuver with the joystick while respecting the traffic rules. Change your driving strategy according to the rules of the road any time you think the situation may become unsafe.”

Below the instructions is the “Start” button. When you press the “Start” button, the game starts in virtual reality with a view corresponding to the view from the driver’s seat of a car moving along the road, and a visual display of the car’s dashboard, rear-view mirrors (right, left, center) and sound corresponding to the road situation and driver actions.

The stimulus material is a virtual reality game, lasting up to 15 minutes, including 10 potentially dangerous road situations that require maneuvering.

During the game, the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle is determined/diagnosed according to the following formula:

Where

IntForesight - an integral assessment of the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event when driving a vehicle;

Si - situation of potential danger;

Attent - level of development of skill in identifying potentially dangerous situations;

Decis - the level of development of the skill to make correct/correct decisions on driving a vehicle, taking into account the identified danger;

SR - reaction rate.

The level of development of the skill of identifying potentially dangerous situations is determined by measuring the number of fixations in areas of potential danger (set during the creation of the game) and the time of concentration of attention on them (recorded by an eye tracker during the game in the form of fixations and saccades, in the form of a so-called “heat map” - areas of concentration) during the presentation of a scenario with a potentially dangerous situation.

The level of formation of the skill to make correct/correct decisions on driving a vehicle, taking into account the identified danger, is determined by correlating the selected movement strategy with the range of correct/correct decisions, to what extent the test taker made the correct decision out of all possible ones within this range.

The reaction speed is determined based on the time spent making a decision from the moment the development of a potentially dangerous situation begins and/or from the moment attention is fixed on a potentially dangerous situation until the moment the maneuver begins (more than 50% of fixations and saccades are in areas of potential danger). The results of each test taker are stored in an individual cumulative database.

Based on the testing results, the user and/or customer and/or developer are provided with feedback (information) about the level of development of the skill of identifying a potentially dangerous situation and the skill of responding to an event. Feedback includes information about: the level of development of the skill of identifying potentially dangerous situations, the level of formation of the skill to make the right decisions on driving a vehicle, taking into account the identified danger, and the speed of reaction.

Feedback can be provided in the form of a report. The report may include various data, in particular about: assessing the level of development of the skill of responding to at least one event, the correctness of the reaction, the speed of determining the identification of a potentially dangerous situation, the speed of the user’s reaction; direction of the user's gaze. Additionally, the report can contain data about user input throughout the testing, data about generated video sequences, audio sequences, etc., i.e. Based on the report, it is possible to restore testing data. In some options, it is possible to include additional data in the report, for example, recording the entire test in the form of a log, video recording, etc. In some embodiments, the user, developer, and customer may preempt the composition of the report. In addition, the report can be provided to the user, customer and developer in various formats, for example, the user will be largely interested in his level of skill development, reaction speed, and correctness of input, respectively, the report will contain only this data. The report for the customer may additionally contain data on events that were included in testing. The report for the developer may not contain data on the user’s reaction speed or level of development of his skill, because This is individual data specific to a specific user and a specific test, but the report for the developer may include data on user input, correctness of input, and a log of events and actions. A report for the developer will allow the developer to make adjustments to subsequent testing, eliminate software errors, and check and confirm the correct operation of the system.

In one of the embodiments of the invention, a method is used to determine the level of development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event, which includes several actions. Generation of stimulus material in the form of at least a video sequence. This action can be carried out on various equipment known to a person skilled in the art, for example, the generation of video footage can be carried out on a computer, using open or licensed databases of well-known computer games as a base. In this case, not only a video sequence can be generated, but also several video sequences (separate video sequences for the windshield of a vehicle, for a rear-view mirror, for side mirrors, for side windows, etc.), in addition, video sequences can be interconnected, i.e. e. driving past a tree, the user sees the tree moving from the windshield to the side window).

In addition to generating a video sequence, it can also generate other series, for example, an audio series, vibration when moving a railway track (if it is possible to control the user’s chair), simulating the tilt of the cabin when going uphill or descending from a mountain (if it is possible to change the angle of the chair or the entire room in which the user is taking the test).

Inclusion in the video sequence of at least one event that has a beginning at the first time point, an end at the second time point and a critical moment between the first and second time points. The invention checks the maturity of the user's skills, allows the user to be trained, and his readiness to test the vehicle to be tested by analyzing the user's behavior in response to the situation. In the context of the invention, a situation should be understood, for example, as a potentially dangerous road situation: a tree falling on the road, blocking part of the road, maneuvers by other road users (including in violation of the rules, for example, sudden braking with the rear brake lights off, crossing the road in the wrong place, etc.), the presence of a slippery section on the road (for example, one lane is completely covered with ice, and the other lane is free of ice). Such situations carry potential danger and an experienced driver (user) is able to see them and take actions to reduce or eliminate possible risks.

An event is characterized by several temporal characteristics. In fig. 1 shows a time line of a video sequence, the event begins at the point indicated by position 1, ends at the point indicated by position 2, and also at the critical moment, which is indicated by position 3. The end (2) of the event is the termination of an event that can have several outcomes, for example, fact of an accident and avoidance of an emergency. As an example to better understand the essence of the invention: End (2) can be characterized by the fact that the user drove around a car that suddenly stopped in front of him, as an additional example - the user, by driving around the car from the previous example, caused an accident, i.e. made the wrong maneuver. The event is also characterized by a critical moment (3), when the user’s actions can no longer prevent an accident (for example, the car will not have time to stop because the user began to perform this action too late). The critical moment (3) on the time line is located between the beginning (1) and the end (2) of the event.

Determining, for each event, at least one user gaze direction control zone and at least one valid user input. In order for a user to notice a vehicle approaching from the side, the user's gaze direction must be directed towards the appropriate area, as a second example, the user's gaze must be directed towards the rear view mirror to see a vehicle approaching him from behind. Thus, for each event, one or more control zones can be identified that the user must look into to identify a potentially dangerous situation. In particular, control zones can be located on one screen, on several separate screens, and can be characterized by all or part of the screen area.

Also, for each event there is at least one correct sequence of user actions (correct user input). When an event occurs, the user can drive around an unexpected obstacle to the left or right (any of the user inputs will lead to a positive completion of the event), stop (for this example, the third correct user input). Most events are characterized by several correct user inputs, and the user may first choose the option of avoiding an obstacle, but having doubts about the success of his maneuver, choose to stop; in this case, a conclusion can be made about an insufficient level of formation of the maneuvering skill at the same time as a sufficient level of formation of the emergency braking skill.

Demonstration to the user of a video sequence that includes at least one event. The demonstration can be carried out as described in the examples above: on a monitor or several monitors, using glasses or a virtual reality helmet, using augmented reality devices, etc.

Receiving from the user state monitoring unit at least data about the direction of the user's gaze. In particular, the eye tracking technology described above can be used. Additionally, the user's condition monitoring unit can measure other user parameters, for example, dilation or constriction of the pupils, changes in pulse, blood pressure, body temperature, tremor, etc. All such data can be taken into account when analyzing the development of the skill of identifying a potentially dangerous situation and the skill of reacting to an event.

Receiving user input data from the user input control unit. The user input device or devices can be: keyboard, joysticks, mice, steering wheels, pedals, eye control technologies, spatial manipulators (for example, such as those used in PlayStation or XBox) and other input devices. In this case, the user input control unit can record user input simultaneously from several input devices (for example, steering wheel and pedals), as well as the absence of user input on one, several or all input devices).

Changing footage in response to the presence and/or absence of user input. The video sequence should change depending on the user's actions, for example, when the user input corresponding to a turn to the left, the video sequence should show the car moving into the left lane of the virtual road, similarly, when receiving input on the pedal unit corresponding to the brake pedal, the video sequence should simulate slowing down. In the absence of user input, the video sequence can also change, for example, when driving down a mountain, there should be an imitation of an increase in car speed.

In addition, other rows can also change depending on the user's input or lack of input, for example, when crossing a railway track at low speed, two small jolts of the user's chair can be generated, and if the user increases the speed with the joystick before moving, then the amplitude chair shocks can be increased, the volume of the audio series can also be increased and the sounds of the car suspension activated can be added to the audio series.

Determining the point in time of the user's attention in response to an event. Determining the moment in time of the user's attention in response to an event is carried out by correlating data on the direction of the user's gaze and the control zone, and in some embodiments is also carried out by analyzing user input. The criterion for assessing the recognition of a hidden (open) source of danger is, first of all, the presence of fixations and saccades in the control zone, indicating the identification of a potentially dangerous situation, and, in a smaller area, a heat map, since the driver’s attention is generalized along the path of the vehicle.

In some embodiments of the invention, control zones are defined in conjunction with time frames, i.e. the control zone, for example, is the left side mirror (or part of it), and the event begins at the first time point and lasts 20 seconds (until the second time point). If the user's attention is focused on the left side mirror before the event begins, then it cannot be concluded that the moment of concentration corresponds to the event, because the event has not yet started. Similarly with the second moment of time, because the event will already end. In some embodiments, a moment of attention that is outside the time interval between the first point in time (the onset of the event) and the critical point in time is not captured.

In addition, the present invention involves training not only the user, but also feedback from the user to the program and devices with which the user interacts, for example, widening or narrowing control zones, expanding the range of valid user inputs, etc. Accordingly, such training may employ a neural network that will take into account the user's (or users') previous experience, make averages, and improve subsequent user interactions with the present invention.

In some embodiments, the implementation under the correct user input does not have specific sequences of user actions and time intervals in which the user must perform these actions. Valid user input can be defined as any user input that results in a positive outcome for the event. For example, for an invention related to testing and training to drive a car, positive user input could be any action (turning the steering wheel, pressing the pedals, turning on the turn signals, tilting the joystick, etc.) that avoided a traffic accident. Accordingly, the user is free in his choice, but the choice (user input) must ensure that there is no accident.

In fig. Figure 2 shows the moment (4) of the user’s concentration time. With optimal user actions, moment (4) of the time of concentration should be before the critical moment (3).

Recording the time of user input or lack of user input is important to the invention because Based on knowledge of the moment in time of user input or knowledge of the absence of user input, it is possible to determine the speed of the user's reaction (or lack of reaction).

In fig. Figure 3 shows the moment (5) of user input. One skilled in the art will understand that in an optimal situation, moment (5) of user input is located on the time line between moment (4) of attention time and critical moment (3). It is clear that moment (5) of user input may be located further to the right on the timeline than critical moment (3), however, this will mean that the user’s reaction is late and will no longer lead to a positive resolution of a potentially dangerous situation.

User input changes the course of the event, the video sequence and other series (stimulus material in whole or in part), so that the end of the event can end, for example, with an accident (incorrect user actions, delayed user actions, etc.) and the continuation of the method (timely and correct user actions), for example, to continue training or testing, including generating the next event (potentially dangerous situation). Thus, with optimal user actions, the method involves repeating its main stages, and the user’s actions can be assessed both for each event separately and for the entire training or testing period, i.e. for all events that were presented to the user.

If we consider Fig. 3, then the time distance between moments 1 and 2 (the beginning and end of the event) is the duration of the event; between 1 and 3 (the beginning of the event and the critical moment) - the allowable time for identifying a potentially dangerous situation, making a decision on driving a vehicle taking into account the identified danger and performing a maneuver; 1-4 (between the beginning and concentration of attention) - time for the user to identify a potentially dangerous situation; between 4 and 3 (concentration of attention and critical moment) - the permissible time for making a decision on driving a vehicle, taking into account the identified danger and performing a maneuver; between 4 and 5 (attention concentration and user input) - the time of making a decision to drive a vehicle, taking into account the identified danger and the user’s reaction speed; between 5 and 3 (user input and critical moment) - time of development of the traffic situation, taking into account correct/incorrect user input; between 3 and 2 (critical moment and end of the event) - the time of development of the traffic situation, taking into account the absence of user input.

The following options are possible:

- the user (driver) did not see (did not identify) a potentially dangerous situation (event) and, accordingly, did not react (lack of user input);

- the user did not see (did not identify) a potentially dangerous situation, but reacted to an insignificant situation (user input does not correspond to the range of correct user inputs, in particular, without taking into account the point in time of the user input), i.e. with user input that corresponds to the range of correct user inputs, but when the user's attention is concentrated outside the control zone, the assessment of the user's actions may be negative, because actions are random in relation to the event and potentially dangerous situation);

- the user saw (identified) a potentially dangerous situation, reacted in a timely and correct manner;

- the user saw the situation correctly, but did not react in time (inappropriate user input: premature, late);

- the user saw the situation in time, but reacted incorrectly (incorrect user input);

- the user saw the situation and took no action.

In some embodiments, the critical moment of attention specific to the event is also taken into account, because After this point in time, even the user's identification of a potentially dangerous situation will prevent him from making user input. In some embodiments of the invention, the presence of user input, user state data may not be recorded, because no user actions or reactions will allow the event to be completed positively.

Assessment of the level of development of the skill of responding to at least one event is carried out on the basis of:

- comparison of the moment of attention concentration with at least the time of the beginning of the event and/or the critical moment and/or the second moment of time,

- comparing the time point of the user input with at least the time point of the beginning of the event and/or with the time point of attention and/or the critical moment and/or the second time point or absence of user input,

- comparing the user input to at least one valid user input.

In the invention, a comparison of a moment in time of concentration of attention to a critical moment and/or a second moment should be understood, for example, to determine the time difference between these moments. The moment of concentration must precede the second moment of time (the end of the event), because at the second moment of time an accident will already occur. Also, the moment of concentration must precede the critical moment, because otherwise, the user will be able to identify a potentially dangerous situation, but no user actions will avoid the accident. In some embodiments, the difference between the moment of attention and the first point in time (the onset of the event) may also be assessed, since This difference allows us to understand how quickly after the start of the event the user was able to identify the occurrence of a potentially dangerous situation.

Comparing a user input time point with at least a critical time point and/or a second time point or determining the absence of user input in some embodiments may also be determined as the difference between the respective times.

One skilled in the art will understand that there is no point in measuring the user's reaction speed if the user's concentration is not in the control zones (the correct zones of attention to detect a potentially dangerous event).

The invention is industrially applicable and can be implemented and applied in various fields of technology, in particular, in the field of training and testing of vehicle drivers.

Claims (16)

1. A method for determining the level of development of the skill of identifying a potentially dangerous situation and the skill of responding to an event, including: generating stimulus material in the form of at least a video sequence; including in the video sequence at least one event having a beginning at a first time point, an end at a second time point, and a critical moment between the first and second time points; determining for each event at least one control zone of the user's gaze direction and at least one correct user input; showing the user a video sequence including at least one event; receiving from the user state monitoring unit at least data about the direction of the user's gaze and from the user input monitoring unit data about the user input; changing footage in response to the presence and/or absence of user input; determining the point in time of the user's attention in response to the event by comparing data about the direction of the user's gaze and the control zone and/or through user input; fixing the moment in time of user input or absence of user input; assessment of the level of formation of the skill of responding to at least one event based on comparison of the moment of concentration of attention with at least the first moment of time, and/or with the critical moment, and/or the second moment of time, comparison of the moment of time of user input with at least the first a point in time and/or a critical point and/or a second point in time and/or a point in time of concentration, or lack of user input, comparing the user input with at least one correct user input. 2. The method according to claim 1, in which the user’s reaction speed is determined as the difference between the first moment of time and/or concentration of attention and the moment of user input. 3. The method according to claim 1, in which the speed of determining the identification of a potentially dangerous situation is determined as a comparison of the first moment in time and the moment in time of concentration of attention. 4. The method of claim 1, wherein the correctness of the response is determined by comparing the user input with at least one correct user input. 5. The method according to claim 1, in which a video sequence simulating the movement of road transport is used as stimulus material, and at least one event is used a vehicle event selected from at least the following series: overtaking, turning around, turning, crossing the road, repair work on the road, release of gravel, splashes and/or sand from under the wheels, wheel explosion, loss of vehicle integrity, appearance of foreign objects on the road or side of the road, appearance of vehicles, people, animals on the road and/or side of the road, reduced visibility on the road and/or side of the road, and the video shown to the user includes at least one specified event. 6. The method according to claim 1, in which an audio sequence is added to the stimulus material, simulating at least the sound of a moving car, sounds inside the car and/or the environment. 7. Method according to any one of paragraphs. 1-6, in which a report is generated on at least one of: assessing the level of formation of the skill of responding to at least one event, the correctness of the reaction, the speed of determining the identification of a potentially dangerous situation, the speed of the user’s reaction; direction of the user's gaze, and the report or part of the report is shown to the user and/or saved for transmission to the developer and/or customer.

Images