JP7260440B2 - SIMULATOR CONTROL DEVICE, TRAIN OPERATION SIMULATION SYSTEM, TRAIN OPERATION SIMULATION METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to a simulator control device, a train operation simulation system, a train operation simulation method, and a program.

As a technique related to train driving training, a technique of simulating a simulated train (railway vehicle) operation and evaluating a user's checking action is known (see, for example, Patent Document 1).

JP 2014-71319 A

According to the technique disclosed in Patent Literature 1, it is possible to evaluate whether or not the trainee was able to properly direct his or her line of sight to the traffic signal or the like set in the simulated railway operation section. However, according to the technique of Patent Document 1, the abnormal event to be generated in the simulation is determined in advance, and all the trainees are simulated under the same scenario.

In most of the simulations for driving training, when an abnormal event is discovered, the trainee must stop urgently. In other words, the occurrence of an abnormal event is discovered by the trainee, and a temporary stop operation is performed, and the simulation is temporarily terminated. In other words, basically only one abnormal event can be simulated in one training session.

There are a wide variety of places to pay attention to during driving, depending on the timing. In other words, if all the trainees are allowed to simulate the same scenario, it is necessary to prepare a large number of scenarios so that all the trainees can simulate a wide variety of abnormal events. In addition, since driving training is given to a plurality of trainees, there are trainees who have mastered points to be careful of when driving and those who have not. It is difficult to say that it is efficient training for trainees who have acquired sufficient skills to detect the occurrence of a certain abnormal event and trainees who do not have such skills to undergo the same scenario simulation.

Therefore, the present invention solves the above problems, that is, a simulator control device, a train operation simulation system, and a train operation simulation that can generate an abnormal event only when the operator's attention is not directed. An object is to provide a method and a program.

In order to solve the above problems, one aspect of the simulator control device of the present invention is simulation control means for controlling the generation of display images displayed on the train driving simulator, and the response of the visual function of the operator of the train driving simulator. Reaction information acquisition means for acquiring detection results, and based on the detection results of the operator's reactions acquired by the reaction information acquisition means, the operator pays attention that matches the scene that is progressing in the train driving simulator. The simulation control means has an operator situation judgment processing means for judging whether or not the operator is paying attention according to the scene progressing in the train operation simulator by the operator situation judgment processing means. It is characterized by controlling generation of an image indicating an abnormal event when it is determined that there is no abnormal event.

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information regarding the line of sight of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means comprises the reaction information acquiring means. Based on whether or not the operator has not looked at the front for a predetermined time or longer in the detection result of the reaction acquired by It is characterized by determining whether

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information regarding the line of sight of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means comprises the reaction information acquiring means. Based on whether or not the operator is looking at one point in front for a predetermined time or longer in the reaction detection results obtained by It is characterized by judging whether or not there is.

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information regarding the line of sight of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means comprises the reaction information acquiring means. Based on whether or not the operator is looking at a point that requires attention in the scene that is progressing in the train driving simulator, in the response detection result obtained by the operator is progressing in the train driving simulator It is characterized by judging whether or not attention is paid in accordance with the scene.

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information on the pupil size of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means It is characterized in that it is determined whether or not the operator is paying attention that matches the scene that is progressing in the train operation simulator based on the size or the amount of change of the operator's pupil acquired by the information acquisition means. and

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information regarding the line of sight of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means comprises the reaction information acquiring means. It is characterized by determining whether or not the operator is paying attention that matches the scene that is progressing in the train driving simulator, based on the amount of movement of the operator's line of sight acquired by .

In another aspect of the simulator control device of the present invention, the reaction information acquiring means acquires information on the number of blinks of the operator of the train driving simulator as the operator's reaction, and the operator situation determination processing means acquires the reaction information It is characterized in that it is determined whether or not the operator is paying attention that matches the scene that is progressing in the train operation simulator, based on the number of blinks of the operator obtained by the obtaining means.

In another aspect of the simulator control device of the present invention, the reaction information acquisition means acquires a plurality of reaction detection results, and the operator situation determination processing means detects the operator's reaction detection results acquired by the reaction information acquisition means. are used to determine whether or not the operator is paying attention to the scene progressing in the train driving simulator.

Another aspect of the simulator control device of the present invention is that the simulation control means prepares in advance a scene showing an abnormal event in one of the scenarios that are progressing in the train driving simulator, and the operator situation determination processing means , When it is determined that the operator is paying attention to the scene that is progressing in the train driving simulator, control the generation of the display image of the scene that can be operated normally, and do not display the scene showing the abnormal event. It is characterized by

Another aspect of the simulator control device of the present invention is that the simulation control means does not predetermine a scene indicating an abnormal event in the scenario that is progressing in the train driving simulator, and the operator situation determination processing means allows the operator to: The present invention is characterized by controlling the generation of a display image showing an abnormal event when it is determined that the train operation simulator is not paying attention to match the scene in progress.

One aspect of the train driving simulation system of the present invention is that the train driving simulator displays simulation screen display means for displaying a display screen corresponding to each scene of a scenario progressing in the simulation; A driving operation simulator for performing an operation with reference to an image displayed by a screen display means, and a reaction detection means for detecting a reaction of the operator's visual function, wherein the simulator control device displays on the simulation screen display means. a simulation control means for controlling generation of a display image to be displayed; a reaction information acquisition means for acquiring detection results of the operator's reaction detected by the reaction detection means; and an operator's reaction acquired by the reaction information acquisition means. and an operator situation determination processing means for determining whether or not the operator is paying attention that matches the scene progressing in the train driving simulator based on the detection result of the simulation control means, When it is determined by the operator situation determination processing means that the operator is not paying attention in accordance with the scene progressing in the train driving simulator, the generation of an image showing a pseudo-abnormal event is controlled. Characterized by

One aspect of the train operation simulation method of the present invention includes a reaction information acquisition step of acquiring a detection result of the reaction of the visual function of the operator of the train operation simulator; Based on the detection result, an operator situation judgment processing step for judging whether or not the operator is paying attention that matches the scene that is progressing in the train driving simulator, and the processing result of the operator situation judgment processing step. and a simulation control step for controlling the generation of a display image to be displayed on the train driving simulator, and in the processing of the simulation control step, the operator can proceed in the train driving simulator by the processing of the operator situation determination processing step. It is characterized by controlling generation of an image showing a pseudo-abnormal event when it is determined that the user is not paying attention that matches the scene being played.

One aspect of the program of the present invention is to provide a computer for controlling a train driving simulator for conducting train driving training, a reaction information obtaining step of obtaining detection results of reactions of visual functions of an operator of the train driving simulator; Operator situation judgment for judging whether or not the operator is paying attention that matches the scene that is progressing in the train operation simulator based on the detection result of the operator's reaction acquired by the processing of the information acquisition step. and a simulation control step for controlling the generation of a display image to be displayed on the train driving simulator based on the processing result of the operator situation judgment processing step. When it is determined by the processing of the step that the operator is not paying attention to the scene progressing in the train driving simulator, processing for controlling generation of an image showing a pseudo abnormal event is executed.

According to the present invention, an abnormal event can occur only when the operator's attention is not directed.

It is a figure which shows the structure of the train operation simulation system 1. FIG. 2 is a diagram showing a functional configuration of a simulator control device 11; FIG. FIG. 4 is a diagram for explaining the direction of the line of sight of an operator; FIG. 4 is a diagram for explaining the direction of the line of sight of an operator; FIG. 4 is a diagram showing part of an example scenario of simulation; 4 is a diagram showing an example of an image showing a pseudo abnormal event displayed on the simulation screen display device 15; FIG. 4 is a diagram showing an example of an image showing a pseudo abnormal event displayed on the simulation screen display device 15; FIG. 4 is a diagram showing an example of an image showing a pseudo abnormal event displayed on the simulation screen display device 15; FIG. 4 is a diagram showing an example of an image showing a pseudo abnormal event displayed on the simulation screen display device 15; FIG. 4 is a flowchart for explaining simulation processing 1; 4 is a flowchart for explaining operator attentiveness determination processing 1; 10 is a flowchart for explaining operator attentiveness determination processing 2. FIG. 10 is a flowchart for explaining operator attentiveness determination processing 3. FIG. FIG. 4 is a diagram showing part of an example scenario of simulation; 9 is a flowchart for explaining simulation processing 2;

A train operation simulation system 1 including a simulator control device 11 according to an embodiment of the present invention will be described below with reference to FIGS. 1 to 15. FIG.

A train operation simulation system 1 that is an embodiment of the present invention will be described with reference to FIG.

If an abnormal event occurs during the simulation, the trainee should perform an emergency stop and a predetermined operation corresponding to the abnormal event, so the simulation is once terminated. For example, a trainee who has already mastered the points that must be paid attention to in a certain scene, in other words, who has the skill to discover abnormal events that are highly likely to occur in that scene, can have them occur at an early stage after the start of the simulation. If an appropriate operation (emergency stop) is performed in response to an abnormal event, the simulation cannot be continued. Therefore, efficient training cannot be carried out.

Therefore, in the train operation simulation system 1 which is one embodiment of the present invention, the reaction result of the visual function of the operator of the simulator who is a trainee, for example, the line of sight and the opening of the pupil is detected, and the attention of the operator is detected. Abnormal events that are easy for trainees to miss occur in the simulation only when they are not directed. Therefore, it is possible to generate only abnormal events that are difficult for each trainee to discover, so that it is possible to provide an optimum simulation for improving the skill of each trainee.

A train operation simulation system 1 includes a simulator control unit 2 , a train operation simulator unit 3 and a network 4 . The simulator control unit 2 includes a simulator control device 11 , an operation input device 12 and a display device 13 . The train driving simulator unit 3 includes a simulation screen display device 15 , a driving operation simulator 16 and a reaction information detecting device 17 . In the example of FIG. 1, only one set of the simulation screen display device 15, the driving operation simulator 16, and the reaction information detection device 17, which constitute the train driving simulator unit 3, is shown. If so, a plurality of these can be provided.

The simulator control device 11 of the simulator control unit 2 communicates with the train operation simulator unit 3 via the network 4, and based on the operation input of the user (such as an instructor regarding train operation) supplied from the operation input device 12, train operation is performed. It controls execution of simulation by the simulator unit 3 . Specifically, the simulator control device 11 acquires from the driving operation simulator 16 information about the operation of the simulator operator (a person who trains driving techniques, etc.), and from the reaction information detection device 17, the simulator operator's The detection result of the information indicating the response of the visual function during the simulation is obtained, and the display image displayed on the simulation screen display device 15 is controlled based on the detection result. Details of the functional configuration of the simulator control device 11 will be described later with reference to FIG.

The operation input device 12 is composed of, for example, input devices such as a keyboard, mouse, and touch pad, receives operation input from a user (instructor regarding train operation, etc.), and supplies the operation input to the simulator control device 11 .

The display device 13 displays information for prompting an operation input by the user (such as an instructor regarding train operation) using the operation input device 12 . Information for prompting operation inputs includes, for example, information for selecting simulation conditions such as simulation route names and weather. Also, the display device 13 displays information corresponding to the results of the simulation described later.

A simulation screen display device 15 of the train operation simulator unit 3 displays a simulation screen generated by the simulator control device 11 of the simulator control unit 2 .

The driving operation simulator 16 has a form that simulates a driver's seat or a crew room, and is provided with levers, buttons, instruments, etc. necessary for driving, etc. Equipped with the necessary equipment for The driving operation simulator 16 supplies the operation input executed by the simulator operator (the person who trains the driving technique) to the simulator control device 11 via the network 4 . Further, the driving operation simulator 16 may be provided with a voice data acquisition device such as a microphone to acquire the voice of the simulator operator and supply it to the simulator control device 11 via the network 4 .

The reaction information detection device 17 detects information indicating the reaction of the simulator operator's visual function during the simulation, and supplies the information to the simulator control device 11 . Here, as an example of the reaction of the operator's visual function during the simulation, the reaction information detection device 17 detects the movement of the simulator operator's line of sight and the size of the pupil, and supplies them to the simulator control device 11 . do. The reaction information detection device 17 may be realized as a non-wearable stationary device installed in front of the simulator operator as shown in FIG. may be realized as Also, the method of detecting the movement of the line of sight is not particularly limited. For example, by providing an imaging sensor in the reaction information detection device 17, photographing the simulator operator from the front, and processing the photographed image, it is possible to determine the orientation of the face and the orientation of the eyes and calculate the line-of-sight direction. is. Similarly, the size of the pupil can also be determined by image processing or the like. Further, the reaction information detection device 17 may be configured by integrating a sensor portion such as an imaging sensor and an arithmetic processing portion, or may be configured as separate devices. Further, the reaction information detection device 17 may be composed only of a sensor portion such as an image sensor, and arithmetic processing may be executed in the simulator control device 11 .

Next, the functional configuration of the simulator control device 11 will be described with reference to FIG.

The simulator control device 11 includes an operation input control unit 31, a simulation control unit 32, a control device display control unit 33, a reaction information acquisition control unit 34, a simulator operation input acquisition control unit 35, an operator situation determination processing unit 36, an operator It includes a situation recording unit 37 , a simulator display normal state image data storage unit 38 , a simulator display abnormal event image data storage unit 39 , a simulator screen generation unit 40 , and a simulator screen display control unit 41 .

The operation input control unit 31 (hereinafter referred to as the input control unit 31) controls acquisition of operation input from the user (such as an instructor regarding train operation) supplied from the operation input device 12, and simulates the acquired operation input information. It is supplied to the control section 32 .

The simulation control section 32 controls the operation of the simulator control device 11 based on the operation input supplied from the input control section 31 . Specifically, the simulation control unit 32 causes the display device 13 to display information for prompting a user (such as an instructor regarding train operation) to input an operation using the operation input device 12. (hereinafter referred to as a display control unit 33). In addition, the simulation control unit 32 causes the simulation screen display device 15 to display a display screen that matches the route, weather, etc., corresponding to the simulation scenario selected by the user, based on the operation input supplied from the input control unit 31. Therefore, the simulator screen generator 40 (hereinafter referred to as the screen generator 40) is controlled.

For example, when a driving operation for departure is performed in the driving operation simulator 16 at the start of the simulation, the simulation control unit 32 receives a signal from the operator situation determination processing unit 36 (hereinafter referred to as the determination processing unit 36). Based on this information, the screen generator 40 is controlled so that the image displayed on the simulation screen display device 15 is a moving image corresponding to the actual departure of the vehicle. Further, during execution of the simulation, when an operation input for driving such as changing the speed or braking is performed in the driving operation simulator 16, the simulation control unit 32 receives information supplied from the judgment processing unit 36 described later. , the screen generator 40 is controlled so that the image displayed on the simulation screen display device 15 corresponds to the driving operation executed by the simulator operator.

Based on the information supplied from the determination processing unit 36, the simulation control unit 32 controls the screen generation unit 40 to store the simulator display normal state image data storage unit 38 (hereinafter referred to as normal image data storage unit 38). and the image data corresponding to the scene of the scenario being executed is read from the abnormal event image data storage unit 39 for simulator display (hereinafter referred to as the abnormal image data storage unit 39), and the image displayed on the simulation screen display device 15 generate the data.

In addition, the simulation control unit 32 controls the display control unit 33 to display information corresponding to the simulation result on the display device 13, and controls the operator situation recording unit 37 (hereinafter referred to as the recording unit 37). The information about the operator's situation during the simulation recorded in is read out, and the display data corresponding to the result of the simulation desired by the user is generated and displayed on the display device 13 .

Under the control of the simulation control unit 32 , the display control unit 33 causes the display device 13 to display information for prompting a user (such as an instructor regarding train operation) to input an operation using the operation input device 12 . Information for prompting operation inputs includes, for example, information for selecting simulation conditions such as simulation route names and weather. In addition, the display control unit 33 causes the display device 13 to display information corresponding to the results of the simulation described later under the control of the simulation control unit 32 .

The reaction information acquisition control unit 34 (hereinafter referred to as the information acquisition control unit 34) is supplied from the reaction information detection device 17 via the network 4, the reaction information of the visual function of the simulator operator, that is, the movement of the line of sight and the Information about the size of the pupil is acquired and supplied to the determination processing unit 36 .

The simulator operation input acquisition control unit 35 (hereinafter referred to as the operation acquisition control unit 35) acquires information related to the operation input of the simulator operator, which is supplied from the driving operation simulator 16 via the network 4, and the judgment processing unit 36 supply to

The determination processing unit 36 receives information regarding the movement of the simulator operator's line of sight and pupil size supplied from the information acquisition control unit 34 and information regarding the operation input of the simulator operator supplied from the operation acquisition control unit 35. Based on this, the operator's situation is determined and supplied to the simulation control unit 32 . Specifically, the operator who is a trainee must look at the simulation screen display device 15 as shown in FIG. 3 while the simulation is in progress. You must not be in a state of being distracted by staring at the table and not looking ahead for a long time, not checking the necessary points corresponding to each scene of the simulation, or being distracted. In other words, the operator, who is a trainee, must keep his eyes on the front for a long time and must watch the area that matches the driving scene in the scenario. The determination processing unit 36 focuses on a specific area, such as when a predetermined time (for example, predetermined T seconds) has passed without the operator's line of sight directed to the display screen, or when the operator gazes at a specific area such as the entrance to the platform or before a railroad crossing. If the operator does not pay attention to the area that needs to be confirmed in the scene corresponding to the specific section that must be checked, it is determined that the operator is not paying sufficient attention to the scene. In addition, the determination processing unit 36 determines that the pupil is not dilated, that is, if the size of the pupil is smaller than a predetermined value (e.g., diameter N mm) and a predetermined time (e.g., predetermined T seconds) has passed, It is determined that the operator's attention is distracted, that is, the operator is not paying enough attention to the scene. In addition, the determination processing unit 36 receives information on the movement of the simulator operator's line of sight and the size of the pupil supplied from the information acquisition control unit 34, and information on the operation input of the simulator operator supplied from the operation acquisition control unit 35. The operator's situation obtained based on the information is supplied as a simulation result to the recording unit 37 to be recorded.

The recording unit 37 records the simulation results supplied from the judgment processing unit 36 .

The normal image data storage unit 38 stores image data corresponding to a normal state (not an abnormal event, but a state in which the train can be operated normally) to be displayed on the simulation screen display device 15 .

The abnormal image data storage unit 39 stores an image corresponding to a state in which an abnormal event (not a state in which the train can be operated normally but an abnormal event requiring an emergency stop or the like) to be displayed on the simulation screen display device 15 has occurred. Store data.

Based on the control of the simulation control unit 32, the screen generation unit 40 reads the image data stored in the normal image data storage unit 38 and the abnormal image data storage unit 39, and displays the image data for simulation on the simulation screen display device 15. image data is generated and supplied to the display control unit 41 .

A simulator screen display control unit 41 (hereinafter referred to as a display control unit 41) supplies the simulation image data generated by the screen generation unit 40 to the simulation screen display device 15 via the network 4, and displays the data. Control.

Examples of images generated by the screen generation unit 40 and displayed on the simulation screen display device 15 by controlling the display by the display control unit 41 when an abnormal event occurs are shown in FIGS. 6 to 9. will be used later.

The abnormal event generation control that causes an abnormal event that is likely to be overlooked by the trainee in the simulation only when the operator's attention is not directed, which is executed by the simulator control device 11 having the functional configuration described above, includes, for example, Two forms can be used.

A first form is configured assuming that an abnormal event occurs in a predetermined scene in a simulation scenario. At the timing immediately before the abnormal event occurs, the judgment processing unit 36, based on the response information of the simulator operator's visual function supplied from the information acquisition control unit 34, when the image of the abnormal event is displayed, It is determined whether or not there is a high possibility that the operator can recognize the abnormal event from the image, and the determination result is supplied to the simulation control section 32 . When the simulator operator has a high possibility of discovering an abnormal event scheduled to occur in a predetermined scene, the simulation control unit 32 controls the screen generation unit 40 to skip the scheduled occurrence of the abnormal event. By doing so, only abnormal events that are easily overlooked by trainees are generated in the simulation.

In the second mode, the location of the occurrence of the abnormal event is not determined in advance in the simulation scenario, and the simulation is performed only when the judgment processing unit 36 judges that the simulator operator is not paying sufficient attention. In this mode, the control unit 32 controls the screen generation unit 40 to generate abnormal events corresponding to the scene, thereby generating only abnormal events that trainees tend to overlook in the simulation.

An abnormal event that is likely to be overlooked by the trainee is generated in the simulation only when the operator's attention is not directed, that is, when the operator's attention is directed to a previously prepared abnormal event A specific example of implementation of the first mode in which the occurrence of an abnormal event is skipped will be described later with reference to FIGS. In addition, in the simulation scenario, the place where the abnormal event occurs is not determined in advance, and if it is determined that the operator's attention is not directed, the abnormal event corresponding to the scene in progress is generated in the simulation. A specific example of the implementation of the second embodiment will be described later using FIGS. 14 and 15 and also referring to FIGS. 6 to 9 and FIGS.

Next, with reference to FIGS. 5 to 9, an example of image display on the simulation screen display device 15 corresponding to the progress of the simulation in the above-described first mode and the conditions such as the line of sight and pupil dilation of the simulator operator. will be explained.

First, FIG. 5 shows part of an example of a simulation scenario selected by a user (instructor on train operation, etc.) in the first mode.

Since there are railroad crossings, platforms, tunnels, etc. on a railway line, there are corresponding scenes in the simulation as well, as shown in FIG. There are certain intervals that need to be watched. In addition, as shown in FIG. 5, railroad lines have track sections other than the specific sections described above. Here, in the simulation, it is assumed that an abnormality occurs in advance at the locations indicated by A to D in FIG.

In a scene where the platform is visible in front of the operator, the operator can check whether a special signal is lit to notify that there is an abnormality on the station platform, where the stop position is, whether there is an abnormality on the platform, and whether there is an abnormality in front of the platform. It is necessary to pay close attention to various parts, such as whether people or objects are encroaching on the tracks. In scenario A in FIG. 5 (a scene in which the platform is visible in front), for example, as shown in FIG. It is assumed that an abnormal event is prepared.

In addition, the operator must keep his eyes on the entire area in front of the scene other than the scene corresponding to the specific section, that is, even in the scene of the railroad section. As described with reference to FIG. 4, the timetable or the like is watched for a certain period of time or more, or only in a certain direction within the display screen of the simulation screen display device 15, specifically, on the downward track or upward. Looking only to the right or left of the overhead wire, or lacking attention even if the line of sight is in front of you. should not. In scenario B in FIG. 5 (a scene corresponding to a normal track section), for example, as shown in FIG. Abnormal events shall be provided.

Also, in scenes where the tunnel entrance is visible in front of the operator, it is necessary to pay close attention to whether there is any sign of abnormality occurring at the tunnel entrance or inside, but not only at the tunnel entrance , and its surroundings, such as railroad tracks and overhead wires, must also be carefully observed. Therefore, in scenario C (tunnel entrance scene) in FIG. 5, for example, as shown in FIG. It is assumed that there is

Furthermore, even in scenes other than the scene corresponding to the specific section, that is, even in the scene of the railroad section, the operator needs to be careful of landslides and fallen trees depending on the weather and topography. In this scenario, in D in FIG. 5 (a scene corresponding to a normal track section), for example, as shown in FIG. It is assumed that an abnormal event is prepared.

Since an abnormal event may occur in any section, the trainee must keep his eyes on the simulation screen display device 15 as shown in FIG. As such, the driver must not be in a state where he does not look ahead for a long period of time, or his attention is distracted. In other words, the operator, who is a trainee, must keep his eyes on the front for a long time and must watch the area that matches the driving scene in the scenario. The simulator control device 11 detects the movement of the simulator operator's line of sight and the size of the pupil supplied from the reaction information detection device 17 via the network 4 from a stage a certain time before the scene in which the abnormal event is scheduled to occur. As shown in FIG. 3, when the trainee correctly gazes at the simulation screen display device 15 and pays attention that matches the driving scene, in each scene in the scenario, an abnormal There is a possibility that a prepared abnormal event may be overlooked, such as when the trainee does not cause the event to occur and the trainee does not properly gaze at the simulation screen display device 15 as shown in FIG. is high, each scene in the scenario causes an anomalous event.

Specifically, the determination processing unit 36 receives information on the movement of the simulator operator's line of sight and the size of the pupil supplied from the information acquisition control unit 34, and the simulator operator's information supplied from the operation acquisition control unit 35. Based on the information about the operation input, the operator is not able to gaze at the part that matches the situation of each scene of the scenario currently in progress, or the operator has a small pupil, that is, the attention is reduced. It judges whether or not it is in a distracted situation, and supplies the judgment result to the simulation control section 32 . If the simulation control unit 32 determines that the operator is gazing at the correct portion based on the determination result supplied from the determination processing unit 36, the simulation control unit 32 displays a pseudo image on the display screen displayed on the simulation screen display device 15. do not cause any abnormal events. In other words, when it is presumed that the operator is not paying sufficient attention to each scene of the simulation scenario, a pseudo-abnormal event is generated according to the simulation scenario.

For example, in the ongoing scenario A in FIG. 5 (a scene in which the platform is visible in front), for example, as shown in FIG. ) is prepared, the judgment processing unit 36 receives the information from the information acquisition control unit 34 from a stage a predetermined distance before the scene progressing in the simulation scenario. It is determined whether or not the line of sight corresponding to the platform entrance is detected based on the information indicating the line of sight and pupil dilation, and the result of the determination is supplied to the simulation control unit 32 . When the line of sight corresponding to the platform entrance is detected and it is determined that the operator is paying sufficient attention to the scene, in other words, the prepared abnormal event of turning on the special signal is detected by the operator. If it is determined that the abnormal event can be easily detected, the simulation control unit 32 controls the screen generation unit 40 to acquire a suitable image from the normal image data storage unit 38, and the pseudo abnormal event has occurred. The simulation screen display device 15 is caused to perform processing for displaying an image that does not exist. That is, in the progress of the simulation, the occurrence of an abnormal event for which the special signal is lit is skipped. On the other hand, if the line of sight corresponding to the entrance of the platform is not detected, the simulation control unit 32 controls the screen generation unit 40 to inform the abnormality of the station platform supplied from the abnormality image data storage unit 39. A process for displaying an image showing a pseudo abnormal event such as lighting of a special signal on the simulation screen display device 15 according to the scenario is executed.

Also, for example, in B in FIG. 5 of the ongoing scenario (scene corresponding to a normal track section), as shown in FIG. If an abnormal event is prepared, the judgment processing unit 36, based on the information indicating the situation such as the line of sight and the opening of the pupil supplied from the information acquisition control unit 34, Actions such as not looking for more than a certain amount of time or looking at a certain point in the front for more than a predetermined amount of time are detected. It is supplied to the control section 32 . When the situation can be accurately determined, in other words, when it is determined that the operator can easily find the prepared abnormal event that an obstacle is attached to the overhead wire, simulation control is performed. The unit 32 controls the screen generation unit 40 to obtain a suitable image from the normal image data storage unit 38, and displays an image in which no pseudo abnormal event has occurred on the simulation screen display device 15. Let the process run. In other words, in the progress of the simulation, the occurrence of an abnormal event that an obstacle such as vinyl is attached to the overhead wire is skipped. On the other hand, if there is a high possibility that the situation cannot be accurately determined, the simulation control unit 32 controls the screen generation unit 40 to In accordance with the scenario, the simulation screen display device 15 executes processing for displaying an image showing an abnormal event with an obstacle attached.

Also, for example, in the ongoing scenario C (tunnel entrance scene) in FIG. 5, for example, as shown in FIG. is prepared, the judgment processing unit 36 starts from the stage where the scene progressing in the simulation scenario is a predetermined distance before the entrance of the tunnel, and obtains the line of sight, pupil opening, etc. supplied from the information acquisition control unit 34. Based on the information indicating the situation, it is determined whether or not the line of sight corresponding to the tunnel entrance has been detected, and the determination result is supplied to the simulation control unit 32 . When the line of sight corresponding to the tunnel entrance is detected, in other words, when it is determined that the operator will easily discover the prepared abnormal event that the worker is present at the tunnel entrance, the simulation control unit 32 controls the screen generation unit 40 to obtain a suitable image from the normal image data storage unit 38, and displays an image in which no pseudo abnormal event has occurred on the simulation screen display device 15. to run. That is, in the progress of the simulation, the occurrence of an abnormal event that a worker is present on the track (indicated by 63 in the figure) is skipped. On the other hand, when the line of sight corresponding to the tunnel entrance is not detected, the simulation control unit 32 controls the screen generation unit 40 to generate the image supplied from the abnormal image data storage unit 39, for example, shown in FIG. According to the scenario, the simulation screen display device 15 executes processing for displaying an image showing a pseudo abnormal event such as the presence of a worker on the track.

Also, for example, in the ongoing scenario D in FIG. 5 (scene corresponding to a normal track section), as shown in FIG. If an abnormal event is prepared, the judgment processing unit 36 reduces the pupil, based on the information indicating the situation such as the line of sight and the dilation of the pupil, which is supplied from the information acquisition control unit 34. Actions such as not looking for more than a certain amount of time or looking at a certain point in the front for more than a predetermined amount of time are detected. It is supplied to the control section 32 . If it is possible to accurately determine the situation, in other words, if it is determined that the operator will easily discover the abnormal event that the track has been buried by earth and sand, etc. The simulation control unit 32 controls the screen generation unit 40 to acquire a suitable image from the normal image data storage unit 38, and causes the simulation screen display device 15 to display an image in which no pseudo abnormal event has occurred. process for That is, in the progress of the simulation, the occurrence of an abnormal event such as a state in which the railroad track is buried by earth and sand (indicated by 64 in the figure) is skipped. On the other hand, if there is a high possibility that the situation cannot be accurately determined, the simulation control unit 32 controls the screen generation unit 40 to cause the railroad tracks to be buried by earth and sand, etc., as shown in FIG. According to the scenario, the simulation screen display device 15 is made to display an image showing a pseudo abnormal event such as a state that has occurred.

As described above, in the first embodiment, the simulation control unit 32 receives the conditions of the operator's driving operation supplied from the judgment processing unit 36 and the line of sight and pupil in correspondence with each scene in the scenario of the simulation. Skipping the display of an image indicating that a pseudo-abnormal event has occurred when it is estimated that there is no possibility of overlooking it in the current situation of the operator based on the information indicating the situation such as the opening of the That is, since the simulation can be continued without being displayed on the simulation screen display device 15, it is possible to provide the optimum simulation for improving the skill of each trainee.

Next, the simulation process 1 in the first mode executed in the train operation simulation system 1 will be described with reference to the flowchart of FIG. 10 .

In step S<b>1 , the operation input device 12 receives an operation input from a user (such as an instructor regarding train operation) and supplies it to the simulator control device 11 . The simulator control device 11 prepares to start a simulation that matches conditions such as route and weather based on user's operation input.

In step S2, the simulation control unit 32 of the simulator control device 11 controls the simulator screen generation unit 40 to start image generation according to the simulation scenario. The image generated by the simulator screen generator 40 is displayed on the simulation screen display device 15 of the train operation simulator unit 3 via the display controller 41 . In addition, the operation acquisition control unit 35 of the simulator control device 11 supplies information regarding the operation input from the driving operation simulator 16 to the judgment processing unit 36 via the network 4 . The judgment processing unit 36 supplies the supplied information regarding the operation input to the simulation control unit 32 . Based on the information supplied from the determination processing unit 36, the simulation control unit 32 controls the screen generation unit 40 to perform the simulation when an operation input for driving such as changing the speed or braking is performed. A simulation is performed based on the operation input, such as making the image displayed on the screen display device 15 correspond to the driving operation performed by the simulator operator.

In step S3, the judgment processing unit 36 judges whether or not the scene in which the abnormal event is prepared is a predetermined distance before the scene in the currently ongoing simulation. In step S3, if it is determined that the scene does not precede the scene in which the abnormal event is prepared by the predetermined distance, it is determined that the scene is in front of the scene in which the abnormal event is prepared by the predetermined distance. The process of step S3 is repeated until the

If it is determined in step S3 that the scene is a predetermined distance before the scene in which the abnormal event is prepared, then in step S4, the information acquisition control unit 34 transmits the line of sight of the simulator operator through the network 4. etc. is acquired and supplied to the determination processing unit 36 .

In step S5, at least one of the operator attentiveness determination processes 1 to 3 described later with reference to FIGS. 11 to 13, preferably all of the operator attentiveness determination processes 1 to 3 executed.

In step S6, the determination processing unit 36 performs at least one of the operator attention determination processes 1 to 3, which will be described later with reference to FIGS. Based on the results of all of the driver attention determination processes 1 to 3, it is determined whether or not the simulator operator is paying attention that matches the driving scene. In step S6, when it is determined that information such as the line of sight of the simulator operator does not pay attention that matches the driving scene, the determination processing unit 36 supplies the determination result to the simulation control unit 32. The scene progresses according to the scenario, including a pseudo-abnormal event as described with reference to FIGS. 6 to 9, and the process proceeds to step S8, which will be described later.

If it is determined in step S6 that the simulator operator is paying attention that matches the driving scene, the determination processing section 36 supplies the determination result to the simulation control section 32 in step S7. The simulation control unit 32 controls the screen generation unit 40 to acquire suitable image data from the normal image data storage unit 38, and for example, simulates a pseudo abnormal event as described with reference to FIGS. is not displayed on the simulation screen display device 15, but an image of a state in which normal operation is possible is displayed, and the simulation is continued.

In step S<b>8 , the determination processing unit 36 determines whether an emergency stop operation has been performed by the simulator operator based on the information supplied from the operation acquisition control unit 35 .

When it is determined in step S8 that the simulator operator did not perform the emergency stop operation, in step S9, the determination processing unit 36 stores the execution content of the simulation, the simulation operator's operation input, and , information such as the line of sight of the simulator operator is stored, the process returns to step S3, and the subsequent processes are repeated.

If it is determined in step S8 that the simulator operator has performed an emergency stop operation, the determination processing unit 36 supplies the supplied information regarding the operation input to the simulation control unit 32 in step S10. The simulation control unit 32 controls the screen generation unit 40 to cause the simulation screen display device 15 to display an image corresponding to a situation corresponding to an emergency stop or the like. and the process ends.

Next, the operator attentiveness determination process 1, which is the first process executed in step S5 of FIG. 10, will be described with reference to the flowchart of FIG.

In step S<b>21 , the judgment processing unit 36 adjusts the line of sight of the operator based on the information about the movement of the line of sight of the simulator operator and the size of the pupil supplied from the information acquisition control unit 34 . Determine if it is aimed at the screen. If it is determined in step S21 that the line of sight of the operator is directed toward the display screen, the process proceeds to step S6 in FIG.

When it is determined in step S21 that the line of sight of the operator is not directed to the display screen, in step S22, the determination processing unit 36 receives the movements of the line of sight of the simulator operator and the pupils supplied from the information acquisition control unit 34. Based on the information about the size, it is determined whether or not a predetermined time T seconds has passed while the operator's line of sight is not directed to the display screen. If it is determined in step S22 that the operator's line of sight is not directed to the display screen and the predetermined time T seconds has not elapsed, the process proceeds to step S6 in FIG.

If it is determined in step S22 that the predetermined time T seconds has elapsed while the operator's line of sight is not directed to the display screen, the determination processing unit 36 determines in step S23 that the operator does not match the driving scene. It is determined that the user has not paid the attention he or she paid, and the process proceeds to step S6 in FIG.

With such processing, the determination processing unit 36 can, for example, look away from the display screen of the simulation screen display device 15 and watch the timetable or the like for a certain period of time or more as described using FIG. When the line of sight of the operator is not directed to the display screen of the simulation screen display device 15, it can be determined that the operator is not paying attention that matches the driving scene.

Next, the operator attentiveness determination process 2, which is the second process executed in step S5 of FIG. 10, will be described with reference to the flowchart of FIG.

In step S31, the judgment processing unit 36 determines that the scene currently in progress is, for example, before a railroad crossing, before the entrance to the platform, before departure from the platform, before the entrance or exit of a tunnel, which is explained using FIG. It is determined whether the vehicle is traveling in a specific section corresponding to a specific scene of the simulation that the live should watch. If it is determined in step S31 that the vehicle is not traveling in the specific section, the process proceeds to step S6 in FIG.

When it is determined in step S31 that the vehicle is traveling in the specific section, in step S32, the determination processing unit 36 receives the information regarding the movement of the simulator operator's line of sight and the size of the pupil supplied from the information acquisition control unit 34. , it is determined whether or not the line of sight of the operator is directed to the driving scene, that is, to the area corresponding to the specific section. Specifically, for example, in a scene in which a railroad crossing is visible in front of the operator, the operator can confirm that no vehicle or person has entered the railroad crossing, or that there is no vehicle or person likely to invade the railroad crossing. , It is necessary to pay close attention not only to the inside of the railroad crossing, but also to the left and right of it. In addition, for example, in a scene where the platform is visible in front, the operator can check whether a special signal indicating an abnormality on the station platform is lit, where the stop position is, and whether there is an abnormality on the platform. It is necessary to pay close attention to various parts such as whether there are people or objects on the track in front of the platform. Also, for example, in a scene in which the entrance of a tunnel is visible in front of the operator, it is necessary to pay close attention to whether there is any sign of an abnormality occurring at the tunnel entrance or inside. In addition to this, there are various points that trainees must pay attention to as they approach each point, such as when departing from a platform or exiting a tunnel. If it is determined in step S32 that the line of sight of the operator is directed to the area corresponding to the driving scene, the process proceeds to step S6 in FIG.

If it is determined in step S32 that the operator's line of sight is not directed to the area corresponding to the driving scene, for example, the operator is looking at a certain point on the front surface for a predetermined time or longer, then in step S33 the determination processing unit 36 determines that the operator is not paying attention that matches the driving scene, and the process proceeds to step S6 in FIG.

Through such processing, the determination processing unit 36 can, for example, determine a state in which the operator's line of sight is not directed to the driving scene, that is, to the area corresponding to the specific section, and the operator is paying attention to the driving scene. It can be determined that no

Next, the operator attentiveness determination process 3, which is the third process executed in step S5 of FIG. 10, will be described with reference to the flowchart of FIG.

In step S41, the determination processing unit 36 determines whether or not the operator's pupil is larger than a predetermined N mm, based on the information regarding the movement of the simulator operator's line of sight and the size of the pupil supplied from the information acquisition control unit 34. to judge. If it is determined in step S41 that the operator's pupil is larger than the predetermined N mm, the process proceeds to step S6 in FIG.

If it is determined in step S41 that the operator's pupil size is smaller than the predetermined N mm, in step S42, the determination processing unit 36 determines the movement of the simulator operator's line of sight and the size of the pupil supplied from the information acquisition control unit 34. Based on the information about the depth, it is determined whether or not a predetermined time T seconds has passed while the visual pupil of the operator is smaller than N mm. If it is determined in step S42 that the operator's visual pupil is smaller than N mm and the predetermined time T seconds has not elapsed, the process proceeds to step S6 in FIG.

If it is determined in step S42 that the predetermined time T seconds has passed while the operator's visual pupil is smaller than N mm, the determination processing unit is not paid, and the process proceeds to step S6 in FIG.

Through such processing, the determination processing unit 36 can detect a state in which the pupils are not dilated, that is, a state in which it is assumed that the attention is distracted, as if the operator is not paying attention in accordance with the driving scene. can judge.

As described above, in step S5 of FIG. 10, at least one of the operator attentiveness determination process 1 to the operator attentiveness determination process 3, preferably all the processes are executed, so that the operator It is possible to judge whether or not the driver is paying attention that matches the driving scene.

In this way, the detection result of the reaction of the operator's visual function of the train operation simulator unit 3 is acquired (for example, step S3 in FIG. 10), and based on the acquired detection result of the operator's reaction, the operator , it is determined whether or not the operator is sufficiently paying attention to the scene progressing in the train driving simulator (for example, steps S5 and S6 in FIG. 10), and based on the result of determination processing, the operator determines whether the train If it is determined that the driving simulator unit 3 is paying enough attention to match the scene that is going on, the generation of an image that does not show a pseudo-abnormal event is controlled (for example, step S7 in FIG. 10). Therefore, if the operator is presumed to be paying enough attention to match each scene of the simulation scenario, the occurrence of the pseudo-abnormal event is skipped. Therefore, the operator can continue the simulation at least until it is determined that he or she is not paying enough attention to match the scene, so that more efficient training can be achieved.

Next, with reference to FIG. 14, progress of the simulation in the above-described second mode will be described.

In the first embodiment described above, the train operation simulation system 1 detects the line of sight and pupil opening of the operator of the simulator who is a trainee, and the operator detects the correct area corresponding to each scene of the simulation scenario. When it is determined that attention is directed to the scenario, the occurrence of the abnormal event prepared in the scenario is skipped, but the train operation simulation system 1, as a second form, can Instead of preparing a scene in which an abnormal event occurs in advance, the simulator operator, who is a trainee, responds to each scene in the simulation scenario based on the detection results of the operator's line of sight and pupil dilation. If it is determined that the correct area is not being attended to, an anomalous event corresponding to the scene can be generated.

That is, in the second mode, as shown in FIG. 14, the simulation scenario is not constructed such that an abnormal event occurs in a predetermined scene. The determination processing unit 36 determines whether the operator is unable to gaze at a portion that matches the situation of each scene of the scenario currently in progress, or whether the operator has a small pupil, that is, whether the operator's attention is distracted. Based on this, it is determined whether or not the operator is sufficiently paying attention to each scene, and the result of determination is supplied to the simulation control section 32 . When the simulation control unit 32 receives the judgment result from the judgment processing unit 36 that the operator is not paying sufficient attention to each scene, the simulation control unit 32 controls the screen generation unit 40 to generate abnormal image data. Image data corresponding to the scene is read out from the storage unit 39, image data corresponding to the state in which a pseudo abnormal event has occurred is generated, and displayed on the simulation screen display device 15. FIG.

An example of the pseudo abnormal event display corresponding to the situation such as the line of sight of the simulator operator and the dilation of the pupil is the same as the case described with reference to FIGS.

Specifically, for example, in the scene shown in FIG. 14 where a railroad crossing is visible ahead, the operator confirms that no vehicle or person has entered the railroad crossing, or that there is no vehicle or person likely to enter the railroad crossing. In order to confirm this, it is necessary to pay close attention not only to the inside of the railroad crossing, but also to the left and right of it. The determination processing unit 36 acquires the information indicating the situation such as the line of sight and the opening of the pupil supplied from the information acquisition control unit 34 from the stage a predetermined distance before the scene in which the simulation scenario enters the railroad crossing. Based on this, it is determined whether or not the line of sight corresponding to before the railroad crossing has been detected, and the determination result is supplied to the simulation control section 32 . When the line of sight corresponding to before the railroad crossing is not detected, the simulation control unit 32 controls the screen generation unit 40 to acquire suitable image data from the abnormal image data storage unit 39, and for example, when the car is at the railroad crossing. It causes the simulation screen display device 15 to perform processing for displaying an image showing a pseudo abnormal event such as entry.

In this case, for example, based on the detected direction of the operator's line of sight, an image showing a pseudo abnormal event such as a car entering a railroad crossing from a direction that the operator is not gazing at. can be displayed on the simulation screen display device 15 .

Further, for example, in the scene shown in FIG. 14 where the platform is visible in front, the operator can check whether a special signal indicating an abnormality on the station platform is lit or whether an abnormality has occurred on the platform. , It is necessary to pay close attention to various parts such as whether there are people or objects on the track in front of the platform. The determination processing unit 36 starts the scene progressing in the simulation scenario from a stage a predetermined distance before the platform entrance based on the information indicating the situation such as the line of sight and the opening of the pupil supplied from the information acquisition control unit 34. Then, it is determined whether or not the line of sight corresponding to the platform entrance has been detected, and the determination result is supplied to the simulation control section 32 . When the line of sight corresponding to the home entrance is not detected, the simulation control unit 32 controls the screen generation unit 40 to acquire suitable image data from the abnormal image data storage unit 39, and for example, using FIG. The simulation screen display device 15 is caused to display an image showing a pseudo abnormal event such as lighting of a special signal (indicated by 61 in the figure) that indicates an abnormality on the station platform.

In this case, for example, based on the detected direction of the operator's line of sight, if the operator is gazing only at the platform, an image indicating that an abnormal event such as the lighting of a special traffic signal has occurred is displayed. If the operator is watching only the special traffic signal, it is possible to display an image on the simulation screen display device 15 indicating that a pseudo-abnormal event is occurring on the platform.

Also, for example, in the scene shown in FIG. 14 where the entrance of the tunnel is visible in front, the operator needs to pay close attention to whether there is any sign of abnormality at the tunnel entrance or inside. The determination processing unit 36 starts the scene progressing in the simulation scenario from the stage a predetermined distance before the tunnel entrance, based on the information indicating the situation such as the line of sight and the dilation of the pupil supplied from the information acquisition control unit 34. Then, it is determined whether or not the line of sight corresponding to the tunnel entrance has been detected, and the determination result is supplied to the simulation control section 32 . When the line of sight corresponding to the tunnel entrance is not detected, the simulation control unit 32 controls the screen generation unit 40 to acquire suitable image data from the abnormal image data storage unit 39, for example, using FIG. A process for causing the simulation screen display device 15 to display an image showing a pseudo abnormal event such as the presence of a worker on the track (indicated by 62 in the figure) is executed.

In addition to this, there are various points that the trainee must pay attention to as they approach each point, such as when departing from the platform and exiting a tunnel, as shown in FIG. 14 . For example, when the line of sight corresponding to the tunnel exit is not detected a predetermined distance before the tunnel exit, the simulation control unit 32 controls the screen generation unit 40 to select a suitable image from the abnormal image data storage unit 39. Image data is acquired and, for example, a process for displaying an image showing a pseudo abnormal event such as falling rocks at the exit of a tunnel on the simulation screen display device 15 is executed.

Furthermore, even in cases other than these specific scenes, that is, the scene of the railroad section shown in FIG. Based on the information, actions such as pupil contraction, not looking at the front surface for a predetermined time or longer, or looking at a certain point on the front surface for a predetermined time or longer are detected. It determines whether or not there is, and supplies the determination result to the simulation control unit 32 . In the case where there is a high possibility that the situation cannot be determined accurately, the simulation control unit 32 controls the screen generation unit 40 to generate an image suitable for the scenario of the simulation at that point from the abnormal image data storage unit 39. Data is acquired, for example, as shown in FIG. ) or the like to cause the simulation screen display device 15 to display an image showing a pseudo-abnormal event such as a track being buried.

In this case, for example, based on the detected direction of the line of sight of the operator, if the operator is not looking at the upper part of the display screen, there is an abnormality in the overhead wire, or if the operator is looking at the lower part of the display screen. If the operator does not look at either the left or right part of the display screen, an abnormality has occurred in the railroad track, or if the operator does not look at either the left or right part of the display screen, an abnormality has occurred in a building, etc., beside the railroad track. It is also possible to cause the simulation screen display device 15 to display an image indicating that a pseudo abnormal event is occurring in the direction in which it is determined that it is not.

As described above, in the second embodiment, the simulation control unit 32, corresponding to each scene in the scenario of the simulation, provides the operator's driving operation status and line of sight and pupil distance supplied from the judgment processing unit 36. An image showing the occurrence of a pseudo-abnormal event is displayed on the simulation screen only when it is estimated that there is a high possibility that it will be overlooked in the current situation of the operator based on the information showing the situation such as opening. Since it can be displayed on the device 15, it is possible to efficiently generate a pseudo-abnormal event for the part where the trainee lacks the skill to which the trainee should pay attention, and perform training. As a result, it is possible to provide an optimal simulation for skill improvement of each trainee without increasing the number of simulations more than necessary.

Next, with reference to the flowchart of FIG. 15, the simulation process 2 in the second mode executed in the train operation simulation system 1 will be described.

In steps S61 and S62, the same processes as steps S1 and S2 of the simulation process 1 described using the flowchart of FIG. 10 are executed. That is, in step S61, preparation for starting the simulation is executed, and in step S62, execution of the simulation is started.

Then, in steps S63 and S64, the same processes as steps S4 and S5 of the simulation process 1 described using the flowchart of FIG. 10 are executed. That is, in step S63, information such as the line of sight of the simulator operator is acquired, and in step S64, at least one of the operator attentiveness determination processes 1 to 3 described with reference to FIGS. , Preferably, all of the operator attentiveness determination processes 1 to 3 are executed.

In step S65, the determination processing unit 36 executes at least one of the operator attention determination processes 1 to 3 described with reference to FIGS. Based on the results of all of the driver attention determination processes 1 to 3, it is determined whether or not the information such as the line of sight of the simulator operator is paying attention that matches the driving scene. In step S65, when it is determined that information such as the line of sight of the simulator operator is paying attention that matches the driving scene, the determination processing unit 36 supplies the determination result to the simulation control unit 32, so that the simulation , the process proceeds without generating a pseudo abnormal event as described with reference to FIGS. 6 to 9, and the process proceeds to step S67 described later.

If it is determined in step S65 that the simulator operator is not paying attention that matches the driving scene, the determination processing section 36 supplies the determination result to the simulation control section 32 in step S66. The simulation control unit 32 controls the screen generation unit 40 to acquire appropriate image data from the abnormal image data storage unit 39, and generates a pseudo abnormal event such as that described with reference to FIGS. is displayed on the simulation screen display device 15 .

In step S<b>67 , the determination processing unit 36 determines whether or not an emergency stop operation has been performed by the simulator operator based on the information supplied from the operation acquisition control unit 35 .

When it is determined in step S67 that the simulator operator did not perform the emergency stop operation, in step S68, the determination processing unit 36 stores the execution contents of the simulation, the simulation operator's operation input, and , information such as the line of sight of the simulator operator is stored, the process returns to step S3, and the subsequent processes are repeated.

If it is determined in step S67 that the simulator operator has performed an emergency stop operation, the determination processing unit 36 supplies the supplied information regarding the operation input to the simulation control unit 32 in step S69. The simulation control unit 32 controls the screen generation unit 40 to cause the simulation screen display device 15 to display an image corresponding to a situation corresponding to an emergency stop or the like. and the process ends.

In this way, the detection result of the reaction of the visual function of the operator of the train operation simulator unit 3 is acquired (for example, step S63 in FIG. 15), and based on the acquired detection result of the operator's reaction, the operator , it is determined whether or not the operator is sufficiently paying attention to the scene that is progressing in the train driving simulator (for example, steps S64 and S65 in FIG. 15). If it is determined that the driving simulator unit 3 is not paying enough attention to match the scene that is going on, the generation of the image showing the pseudo-abnormal event is controlled (for example, step S66 in FIG. 15). Therefore, in the situation where the operator pays attention that matches the situation of each scene of the scenario currently in progress, no pseudo abnormal event occurs in the simulation scenario, and the operator can Pseudo-abnormal events are generated when it is assumed that not enough attention has been paid to match each scene. Therefore, it is possible to efficiently generate a pseudo-abnormal event and conduct training for a lack of skills that the trainee should pay attention to. As a result, it is possible to provide a simulation in which an abnormal event occurs that is optimal for improving the skill of each trainee without increasing the number of simulations more than necessary.

As described above, in the simulator control device 11 capable of realizing the first form and the second form, the detection result of the reaction of the visual function of the operator of the train operation simulator unit 3 is acquired (for example, FIG. 10 and step S63 of FIG. 15), whether the operator is paying sufficient attention to the scene progressing in the train driving simulator based on the acquired detection result of the operator's reaction. (for example, steps S5 and S6 in FIG. 10 and steps S64 and S65 in FIG. 15), and the display displayed on the simulation screen display device 15 of the train operation simulator unit 3 based on the determination processing result Control image generation (eg, step S7 in FIG. 10 and step S66 in FIG. 15). That is, in the generation of the display screen, when it is determined by the determination processing unit 36 that the operator is not sufficiently paying attention to the scene progressing in the train operation simulator unit 3, in other words, the operation Only in situations where the person is unable to gaze at the part that matches the situation of each scene in the scenario currently in progress, or in a state where the pupil is small, that is, in a situation where the attention is distracted. A pseudo abnormal event occurs in the simulation scenario (for example, the process of step S7 in FIG. 10 is not executed, or the process of step S66 in FIG. 15 is executed), and the operator When it is estimated that sufficient attention is paid to the scene, no pseudo abnormal event is generated (for example, the process of step S7 in FIG. 10 is executed, or the process of step S66 in FIG. 15 is performed. no processing). Therefore, since the abnormal event does not occur at the timing when the abnormal event can be easily dealt with, the simulation can be continued, and it is possible to provide the optimum simulation for improving the skill of each trainee.

In addition, as the reaction of the operator, information about the line of sight of the operator of the train driving simulator is acquired, and based on whether or not the operator has not looked at the front for a predetermined time or longer in the detection result of the reaction, the operator In the train operation simulator unit 3, it is determined whether or not the user is sufficiently paying attention to the scene in progress (for example, step S22 in FIG. 11). , an abnormal event can be generated.

In addition, as the reaction of the operator, information about the line of sight of the operator of the train driving simulator is acquired, and in the detection result of the reaction, based on whether the operator is looking at a certain point in front for a predetermined time or longer, the operator However, it is determined whether or not the train driving simulator unit 3 is paying enough attention to match the scene that is progressing (for example, step 22 in FIG. 11 or step S33 in FIG. 12). Abnormal events can be generated even when attention is declining.

In addition, as the reaction of the operator, information about the line of sight of the operator of the train driving simulator is acquired, and in the detection result of the reaction, the operator finds a place that needs to be watched in the scene that is progressing in the train driving simulator unit 3. It is determined whether the operator is paying sufficient attention to the scene progressing in the train operation simulator unit 3 (for example, step 33 in FIG. 12). , when looking to the front and not paying the necessary attention in the scene, anomalous events can occur.

In addition, as a reaction of the operator of the train driving simulator unit 3, information about the size of the pupil is acquired, and based on the size of the pupil of the operator, the operator decides the scene progressing in the train driving simulator unit 3. (for example, step 42 in FIG. 13). Abnormal events can be generated.

In addition, a scene indicating an abnormal event is prepared in advance in one of the scenarios that are progressing in the train driving simulator unit 3, and the judgment processing unit 36 allows the operator to When it is determined that sufficient attention is paid to the scene, the simulation control unit 32 controls the generation of the display image of the scene in which normal operation is possible, and prevents the display of the scene showing the abnormal event (for example, Because of step 7) in FIG. 10, even if the simulation ends when the trainee performs an appropriate operation to the abnormal event prepared in the scenario in advance, the abnormal event occurs at a timing that can be easily dealt with. Therefore, a trainee who has the skill of detecting an abnormal event can continue the simulation and conduct efficient training.

In addition, without predetermining a scene indicating an abnormal event in the scenario that is progressing in the train driving simulator unit 3, the judgment processing unit 36 allows the operator to make a warning that matches the scene that is progressing in the train driving simulator unit 3. is not sufficiently paid, the simulation control unit 32 controls the generation of the display image showing the abnormal event (for example, step 66 in FIG. 15). Abnormal events can be generated at timings when attention is not paid, even in scenes where there is no scene.

In the above, as information indicating the reaction of the visual function during the simulation, information on the movement of the line of sight and the size of the pupil of the simulator operator is acquired, and if the operator does not look at the front for a predetermined time or more, the operator If the operator is looking at a certain point in the front for more than a predetermined time, if the operator does not look at the part that requires attention in the scene that is progressing in the train driving simulator, or if the operator's pupil is smaller than the predetermined size. In the case described above, the operator determines that he/she is not paying attention to the scene progressing in the train driving simulator. Other criteria may be used to determine whether or not attention is paid to the scene that is progressing in the train driving simulator.

For example, in the same manner as described above, when using information about the size of the pupil of the simulator operator as the information indicating the reaction of the visual function during the simulation, the operator, based on the change in the size of the pupil, It may be determined whether or not the attention is paid in accordance with the scene progressing in the train driving simulator. The pupil dilates while thinking about a problem and constricts when finding a solution to the problem. In addition, when the attention is distracted and in a so-called absent-minded state, the size of the pupil does not change. In addition, when the subject is aware of drowsiness, large low-frequency fluctuations are observed in changes in pupil size. Therefore, for example, when the width between the maximum value and the minimum value of the pupil size within a predetermined time period is equal to or less than a predetermined value, the judgment processing unit 36 determines the number of changes in the pupil size exceeding the predetermined value is less than or equal to a predetermined value, or the pattern of changes in pupil size matches a predetermined pattern obtained, for example, by machine learning, the operator enters the scene progressing in the train driving simulator. It can be determined that you are not paying consistent attention.

Further, for example, in the same manner as described above, when using information about the movement of the line of sight of the simulator operator as information indicating the reaction of the visual function during the simulation, the amount of movement of the line of sight is used as a reference when the operator moves the train It may be determined whether or not the driver is paying attention to the scene that is progressing in the driving simulator. The amount of movement of the line of sight increases when concentration is high. Therefore, the determination processing unit 36 determines, for example, when the amount of movement of the line of sight within a predetermined period of time is equal to or less than a predetermined value, when the amount of movement of the line of sight within the predetermined period of time tends to decrease, or when the amount of movement of the line of sight within the predetermined period of time is below a predetermined amount and tends to decrease, it can be determined that the operator is not paying attention consistent with the scene going on in the train driving simulator.

Further, for example, the number of blinks of the simulator operator may be detected as information indicating the reaction of the visual function during the simulation. The number of blinks is low when the concentration is high, and is very high in the tense state. For example, the reaction information detection device 17 is equipped with an imaging sensor to photograph the simulator operator from the front, and the photographed image is image-processed to detect the number of blinks of the simulator operator. The judgment processing unit 36 receives the detection result of the number of blinks of the simulator operator as information indicating the reaction during the simulation, and if the number of blinks within a predetermined time is equal to or greater than a first predetermined value, or When the number of blinks within a predetermined period of time tends to increase, or when the number of blinks is less than a predetermined number and tends to increase, the operator is paying attention to the scene that is progressing in the train driving simulator. can decide not to. Furthermore, the determination processing unit 36 determines that the operator is in an extremely tense state when the number of blinks within the predetermined time is equal to or greater than a second predetermined value, which is greater than the first predetermined value. It can also be determined that the driver is not paying attention to match the scene that is going on in the driving simulator.

Further, in the above, if at least one of the criteria for determining whether or not the operator is paying attention to the scene progressing in the train driving simulator matches, the operator , the operator is not paying attention to the scene that is progressing in the train driving simulator. It is also possible to determine whether consistent attention is being paid.

For example, if a predetermined number or more of the above-described plurality of judgment criteria are met, the judgment processing unit 36 determines that the operator is not paying attention to the scene progressing in the train driving simulator. can judge. In addition, the judgment processing unit 36 predetermines a plurality of combination patterns of the plurality of judgment criteria described above, and when the criteria that match the patterns are satisfied, the operator gives attention that matches the scene that is progressing in the train operation simulator. You can decide that you are not paying. Specifically, the determination processing unit 36 determines, for example, when the amount of change in the pupil diameter has decreased and the amount of movement of the line of sight has also decreased, or when the amount of change in the pupil diameter has increased but the amount of movement of the line of sight has decreased. It can be determined that the operator is not paying attention in line with the scene that is going on in the train driving simulator, such as when the amount of movement is reduced and the operator is not paying attention to the area that needs to be confirmed.

In this way, as a response of the operator's visual function, information on the size of the operator's pupil is obtained, and based on not only the size of the pupil but also its change, the operator can progress in the train driving simulator. Since it is determined whether or not the operator is paying attention that matches the scene in which the operator is present, when the operator is looking at the front or in a direction according to the scene but his attention is low, an abnormal event is detected. can be generated.

In addition, as the reaction of the operator, information on the line of sight of the operator is acquired, and based on the amount of movement of the line of sight of the operator, whether the operator is paying attention that matches the scene that is progressing in the train operation simulator. Since it is determined whether or not, an abnormal event can be generated when the operator's attention is low.

In addition, as an operator's reaction, information on the number of blinks is acquired, and based on the number of blinks of the operator, whether or not the operator is paying attention that matches the scene that is progressing in the train driving simulator is determined. is determined, an abnormal event can be generated when the operator's concentration is low.

In addition, a plurality of reactions of the operator are acquired, and a plurality of combinations of detection results of the reactions are used to determine whether or not the operator is paying attention that matches the scene progressing in the train driving simulator. Therefore, it can be determined with higher accuracy whether or not the operator is paying attention that matches the scene.

The series of processes described above can be executed by hardware or by software. When a series of processes is executed by software, the programs that make up the software are built into dedicated hardware, or various functions can be executed by installing various programs. It can be installed from a program recording medium, for example, in a general-purpose personal computer.

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be executed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

Moreover, the embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.

1... Train operation simulation system 2... Simulator control unit 3... Train operation simulator unit 4... Network 11... Simulator control device 12... Operation input device 13... Display device 15... Simulation screen display device 16... Driving operation simulator 17 Reaction information detection device 31 Operation input control unit 32 Simulation control unit 33 Control device display control unit 34 Reaction information acquisition control unit 35 Simulator operation input acquisition control unit 36... Operator situation determination processing unit 37... Operator situation recording unit 38... Normal state image data storage unit for simulator display 39... Abnormal event image data storage unit for simulator display 40... Simulator screen generation unit 41... Simulator screen display controller

Claims (11)

A simulator control device for controlling a train operation simulator for conducting train operation training,
controlling generation of a display image to be displayed on the train operation simulator based on a scenario in which a predetermined scene is provided in which an abnormal event indicating a state requiring an emergency stop is displayed;
simulation control means for terminating the driving training when an operator of the train driving simulator executes an emergency stop operation;
Reaction information acquisition means for acquiring a detection result of a reaction of the visual function of the operator of the train driving simulator;
An operation of determining whether or not the operator is paying attention that matches the scene that is progressing in the train operation simulator, based on the detection result of the operator's reaction acquired by the reaction information acquisition means. and a person situation determination processing means,
The simulation control means is
In the predetermined scene in which the abnormal event is displayed, if the operator situation determination processing means determines that the operator is not paying attention that matches the predetermined scene, the abnormal event is displayed based on the scenario. displaying an event, and if it is determined that the operator is paying attention that matches the predetermined scene, omitting the display of the abnormal event in the predetermined scene;
The simulator control device, wherein the indication of the abnormal event is an indication of an obstacle that may hinder train operation. The simulator control device according to claim 1,
The reaction information acquiring means acquires, as the reaction of the operator, information related to the line of sight of the operator of the train driving simulator,
The operator situation determination processing means determines whether or not the operator has not looked at the front surface for a predetermined time or longer in the reaction detection result acquired by the reaction information acquisition means. A simulator control device for determining whether or not attention is paid in accordance with a scene progressing in a train driving simulator. The simulator control device according to any one of claims 1 and 2,
The reaction information acquiring means acquires, as the reaction of the operator, information related to the line of sight of the operator of the train driving simulator,
The operator situation determination processing means determines whether or not the operator is looking at a certain point on the front surface for a predetermined time or longer in the reaction detection result acquired by the reaction information acquisition means. , a simulator control device that determines whether or not attention is paid in accordance with a scene that is progressing in the train driving simulator. The simulator control device according to any one of claims 1 to 3,
The reaction information acquiring means acquires, as the reaction of the operator, information related to the line of sight of the operator of the train driving simulator,
The operator situation determination processing means determines whether or not the operator is looking at a portion requiring attention in the scene progressing in the train operation simulator in the detection result of the reaction acquired by the reaction information acquisition means. A simulator control device, which determines whether or not the operator is paying attention to a scene progressing in the train driving simulator based on whether or not the operator is paying attention. The simulator control device according to any one of claims 1 to 4,
The reaction information acquisition means acquires information about the pupil size of the operator of the train driving simulator as the operator's reaction,
The operator situation determination processing means determines whether the operator agrees with the scene progressing in the train driving simulator based on the pupil size or the amount of change of the operator acquired by the reaction information acquisition means. A simulator control device characterized by judging whether or not the user is paying attention to the details. The simulator control device according to any one of claims 1 to 5,
The reaction information acquiring means acquires, as the reaction of the operator, information related to the line of sight of the operator of the train driving simulator,
The operator situation determination processing means allows the operator to pay attention that matches the scene that is progressing in the train operation simulator based on the movement amount of the operator's line of sight acquired by the reaction information acquisition means. A simulator control device characterized by determining whether or not payment is made. The simulator control device according to any one of claims 1 to 6,
The reaction information acquisition means acquires information about the number of blinks of the operator of the train driving simulator as the reaction of the operator,
The operator situation determination processing means allows the operator to pay attention to a scene progressing in the train driving simulator based on the number of blinks of the operator acquired by the reaction information acquisition means. A simulator control device characterized by determining whether or not the The simulator control device according to any one of claims 1 to 7,
The reaction information acquisition means acquires a plurality of detection results of the reaction,
The operator situation determination processing means uses a plurality of combinations of detection results of the operator's reactions acquired by the reaction information acquisition means to determine whether the operator is in a scene progressing in the train driving simulator. A simulator control device characterized by determining whether or not a consistent attention is paid. A train operation simulation system including a train operation simulator for conducting train operation training and a simulator control device for controlling the train operation simulator,
The train driving simulator is
simulation screen display means for displaying a display screen corresponding to each scene of a scenario progressing in the simulation;
A driving operation simulator in which the trainee who performs the driving training operates by referring to the image displayed by the simulation screen display means;
and reaction detection means for detecting a reaction of the visual function of the operator ,
The simulator control device is
controlling generation of a display image to be displayed on the train operation simulator based on the scenario in which a predetermined scene is provided in which an abnormal event indicating a state requiring an emergency stop is displayed;
simulation control means for terminating the driving training when an operator of the train driving simulator executes an emergency stop operation;
reaction information acquisition means for acquiring a detection result of the reaction of the visual function of the operator detected by the reaction detection means;
An operation of determining whether or not the operator is paying attention that matches the scene that is progressing in the train operation simulator, based on the detection result of the operator's reaction acquired by the reaction information acquiring means. person status determination processing means;
has
If the operator situation determination processing means determines that the operator is not paying attention in accordance with the predetermined scene in the predetermined scene in which the abnormal event is displayed, the simulation control means controls the scenario display the abnormal event based on, and if it is determined that the operator is paying attention that matches the predetermined scene, omit the display of the abnormal event in the predetermined scene;
A train operation simulator control system, wherein the indication of the abnormal event is an indication of an obstacle that may hinder train operation. A train operation simulation method executed by a simulator control device for controlling a train operation simulator for conducting train operation training,
a reaction information acquisition step of acquiring a detection result of a reaction of the visual function of the operator of the train driving simulator;
Based on the detection result of the reaction of the operator acquired by the processing of the reaction information acquiring step, it is determined whether the operator is paying attention that matches the scene that is progressing in the train driving simulator. an operator situation determination processing step for
An operator of the train operation simulator controls generation of a display image to be displayed on the train operation simulator based on a scenario provided with a predetermined scene in which an abnormal event indicating a state requiring an emergency stop is displayed. a simulation control step of terminating the operation training when an emergency stop operation is performed by
In the processing of the simulation control step, in the predetermined scene in which the abnormal event is displayed, it is determined by the processing of the operator situation determination processing step that the operator is not paying attention consistent with the predetermined scene. when the abnormal event is displayed based on the scenario, and if it is determined that the operator is paying attention that matches the predetermined scene, the display of the abnormal event in the predetermined scene is omitted;
A train operation simulation method, wherein the indication of the abnormal event is an indication of an obstacle that may hinder train operation. A computer that controls a train driving simulator for train driving training,
a reaction information acquisition step of acquiring a detection result of a reaction of the visual function of the operator of the train driving simulator;
Based on the detection result of the reaction of the operator acquired by the processing of the reaction information acquiring step, it is determined whether the operator is paying attention that matches the scene that is progressing in the train driving simulator. an operator situation determination processing step for
An operator of the train operation simulator controls generation of a display image to be displayed on the train operation simulator based on a scenario provided with a predetermined scene in which an abnormal event indicating a state requiring an emergency stop is displayed. a simulation control step of terminating the operation training when an emergency stop operation is performed by
In the processing of the simulation control step, in the predetermined scene in which the abnormal event is displayed, it is determined by the processing of the operator situation determination processing step that the operator is not paying attention consistent with the predetermined scene. when the abnormal event is displayed based on the scenario, and if it is determined that the operator is paying attention that matches the predetermined scene, the display of the abnormal event in the predetermined scene is omitted;
The display of the abnormal event is a display of an obstacle that may hinder train operation. A program for executing processing.

Images