JP2022019125A - Operation training evaluation system and operation training evaluation method - Google Patents

Abstract

To provide an operation training evaluation system which can be adapted to various training scenarios.SOLUTION: An operation training evaluation system 1 includes: a simulator 7 which executes a training event in line with a training scenario; a behavior recording unit which records behavior of a trainee who is trained with a simulator; an operation log recording unit which records an operation log of the simulator; a task execution determination unit which determines whether the trainee has executed a task corresponding to the event, on the basis of at least one of the recorded behavior and the operation log; and an evaluation information output unit which outputs evaluation information for evaluating the trainee on the basis of the determination.SELECTED DRAWING: Figure 1

Description

An embodiment of the present invention relates to a driving training evaluation technique.

At plants such as power plants, we are focusing on operation training to improve our ability to respond in the event of a disaster and to pass on technology. The evaluation of training depends on the know-how of the instructor because each role is given to multiple people regarding the operation of the plant and there are complicated exchanges depending on the event. Therefore, there is known a technique of recording audio and video in order to confirm the conversation and operation of the trainee. Further, there is known a technique of clarifying the relationship between the conversation and operation of the trainee by voice recognition and evaluating whether or not an appropriate conversation is possible.

Japanese Unexamined Patent Publication No. 9-60080 Japanese Unexamined Patent Publication No. 2019-36205

In conventional technology, it is necessary to accumulate operation data, plant data, voice data, etc. recorded when good model training was performed for each event that occurs in the plant in order to use it as an index of evaluation. There is. Therefore, accurate evaluation cannot be performed in a scenario that has not yet been well trained as a model. Therefore, an evaluation system that can handle various training scenarios is desired.

An embodiment of the present invention has been made in consideration of such circumstances, and an object of the present invention is to provide a driving training evaluation technique capable of dealing with various training scenarios.

The driving training evaluation system according to the embodiment of the present invention includes a simulator that executes an event for training according to a training scenario, an action recording unit that records the behavior content of a trainee trained by the simulator, and the simulator. The task of determining whether or not the trainee has executed the task corresponding to the event based on the operation log recording unit that records the operation log of the above and the recording of at least one of the action content and the operation log. It includes an execution determination unit and an evaluation information output unit that outputs evaluation information that can evaluate the trainee based on the determination.

An embodiment of the present invention provides a driving training evaluation technique capable of responding to various training scenarios.

Explanatory drawing which shows the structure of the driving training evaluation system. A block diagram showing a driving training evaluation system. The block diagram which shows the main control part of a processing server. The block diagram which shows the storage part of a processing server. A block diagram showing a mobile terminal. A block diagram showing a site monitoring device. A flowchart showing a driving training evaluation method. An explanatory diagram showing a specific example of a combination of a task type and an evaluation index. A flowchart showing a part of the task judgment process. A flowchart showing the first type task determination. A flowchart showing a type 2 task determination. A flowchart showing a type 3 task determination. A flowchart showing a type 4 task determination. A flowchart showing a type 5 task determination. A flowchart showing a type 6 task determination. The flowchart which shows the 7th type task determination. The screen diagram which shows the display example of the evaluation result. A plan view showing the relationship between the training site and the coordinates. A graph showing a display example of the evaluation result of the flow line. Explanatory drawing which shows the locus of a trainee's line of sight. A graph showing the relationship between the number of gazes and time. A flowchart showing a data extraction process in the evaluation process. The flowchart which shows the cluster analysis process in the evaluation process. A dendrogram showing the results of analysis by clustering.

Hereinafter, embodiments of the driving training evaluation system and the driving training evaluation method will be described in detail with reference to the drawings.

Reference numeral 1 in FIG. 1 is the driving training evaluation system of the present embodiment. In this embodiment, a plurality of trainees are trained using a training site (simulator) that imitates the central control room of the plant. The trainee is illustrated as an operator of the main control room. In addition, the trainee is monitored at a place other than the central control room, for example, a worker at a site with a power generation facility in the plant, an office inside the plant site, or an office outside the plant site. May be included. As the plant to be trained, for example, a power plant, a chemical plant, a factory, etc. are assumed. In this embodiment, a training mode of a nuclear power plant is illustrated. The training scenario is preset by the instructor.

Each trainee is given a role to respond to various events that occur at the training site. The driving training evaluation system 1 is used to evaluate each trainee. It is also used to evaluate a team of these trainees. The designated trainee is the conductor who controls the team. In addition, when the trainee includes the above-mentioned worker or observer, each worker or observer is also given a role. Then, the operator, the worker, and the observer are supposed to proceed with the training while communicating information with each other.

At the training site, a main monitor device 2, a main operation panel 3, a sub monitor device 4, a sub operation panel 5, and a conductor operation panel 6 are provided. The main monitor device 2 (FIG. 20) and the sub monitor device 4 are provided with a display area for displaying various information according to the event of the plant. The main operation panel 3, the sub operation panel 5, and the conductor operation panel 6 are provided with a large number of switches and the like operated by the operator (trained person).

The main monitor device 2, the main operation panel 3, the sub monitor device 4, the sub operation panel 5, and the conductor operation panel 6 are connected to the simulator server 7. The simulator server 7 controls the devices according to the training scenario and executes the training event. The simulator server 7 may be constructed by using a conventionally known technique. Further, the simulator server 7 is connected to the processing server 8.

A trainee, such as an operator, a worker, or a watcher, possesses a mobile terminal 9 for recording an action content during training. The content of the utterance of the trainee and the movement of the line of sight are acquired by the mobile terminal 9. The information acquired by the mobile terminal 9 possessed by each trainee is sent to the processing server 8. The record of the action content includes information on the time (time) related to the action.

The mobile terminal 9 of the present embodiment may be, for example, a smartphone provided with a display. Further, the mobile terminal 9 may be a wearable computer (smart glasses) provided with a transmissive head-mounted display worn by the trainee. The wearable computer may have a structure separated from the transmissive head-mounted display. The wearable computer may be equipped with a camera for taking a line-of-sight image showing the situation at the training site and what the trainee is looking at.

The mobile terminal 9 may be composed of a tablet PC or a notebook PC. In addition, it is composed of devices having a shooting function, a calling function, and a communication function, and may be composed of a plurality of devices having each function.

Further, a site monitoring device 10 is provided at the training site. The on-site monitoring device 10 is, for example, a surveillance camera or the like. Based on the video recorded by the on-site monitoring device 10, the change in the position of the trainee and the movement of the trainee can be recorded. Further, the on-site monitoring device 10 is provided with a microphone and can record the voice of the training site. The information acquired by the on-site monitoring device 10 is sent to the processing server 8.

The processing server 8 acquires the content (conversation information) of the trainee's utterance on the mobile terminal 9, and acquires the position of the trainee on the site monitoring device 10. Further, the processing server 8 acquires the simulation operation log from the simulator server 7. Then, the processing server 8 evaluates whether or not the event caused by the training can be appropriately dealt with based on the acquired information. For example, the processing server 8 recognizes the content of each utterance of the trainee and evaluates the training according to the content of each utterance.

Next, the system configuration of the driving training evaluation system 1 will be described with reference to the block diagrams shown in FIGS. 2 to 6.

As shown in FIG. 2, the driving training evaluation system 1 includes a processing server 8, a simulator server 7, a mobile terminal 9, and a site monitoring device 10.

The processing server 8 and the simulator server 7 in the operation training evaluation system 1 of the present embodiment have hardware resources such as a CPU, ROM, RAM, and HDD, and when the CPU executes various programs, information processing by software can be performed. It consists of a computer realized by using hardware resources. Further, the driving training evaluation method of the present embodiment is realized by causing a computer to execute various programs.

The processing server 8 and the simulator server 7 are connected to each other via a network to form a driving training evaluation system 1. It is not always necessary to configure the driving training evaluation system 1 with a plurality of computers. For example, the driving training evaluation system 1 may be configured by using one computer.

The processing server 8 includes an input unit 11, an output unit 12, a storage unit 13, a communication unit 14, and a main control unit 15.

The input unit 11 inputs predetermined information according to the operation of the user who uses the processing server 8. The input unit 11 includes an input device such as a mouse or a keyboard. That is, predetermined information is input to the input unit 11 according to the operation of these input devices.

The output unit 12 outputs predetermined information. The processing server 8 (driving training evaluation system 1) of the present embodiment includes a device that displays an image such as a display that outputs analysis results. That is, the output unit 12 controls the image displayed on the display. The display may be separate from the computer body or may be integrated. Further, the output unit 12 may control the image displayed on the display of another computer connected via the network.

In this embodiment, a display is exemplified as a device for displaying an image, but other embodiments may be used. For example, information may be displayed using a head-mounted display or a projector. Further, a printer that prints information on a paper medium may be used instead of the display. That is, a head-mounted display, a projector, or a printer may be included as an object controlled by the output unit 12.

The communication unit 14 communicates between the simulator server 7, the mobile terminal 9, and the site monitoring device 10 via a communication line such as a LAN (Local Area Network). The communication unit 14 may be mounted on a predetermined network device, for example, a wireless LAN access point or an antenna. The communication unit 14 may communicate with other devices or computers via a WAN (Wide Area Network), an Internet line, or a mobile communication network. In addition, a LAN cable or a USB cable may be used to communicate with other devices or computers.

As shown in FIG. 3, the main control unit 15 comprehensively controls the processing server 8. The main control unit 15 includes a training scenario setting unit 16, a task setting unit 17, a task execution determination unit 18, a task evaluation unit 19, a person position acquisition unit 20, a person position identification unit 21, a speaker management unit 22, and a voice recognition unit. 23, speech identification unit 24, text conversion unit 25, text mining unit 26, video analysis unit 27, line-of-sight acquisition unit 28, line-of-sight identification unit 29, line-of-sight layering unit 30, data extraction unit 31, line-of-sight evaluation unit 32, and object discrimination. It includes a unit 33, an evaluation information output unit 34, a training event display unit 35, an operation log display unit 36, a text display unit 37, a voice reproduction unit 38, and a dictionary correction unit 39. These are realized by executing the program stored in the memory or the HDD by the CPU.

The training scenario setting unit 16 receives the training scenario input by the instructor. Then, the settings are made so that the evaluation can be performed according to the training scenario.

The task setting unit 17 sets the task corresponding to the training scenario. A task indicates an action or operation that the trainee must perform when an event occurs. The task may be set according to the event and may be set according to each trainee. Further, a plurality of tasks may be set in advance in the training scenario, and the task setting unit may select a task required for a predetermined trainee for a predetermined event from a plurality of preset tasks.

Further, the task setting unit 17 sets a predetermined type of task to correspond to a predetermined evaluation index. The process of setting the evaluation index may be a process of selecting a predetermined one from a plurality of evaluation indexes. For example, a plurality of evaluation indexes (FIG. 8) are set, and an evaluation index corresponding to a predetermined type of task may be selected from the plurality of evaluation indexes.

The action content of the present embodiment includes the utterance of the trainee, the line of sight of the trainee, the movement of the trainee, and the position of the trainee. The movement includes the movement of the upper limbs of the trainee, and includes the movement of reaching for a predetermined operation panel.

Further, in the task setting unit 17, a plurality of types of tasks can be set corresponding to one event. In this way, it is possible to set tasks corresponding to various training scenarios. The task may be set manually by the user or may be set automatically based on a predetermined control.

Further, the task setting unit 17 sets an evaluation index for each type of task (FIG. 8). In this way, it is possible to evaluate whether or not the trainee is compatible with various training scenarios.

The task execution determination unit 18 determines whether or not the trainee has executed the task corresponding to the event based on the record of at least one of the action content of the trainee and the operation log. The task execution determination unit 18 individually determines whether or not one trainee has executed each task corresponding to one event.

Further, the task execution determination unit 18 determines whether or not the trainee has executed the task for each type of task. In this way, the trainee can be evaluated in detail.

Further, the task execution determination unit 18 determines whether or not at least one type of task has been executed by combining the utterance of the trainee and the operation log. By doing so, since both the utterance and the operation log are used for the determination, the accuracy of the determination as to whether or not the task is properly executed can be improved.

The task evaluation unit 19 evaluates a predetermined index (FIG. 8) based on the task executed by the trainee.

The person position acquisition unit 20 acquires the position of the trainee at the training site based on the image acquired by the site monitoring device 10. The movement of the trainee can be grasped by the change in the position of the trainee. That is, the person position acquisition unit 20 tracks the trainee. The action content of the present embodiment includes the position of the trainee as the trainee.

The person position acquisition unit 20 may acquire the position of the trainee based on the information acquired by the mobile terminal 9. For example, the mobile terminal 9 may include a motion sensor for acquiring the movement of the trainee, and the position of the trainee may be grasped by this motion sensor.

The person position identification unit 21 determines whether or not the trainee is in a position corresponding to the task based on the information acquired by the person position acquisition unit 20 when an event for training occurs under the control of the simulator server 7. To identify. In this way, it is possible to evaluate whether or not the trainee is in an appropriate position to perform the task.

The speaker management unit 22 manages the trainee who has spoken in an identifiable manner. In this way, it is possible to determine whether or not the task is executed based on the words spoken by the trainee based on the training scenario.

For example, when voice is recorded by the mobile terminal 9, a process of identifying the trainee who possesses the mobile terminal 9 and storing the content of the utterance included in the voice in association with the specified trainee is performed. conduct. Further, when the voice recorded by the on-site monitoring device 10 includes the utterances of a plurality of trainees, the trainees who have made the utterances are identified by the voice recognition technique.

Further, the speaker management unit 22 may have a function of managing the application set in the processing server. For example, a predetermined application is installed on the mobile terminal 9 possessed by the trainee. Based on this application, each trainee may be associated with the information acquired by the mobile terminal 9 to identify the trainee who has spoken.

The voice recognition unit 23 recognizes the recorded voice. For example, using a known voice recognition technique, it is possible to recognize the content of an utterance included in the voice recorded by the mobile terminal 9 or the site monitoring device 10.

The utterance identification unit 24 identifies the utterance of a specific trainee from the voice in which the utterances of the plurality of trainees are recorded by using a technique such as machine learning. By doing so, it is possible to separate the utterance of a specific trainee from the recorded voice and use it for evaluation.

The text conversion unit 25 converts voice into text data based on a dictionary registered in advance. The dictionary contains words related to general conversations, past training scenarios, and training scenarios. In this way, it is possible to distinguish between general conversations and conversations used in training, and it is possible to appropriately convert specialized words spoken in training into text data.

In addition, the process of converting voice into text data is performed in real time. For example, the voice is acquired in real time by using the communication function of the mobile terminal 9, and the voice is converted. If a predetermined display in real time or output of the evaluation result is not required, data including voice may be stored in the mobile terminal 9 and the voice may be converted into text data after training.

The text mining unit 26 performs text mining based on the text data acquired in the training, and extracts the tasks required of the trainee in response to the training event. The extracted tasks can be used for evaluation of training from the next time onward. In this way, the tasks required of the trainee can be automatically extracted using the text mining technique. As the text data, the one converted by the text conversion unit 25 is used.

The video analysis unit 27 can acquire a flow line, which is a locus of movement of each trainee, based on the video recorded by the on-site monitoring device 10. For example, plane coordinates are given to the training place, and the position where the trainee is located is specified based on the plane coordinates. Then, the person is tracked by image analysis of the video, and the position of the trainee is recorded in chronological order. Based on this record, it is possible to evaluate whether or not the trainee is staying at an appropriate position at an appropriate time (time).

Further, the video analysis unit 27 recognizes the movement of the trainee based on the video recorded by the on-site monitoring device 10, analyzes the relationship between the utterance and the training event, and responds to the event. Extract the tasks required of the trainee. The extracted tasks can be used for evaluation of training from the next time onward. In this way, the tasks required of the trainee can be automatically extracted using the video recognition technology.

The line-of-sight acquisition unit 28 acquires the movement of the trainee's line of sight based on the information sent from the mobile terminal 9 worn by the trainee.

The line-of-sight identification unit 29 determines whether or not the trainee is looking at a part corresponding to the task when a training event occurs, based on the movement of the trainee's line of sight acquired by the line-of-sight acquisition unit 28. Identify. In this way, it is possible to evaluate whether or not the trainee is looking at an appropriate place to execute the task.

The line-of-sight layer portion 30 processes the position of the line-of-sight (viewpoint) in the line-of-sight image acquired by the trainee on the mobile terminal 9, and superimposes the locus 56 of the line-of-sight on the line-of-sight image. Processing is performed (FIG. 20). By this layering process, it is possible to obtain line-of-sight layered data including information that can identify the line-of-sight trajectory 56.

The data extraction unit 31 extracts the first feature amount data indicating the feature amount used by the line-of-sight evaluation unit 32 based on the line-of-sight layer data obtained by the line-of-sight layer unit 30 and the process data stored in the storage unit 13. do. The process data includes operation data indicating an operation procedure and work data indicating a work procedure.

The line-of-sight evaluation unit 32 evaluates the line of sight indicating whether or not the trainee is visually recognizing an appropriate position. For example, the display area of the main monitor device 2 (FIG. 20) in which the trainee grasps the state of the plant is divided into predetermined areas, and the display area to be confirmed by the trainee according to the progress of the training scenario and the event. To set. The line-of-sight evaluation unit 32 counts the number of gazes in each display area in chronological order. Based on this count, it is possible to evaluate whether or not the trainee is visually recognizing an appropriate display area at an appropriate time (time).

Further, the line-of-sight evaluation unit 32 compares the first feature amount data extracted by the data extraction unit 31 with the second feature amount data registered in advance, which is model data as a model for a skilled person, so as to be first. The skill level of feature data can be evaluated.

The object determination unit 33 uses the line-of-sight image acquired by the mobile terminal 9 as seen by the trainee and the teacher data of the object such as a predetermined device stored in the storage unit 13, and is a line-of-sight image. Determine the type or shape of the object contained in.

The evaluation information output unit 34 outputs evaluation information that can evaluate the trainee. For example, the evaluation information output unit 34 outputs the evaluation result of a predetermined index obtained based on the determination of the task execution determination unit 18 to a display or the like (FIG. 17 or 19). Further, the evaluation information output unit 34 outputs the evaluation result obtained by the line-of-sight evaluation unit 32 to a display or the like (FIG. 24).

Further, the evaluation information output unit 34 displays a time-series chart that displays events, operation logs, and text data in chronological order on the display (FIG. 17). By doing so, it becomes possible to grasp the contents of events, operations, and utterances in chronological order.

Further, the evaluation information output unit 34 displays an individual evaluation chart that graphs the individual evaluation results of the trainees and a team evaluation chart that graphs the team evaluation results of a team composed of a plurality of trainees. Display (Fig. 17). In this way, the individual evaluation of the trainee or the evaluation of the team can be grasped at a glance.

Further, the evaluation information output unit 34 can present an evaluation result indicating the skill level evaluated by the line-of-sight evaluation unit 32 to a third-party instructor or an operator as a trainee. The evaluation result may be displayed on a general display included in the processing server 8, or may be presented to the trainee using smart glasses or the like. The evaluation result may be fed back to the trainee not only after the training but also during the training.

The training event display unit 35 displays information about the training event. For example, when displaying the evaluation result, the history of the training event 51 is displayed on the display in chronological order (FIG. 17).

The operation log display unit 36 displays information related to the operation log. For example, when displaying the evaluation result, the history of the operation log 52 is displayed on the display in chronological order (FIG. 17).

The text display unit 37 displays text data corresponding to the voice indicating the content of the utterance of the trainee on the display. For example, when displaying the evaluation result, the text data 53 of the utterance contents of each trainee is displayed in chronological order (FIG. 17).

The voice reproduction unit 38 reproduces the voice indicating the content of the utterance of the trainee recorded during the training. By doing so, it is possible to confirm the content of the actual utterance of the trainee at the timing of looking back after the training (reflection meeting, etc.).

The dictionary correction unit 39 corrects a part of the contents registered in the dictionary corresponding to the erroneous conversion when the text conversion unit 25 causes an erroneous conversion. By doing so, the registered contents of the dictionary that causes the erroneous conversion can be corrected, and the corrected dictionary can be used in the next and subsequent trainings.

As shown in FIG. 4, the storage unit 13 stores various information necessary for performing an evaluation regarding training. The storage unit 13 includes a dictionary registration unit 40, an action recording unit 41, an operation log recording unit 42, a process data storage unit 43, and a model data storage unit 44.

Further, the storage unit 13 may include a database for accumulating information. A database is a collection of information stored in a memory or HDD and organized so that it can be searched or stored.

The dictionary registration unit 40 registers a dictionary for converting the voice including the content of the utterance of the trainee recorded at the training site into text data.

The behavior recording unit 41 records the behavior content of the trainee who is trained at the training site (simulator). The behavior of the trainee is recorded in real time. Further, predetermined information indicating the record of the behavior content of the trainee may be stored in the mobile terminal 9, and the record of the behavior content of the trainee may be acquired from the mobile terminal 9 after the training.

The operation log recording unit 42 records operation logs related to the main operation panel 3, the sub operation panel 5, and the conductor operation panel 6 provided at the training site. The operation log is transmitted from the simulator server 7. The process of acquiring the operation log is performed in real time. Further, the data including the operation log may be accumulated in the simulator server 7, and the operation log may be acquired from the simulator server 7 after the training.

The process data storage unit 43 stores process data related to various processes showing the operation procedure or work procedure of the trainee.

The model data storage unit 44 stores model model data (including second feature amount data) acquired based on the behavior of an expert. In the present embodiment, an image for instruction created based on the model data can be presented to a third-party instructor, an operator as a trainee, or the like.

As shown in FIG. 5, the mobile terminal 9 includes an individual voice recording unit 45, a line-of-sight measurement unit 46, a line-of-sight measurement camera 47, and a result layered unit 48.

The individual voice recording unit 45 records voices related to the utterances of the trainee. For example, the microphone function of the mobile terminal 9 can be used as the individual voice recording unit 45. Further, a pin microphone or a directional microphone may be connected to the mobile terminal 9 to record audio. By doing so, it is possible to prevent recording of ambient voice or noise and improve the recognition rate of voice.

The line-of-sight measurement unit 46 measures the line of sight of the trainee. For example, the line-of-sight measurement unit 46 detects a human pupil position and acquires line-of-sight data indicating the position of the human line-of-sight based on the pupil position. For example, the line-of-sight measurement unit 46 acquires the position of the pupil based on near-infrared rays, electrooculogram, and the like. Generally, a glasses-type device worn around the human eye is used. Further, the position of the pupil may be acquired by a device other than the glasses-type device.

The line-of-sight measurement camera 47 acquires an image (line-of-sight image) of the visual field seen by the trainee. The line-of-sight measurement camera 47 may be mounted on a glasses-type device worn around the human eye, or may be mounted on other devices. Further, the line-of-sight image may be either a video image (moving image) or a still image.

The result layered unit 48 displays an image showing the evaluation result of the line-of-sight evaluation unit 32 layered on an object such as a predetermined device determined by the object discriminating unit 33. For example, an instruction image 57 (FIG. 20) showing a portion to be seen by the trainee is displayed in a layered manner on the main monitor device 2 as an object.

As shown in FIG. 6, the site monitoring device 10 includes a site video recording unit 49 and a site audio recording unit 50.

The on-site video recording unit 49 records the video of the entire training site. The video of the entire training site includes a video of the trainee's movements.

The on-site voice recording unit 50 records the voice of the entire training site. The video of the entire training site includes audio about the trainee's utterances.

The driving training evaluation system 1 may include a computer equipped with artificial intelligence (AI) for performing machine learning. For example, the utterance identification unit 24, the text mining unit 26, the video analysis unit 27, the object discrimination unit 33, and the like may perform at least a part of the processing by using the machine learning technique. In addition, other configurations may perform processing based on machine learning. Further, the driving training evaluation system 1 may include a deep learning unit that extracts a specific pattern from a plurality of patterns based on deep learning.

For the analysis using the computer of this embodiment, an analysis technique based on learning of artificial intelligence can be used. For example, a learning model generated by machine learning by a neural network, another learning model generated by other machine learning, a deep learning algorithm, a mathematical algorithm such as regression analysis can be used. Further, the form of machine learning includes forms such as clustering and deep learning.

The system of this embodiment includes a computer having artificial intelligence that performs machine learning. For example, the system may be configured by one computer including a neural network, or the system may be configured by a plurality of computers including a neural network.

Here, the neural network is a mathematical model that expresses the characteristics of brain function by computer simulation. For example, we show a model in which artificial neurons (nodes) that form a network by synaptic connection change the synaptic connection strength by learning and have problem-solving ability. In addition, neural networks acquire problem-solving abilities through deep learning.

For example, a neural network is provided with an intermediate layer having six layers. Each layer of this intermediate layer is composed of 300 units. In addition, by having a multi-layer neural network learn in advance using training data, it is possible to automatically extract features in a pattern of changes in the state of a circuit or system. In the multi-layer neural network, an arbitrary number of intermediate layers, an arbitrary number of units, an arbitrary learning rate, an arbitrary number of learning times, and an arbitrary activation function can be set on the user interface.

A reward function may be set for various information items to be learned, and deep reinforcement learning for extracting the information item having the highest value based on the reward function may be used for the neural network.

For example, a CNN (Convolution Neural Network) with a proven track record in image recognition is used. In this CNN, the intermediate layer is composed of a convolution layer and a pooling layer. The convolution layer acquires a feature map by filtering nearby nodes in the previous layer. The pooling layer further reduces the feature map output from the convolutional layer into a new feature map. At this time, by obtaining the maximum value of the pixels included in the region of interest in the feature map, it is possible to absorb a slight deviation in the position of the feature amount.

The convolution layer extracts the local features of the image, and the pooling layer performs a process of summarizing the local features. In these processes, the image is reduced while maintaining the characteristics of the input image. That is, in CNN, the amount of information possessed by an image can be significantly compressed (abstracted). Then, using the abstracted image stored in the neural network, the input image can be recognized and the image can be classified.

There are various methods for deep learning such as autoencoder, RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), and GAN (Generative Adversarial Network). These methods may be applied to the deep learning of the present embodiment.

Next, the driving training evaluation method executed by using the driving training evaluation system 1 will be described with reference to the flowchart of FIG. 7. In addition, the above-mentioned drawings are referred to as appropriate. The following steps are at least a part of the steps included in the driving training evaluation method, and other steps may be included in the driving training evaluation method.

First, in step S11, the instructor sets (inputs) the training scenario in the simulator server 7 and the processing server 8.

In the next step S12, the task setting unit 17 of the processing server 8 sets the task corresponding to the training scenario. Here, the task setting unit 17 may set the task based on the input operation of the instructor, or may automatically set the task based on the information obtained in the training before the previous time. good.

In the next step S13, the simulator server 7 executes a training event according to the set training scenario.

In the next step S14, the mobile terminal 9 or the site monitoring device 10 records information including audio or video showing the content of the behavior of the trainee. Here, the processing server 8 receives information indicating the behavior content of the trainee from the mobile terminal 9 or the site monitoring device 10. Then, the processing server 8 records the information obtained from the mobile terminal 9 or the site monitoring device 10 in the action recording unit 41 of the storage unit 13.

In the next step S15, the simulator server 7 transmits the operation log recorded when the trainee performs an operation in response to the event to the processing server 8. Here, the processing server 8 records the operation log obtained from the simulator server 7 in the operation log recording unit 42 of the storage unit 13.

In the next step S16, the processing server 8 executes the task determination process. Here, the task execution determination unit 18 determines whether or not the trainee has executed the task corresponding to the event based on the record of at least one of the action content of the trainee and the operation log (from FIG. 9). FIG. 16).

In the next step S17, the processing server 8 executes the evaluation process. Here, the task evaluation unit 19 evaluates the index based on the task executed by the trainee. In addition, the line-of-sight evaluation unit 32 evaluates the line-of-sight indicating whether or not the trainee is visually recognizing an appropriate position.

In the next step S18, the processing server 8 executes the evaluation information output processing. Here, the evaluation information output unit 34 outputs evaluation information that can evaluate the trainee (FIGS. 17, 19, and 24).

In the next step S19, the processing server 8 executes post-processing. For example, the instructor and the trainee output the information necessary for reviewing after the training (reflection meeting, etc.). Then, the driving training evaluation method is completed.

Next, the determination of the task and the evaluation of the index in the present embodiment will be described with reference to FIGS. 8 to 19. In addition, the above-mentioned drawings are referred to as appropriate.

The training scenario set by the instructor includes a plurality of events, and the tasks required of each trainee are set in response to the events. In addition, evaluation indexes are set according to the task.

FIG. 8 shows an example of a combination of task types and evaluation indexes. Evaluation indicators are set according to the type of each task.

For example, "A: Do you know the cause of occurrence?" Is set as the first type task. As the second type task, "B: Have you taken measures to spread to others?" Is set. As the third type task, "C: Are you selecting an appropriate procedure manual?" Is set. As the fourth type task, "D: Are you performing actual operation and confirmation according to the selected procedure?" Is set. "E: Is the plant status correctly recognized?" Is set as the fifth type task. As the sixth type task, "F: Is it possible to properly contact the related parties?" Is set. As the 7th type task, "G: Is the necessary action taken at an appropriate timing?" Is set.

In addition, "cognitive evaluation", "processing and operation evaluation", "selection evaluation of procedure manual", "compliance evaluation of maintenance regulations", "communication evaluation" and the like are set as evaluation indexes. Each evaluation is performed according to whether or not the task is executed.

For example, the first type task A is used for determining "cognitive evaluation". The second kind task B is used for the determination of "processing and operation evaluation". The third type task C is used for the determination of "selective evaluation of the procedure manual". Type 4 task D is used to determine "compliance evaluation of maintenance regulations". Type 5 task E is used to determine "cognitive evaluation". Type 6 task F is used to judge "compliance evaluation of maintenance regulations" and "communication evaluation". Type 7 task G is used to determine "communication evaluation".

As shown in FIG. 9, when a predetermined event occurs according to the training scenario, the type of task corresponding to this event is set. Then, it is determined whether or not the task is executed for each type of task. The determination of whether or not each task is executed may be recorded for each trainee or for each team.

In the present embodiment, the determination of the first type task A (step S100), the determination of the second type task B (step S200), the determination of the third type task C (step S300), and the fourth type task D. The determination (step S400), the determination of the fifth type task E (step S500), the determination of the sixth type task F (step S600), and the determination of the seventh type task G (step S700) are executed in parallel. To.

As shown in FIG. 10, in the determination of the first type task A (step S100), it is determined whether or not the trainee is speaking the signal name of the factor (step S101). Then, YES or NO of this determination is recorded (step S102). Further, in parallel with this determination, it is determined whether or not the trainee is uttering the first hit word (step S103). Then, YES or NO of this determination is recorded (step S104). Based on these determinations, the "cognitive evaluation" score for the trainee and the team is recorded (step S105).

As shown in FIG. 11, in the determination of the second type task B (step S200), it is determined whether or not the trainee is able to perform an operation corresponding to the training scenario (step S201). This determination includes a determination as to whether or not an operation log in which an appropriate operation has been performed is recorded, or whether or not an operation has been performed at an appropriate time (timing). Then, YES or NO of this determination is recorded (step S202). Based on this determination, the "treatment and operation evaluation" score for the trainee and the team is recorded (step S203).

As shown in FIG. 12, in the determination of the third type task C (step S300), first, it is determined whether or not the event that has occurred is an event related to Scrum (step S301). Here, if the event is related to Scrum (YES in step S301), the process proceeds to step S302 and step S304. On the other hand, if it is not an event related to Scrum (NO in step S301), the process proceeds to step S306 and step S308.

In step S302, it is determined whether or not the trainee is speaking the name of the cause. Then, YES or NO of this determination is recorded (step S303). Further, in parallel with this determination, it is determined in step S304 whether or not the basic correspondence of Scrum has been performed. For example, it is determined whether or not the operation trend is appropriate. Then, YES or NO of this determination is recorded (step S305). Based on these determinations, the score of "selective evaluation of the procedure manual" regarding the trainee and the team is recorded (step S310).

In step S306, it is determined whether or not the trainee is speaking the name of the corresponding device. Then, YES or NO of this determination is recorded (step S307). Further, in parallel with this determination, it is determined in step S308 whether or not the trainee who is the conductor is speaking to make the response such as "implement XX" known. Then, YES or NO of this determination is recorded (step S309). Based on these determinations, the score of "selective evaluation of the procedure manual" regarding the trainee and the team is recorded (step S310).

As shown in FIG. 13, in the determination of the fourth type task D (step S400), it is determined whether or not the trainee is uttering a word in the procedure manual (step S401). Then, YES or NO of this determination is recorded (step S402). Further, in parallel with this determination, it is determined whether or not the trainee is performing the operation in the procedure manual (step S403). For example, it is determined whether or not the operation trend is appropriate. Then, YES or NO of this determination is recorded (step S404). Based on these determinations, the "processing and operation evaluation" score for the trainee and the team is recorded (step S405).

As shown in FIG. 14, in the determination of the fifth type task E (step S500), it is determined whether or not the trainee is speaking the first parameter item in the procedure manual (step S501). Then, YES or NO of this determination is recorded (step S502). Further, in parallel with this determination, it is determined whether or not the trainee is speaking the second parameter item in the procedure manual (step S503). Then, YES or NO of this determination is recorded (step S504). Based on these determinations, the "cognitive evaluation" score for the trainee and the team is recorded (step S505).

As shown in FIG. 15, in the determination of the sixth type task F (step S600), first, it is determined whether or not the event violates the original disaster law (step S601). Here, if the event conflicts with the original disaster law (YES in step S601), the process proceeds to step 602. On the other hand, if the event does not conflict with the original disaster law, for example, if it is simply a device failure (NO in step S601), the process proceeds to step S604.

In step S602, it is determined whether or not the emergency response room has been contacted. In addition, the contact to the emergency response room will be replaced by the contact to the instructor. Then, YES or NO of this determination is recorded (step S603). Based on this determination, the scores of the "security regulation compliance evaluation" and the "communication evaluation" regarding the trainee and the team are recorded (steps S606 and S607).

In step S604, it is determined whether or not the repair has been contacted. In addition, the contact for repair will be replaced by the contact with the instructor. Then, YES or NO of this determination is recorded (step S605). Based on this determination, the scores of the "security regulation compliance evaluation" and the "communication evaluation" regarding the trainee and the team are recorded (steps S606 and S607).

As shown in FIG. 16, in the determination of the 7th type task G (step S700), it is determined whether or not an appropriate operation is performed within a designated time after the event occurs (step S701). For example, it is determined whether or not the operation trend is appropriate. Then, YES or NO of this determination is recorded (step S702). Based on this determination, the score of "communication evaluation" regarding the trainee and the team is recorded (step S703).

In this way, the entire training scenario can be evaluated by combining the judgment flows of each type of task. Therefore, it is possible to evaluate training without preparing data when good training is performed in advance, and it is possible to evaluate various training scenarios.

From the relationship between the training scenario, the event, and the task, the text mining technique may be applied, and the text mining unit 26 (FIG. 3) may automatically set the task type and the evaluation index corresponding to the training scenario. By doing so, the evaluation work can be made more efficient. Further, the behavior of the trainee may be analyzed based on past training scenarios, surveillance video images, etc., and the task setting unit 17 may automatically extract the tasks required of the trainee.

As shown in FIG. 17, the evaluation result is displayed on the display of the processing server 8. For example, as a time-series chart, an event 51 that has occurred, an operation log 52, and text data 53 indicating the contents spoken by each trainee are displayed in time series. In addition, a radar chart 54 showing the individual evaluation results of the trainee and a radar chart 55 showing the team evaluation results are also displayed. This evaluation result can be confirmed by the instructor and is provided to each trainee at the timing of reflection after training (reflection meeting, etc.).

At the timing of looking back after the training, the voice reproduction unit 38 (FIG. 3) can reproduce the voice recorded during the training. In addition, if an erroneous conversion of text data is confirmed during this voice reproduction, the feedback function can be used to correct the dictionary. For example, the dictionary correction unit 39 (FIG. 3) corrects a part of the contents registered in the dictionary corresponding to an erroneous conversion. A general learning function engine may be applied to daily conversation.

In the field of conventional training, the instructor had to prepare a report in order to make an evaluation using images and graphs. In many cases, the review after the training was carried out two weeks after the training. In the present embodiment, it is possible to look back immediately after the training and provide the evaluation result while the trainee's memory is new.

In addition, during training, audio and video may be taken with a surveillance camera or the like. In the present embodiment, by utilizing the video, the flow line and the movement of the trainee can be acquired together with the utterance. Image analysis technology is applied for flow lines and movements. For example, the video analysis unit 27 detects a person based on the video and tracks the person. Then, it is determined whether or not the position is appropriate according to the event.

Further, for utterances, the utterances of the trainee can be separated by using the voice separation function of the utterance identification unit 24 (FIG. 3) and the like. If the voice separation function is used, it is not necessary to have a mobile terminal 9 for speaker management and voice recording. Further, the voice separation function may classify features such as the frequency spectrum of the speaker's voice by deep learning and make it learn which speaker belongs to the voice.

In the present embodiment, it is possible to determine whether or not the task is being executed based on the flow line which is the locus of movement of each trainee. For example, as shown in FIG. 18, plane coordinates (X coordinate, Y coordinate) are assigned to the training location.

The progress of the event and the time (time) correspond to each other, and it is judged by the graph distribution whether or not each trainee is staying at an appropriate position as the time changes. For example, as shown in FIG. 19, it is possible to graph the plane coordinates of the training place and the staying time when the trainee is staying at a predetermined coordinate position. Based on this graph, it is possible to judge the execution of each trainee's task and evaluate the index.

The determination criteria (coordinates and time) may be set based on the training scenario or may be set by using machine learning based on the past training scenario and the video. In addition, a person's joint detection technique may be used to detect the trainee's reliable pointing confirmation, naming, raising hand, and the like.

Next, the line-of-sight evaluation process in this embodiment will be described with reference to FIGS. 20 to 24. In addition, the above-mentioned drawings are referred to as appropriate.

In the line-of-sight evaluation process, evaluation is performed based on the line of sight of the trainee. For example, by acquiring the line-of-sight data of the trainee, it is possible to grasp the contents confirmed and judged by the trainee, the state such as whether there was any doubt in the judgment, and the operation at hand that does not remain in the operation log. Based on the circumstances, it is possible to judge the execution of each trainee's task and evaluate the index.

The trainee's mobile terminal 9 includes a glasses-type device. A line-of-sight measurement unit 46 and a line-of-sight measurement camera 47 are provided in the frame portion of the glasses. Therefore, even if the trainee moves from one place to another, the line of sight can always be measured.

As shown in FIG. 20, as a specific example, the evaluation of the line of sight of the trainee who is visually recognizing the main monitor device 2 will be described. The main monitor device 2 is divided into a plurality of types of display areas such as an alarm, a water level, a turbine rotation speed, a valve open / closed state, and a predetermined graph.

The display area that the trainee should check is determined according to the event. The time (time) is associated with the progress of the event, and the number of gazes in each display area is detected. Then, as shown in FIG. 21, it is possible to create a graph in which the time (progress of the event) and the number of gazes in the predetermined display area correspond to each other. In this way, it is possible to evaluate whether or not the confirmation of the trainee is properly carried out.

Further, each display area is divided into more detailed display areas according to the type of alarm, individual valves, and the like. Similarly, the number of gazes may be evaluated for these individual display areas, or the number of gazes may be evaluated by setting plane coordinates on the display surface of the main monitor device 2.

As shown in FIGS. 3, 5, and 20, in the line-of-sight evaluation process of the present embodiment, the line-of-sight acquisition unit 28 indicates the position of the line-of-sight of the trainee measured by the line-of-sight measurement unit 46 of the mobile terminal 9. Get the data. The line-of-sight image taken by the line-of-sight measurement camera 47 is also acquired.

Here, the object discrimination unit 33 discriminates the type and shape of the device or the like in the line of sight of the trainee shown in the line-of-sight image by using the teacher data of the object acquired in advance. Then, the result layering unit 48 displays an image based on the evaluation result of the line-of-sight evaluation unit 32 layered on the object determined by the object discrimination unit 33.

For example, as shown in FIG. 20, when the trainee is looking at the main monitor device 2 through smart glasses, a line-of-sight image showing the main monitor device 2 is acquired. Here, the object discrimination unit 33 performs a process of discriminating the main monitor device 2 as an object. For example, it is determined that what is reflected in the line-of-sight image is the main monitor device 2 based on the preset teacher data regarding the main monitor device 2.

Then, the line-of-sight layered portion 30 executes the layering process. The position of the line-of-sight (viewpoint) of the line-of-sight image acquired by the mobile terminal 9 is processed, and the line-of-sight locus 56 is layered on the line-of-sight image. Information indicating the locus 56 of the line of sight of each trainee may be presented to the instructor. Further, when the trainee does not visually recognize the appropriate position during the training, the instruction image 57 (FIG. 20) indicating the part to be seen by the trainee is displayed in a state of being layered on the object. ..

In the multi-layer display, not only the display but also smart glasses are used, and the parts to be seen are displayed with markers according to the skill level. When displaying on the smart glasses, the display is performed according to the shape of the object in cooperation with the discrimination result of the object discrimination unit 33. In this way, it is possible to support the training or work of the trainee according to the skill level.

The data extraction unit 31 can obtain line-of-sight layer data including information that can identify the line-of-sight trajectory 56 of the trainee. Then, the data extraction unit 31 extracts the first feature amount data indicating the feature amount based on the line-of-sight layer data and the process data.

The line-of-sight evaluation unit 32 evaluates the skill level of the first feature amount data by comparing the first feature amount data with the second feature amount data which is the model data. Then, the evaluation information output unit 34 outputs evaluation information capable of evaluating the skill level of the trainee.

The feature amount data of the present embodiment is data corresponding to at least one of the number of line-of-sight stays, the line-of-sight gaze time, the time until the line-of-sight gaze, and the line-of-sight gaze time per one line-of-sight stay as feature amount items. Is.

The number of times the trainee stays in the line of sight is an index for measuring how many times the trainee gazes at the determined range. The line-of-sight gaze time is an index for measuring how long (time) the trainee gazes at the determined range. The time to gaze at the line of sight is an index for measuring the time required for the line of sight to gaze at a predetermined range. The gaze gaze time per gaze stay is a value obtained by dividing the gaze gaze time by the number of gaze stays.

Next, the data extraction process will be described with reference to the flowchart of FIG. In addition, the above-mentioned drawings are referred to as appropriate. This data extraction process is a process for extracting the line-of-sight layer data of the trainee and the process data corresponding to the line-of-sight of the trainee. In addition, it is assumed that a part of the processing for performing the evaluation by the line-of-sight evaluation unit 32 is included.

First, in step S21, the data extraction unit 31 associates the operation log with the operation time and the line-of-sight layer data before (preliminary) the training. Thereby, the line-of-sight layer data can be tagged as to what kind of operation is performed.

In the next step S22, the data extraction unit 31 sets the division of the line-of-sight area before (preliminary) the training. Here, the object to be evaluated included in the line-of-sight layer data is divided into a plurality of sections. For example, in the case of the main monitor device 2 (FIG. 20), each display area is divided as a line-of-sight area.

In the next step S23, the data extraction unit 31 counts the number of line-of-sight stays, the line-of-sight gaze time, the time until the line-of-sight gaze, and the line-of-sight gaze time per one line-of-sight stay during the training. These data are counted for each trainee.

In the next step S24, the data extraction unit 31 extracts line-of-sight data (line-of-sight locus 56) corresponding to each line-of-sight area during training.

In the next step S25, the line-of-sight evaluation unit 32 evaluates the line-of-sight indicating whether or not the trainee is visually recognizing an appropriate position during training.

In the next step S26, the evaluation information output unit 34 provides an instruction image 57 (FIG. 20) indicating a portion to be seen by the trainee when the trainee does not visually recognize an appropriate position during training. Display in a state where it is layered on the object. For example, when the trainee is looking only at the alarm display area when the area to be seen by the trainee is the graph display area when an alarm is generated, the instruction image 57 is displayed in a state of being layered on the graph display area. do. Then, the trainee comes to pay attention to the graph display area by the instruction image 57. Then, the data extraction process is terminated. During the training, steps S23 to S26 may be repeated.

Next, the cluster analysis process will be described with reference to the flowchart of FIG. In addition, the above-mentioned drawings are referred to as appropriate. This cluster analysis process is a process for evaluating the trainee based on the line-of-sight data extracted by the data extraction unit 31.

First, in step S31, the line-of-sight evaluation unit 32 creates input data for clustering analysis based on the line-of-sight data extracted by the data extraction unit 31. The input data is created for each trainee. Input data may be created for each team. That is, the line-of-sight evaluation unit 32 includes a clustering input data creation unit.

In the next step S32, the line-of-sight evaluation unit 32 executes a clustering analysis based on the input data. That is, the line-of-sight evaluation unit 32 includes a clustering analysis unit. The cluster distance between the input data is calculated by rastering analysis.

In the next step S33, the line-of-sight evaluation unit 32 performs cluster classification based on the cluster distance between the calculated input data. If the cluster classification is not always suitable, such as when the number of data is small, this cluster classification may be omitted.

In the next step S34, the line-of-sight evaluation unit 32 extracts the cluster distance between the calculated input data.

In the next step S35, the evaluation information output unit 34 outputs an analysis result including information that can identify the cluster distance between the calculated input data. Then, the cluster analysis process is terminated.

FIG. 24 shows an example of the analysis result based on the cluster classification. Here, the results of cluster classification of a total of four trainees, three trainees and one expert, are shown. This figure is called a dendrogram, and is characterized by the fact that it becomes a group (cluster) in order from the most similar combination, and the intermediate process can be expressed like a hierarchy. The vertical axis represents the cluster distance (height), and it is necessary to select an appropriate method from a plurality of methods such as the Euclidean distance for calculating the distance.

In this embodiment, the data of No. 1, No. 2 and No. 3 are used as the first feature amount data of the three trainees as trainees. The data of No. 4 is used as the second feature amount data of the trained person of the expert.

Here, the trainee with the number 1 classified in the same branch as the second feature amount data (number 4), which is a skilled person, is similar to the trained person with the other numbers 2 and 3 as compared with the trained person with the other numbers 2 and 3. ing. That is, it can be judged that the skill level is high.

By calculating the cluster distance between multiple skilled workers and then the cluster distance with the trainee, it is possible to determine how much the trainee is different from the allowed cluster distance among the trainees. It can be grasped not only numerically but also visually.

In the present embodiment, the cluster distance between the first feature amount data and the second feature amount data can be calculated, and the skill level, hesitation, and concentration of the trainee can be quantitatively evaluated.

As described above, the line-of-sight evaluation unit 32 has the line-of-sight data (first feature amount data) obtained by the data extraction unit 31 and the model data (second feature amount) which is expert data stored in the model data storage unit 44. Data) can be compared.

When specifying a portion that the trainee is looking at but the trainee is not looking at during the training, the difference between the trainee's line-of-sight data and the trainee's line-of-sight data may be calculated. Further, when displaying this difference, a display method called a heat map may be used in which the color in the line-of-sight area is changed depending on the magnitude of the numerical value. In this way, by comparing the line-of-sight data of the trained person with the line-of-sight data of the trainee, it is possible to present the recommended behavior of where to look in the training.

The line-of-sight evaluation unit 32 compares the data of the past expert stored in the model data storage unit 44 with the data of the trainee acquired during the training, and combines it with the time-series data of the work or process. , It may be possible to identify the place where the difference between the data of the expert and the data of the trainer is large. The part where this difference is large may be defined as the part where mistakes are likely to occur and presented to the instructor or the trainee. In addition, when there are a plurality of procedures in a work or process that achieves the same goal, it is possible to identify a place where the difference between the data of the expert and the data of the trainer is small. The part where this difference is small may be defined as the part where mistakes are unlikely to occur and presented to the instructor or the trainee. If you do this

Further, the line-of-sight evaluation unit 32 may use the data of the past expert and the data of the trainee to determine the difficulty level of the work or the procedure depending on the magnitude of the difference. Depending on the difficulty level, a worker who is considered to be able to take appropriate measures in actual operation may be selected.

Although the flowchart of the present embodiment illustrates a mode in which each step is executed in series, the context of each step is not necessarily fixed, and even if the context of some steps is exchanged. good. Also, some steps may be executed in parallel with other steps.

The system of this embodiment includes a control device in which a dedicated chip, a controller such as an FPGA (Field Programmable Gate Array), a GPU (Graphics Processing Unit), or a CPU (Central Processing Unit) is highly integrated, and a ROM (Read Only). Storage devices such as Memory) or RAM (Random Access Memory), external storage devices such as HDD (Hard Disk Drive) or SSD (Solid State Drive), display devices such as displays, and input devices such as mice or keyboards. , With a communication interface. This system can be realized with a hardware configuration using a normal computer.

The program executed by the system of the present embodiment is provided by incorporating it into a ROM or the like in advance. Alternatively, the program may be a non-transient storage medium that is a computer-readable file such as a CD-ROM, CD-R, memory card, DVD, or flexible disk (FD) in an installable or executable format. It may be stored in the computer and provided.

Further, the program executed by this system may be stored on a computer connected to a network such as the Internet, and may be downloaded and provided via the network. The system can also be configured by connecting and combining separate modules that independently exert the functions of the components to each other via a network or a dedicated line.

In this embodiment, a chart showing the evaluation results is created for each trainee and the team, but other embodiments may be used. For example, a chart showing the evaluation result may be created for each event.

In the present embodiment, the processing server 8 collects the operation log and the behavior content of the trainee and performs the processing related to the evaluation, but other embodiments may be used. For example, a server that collects operation logs and behaviors of trainees and a server that performs processing related to evaluation may be constructed separately.

In the present embodiment, the line-of-sight measurement unit 46 and the line-of-sight measurement camera 47 are provided on the frame portion of the trainee's glasses in order to acquire the line-of-sight trajectory and the line-of-sight image of the trainee. It may be. For example, the face or eyes of the trainee may be photographed with a camera provided on the front side of the trainee to acquire the trajectory of the trainee's line of sight. Further, the scenery (line-of-sight image) seen by the trainee may be acquired by a camera provided behind the trainee.

According to the embodiment described above, the task execution determination unit for determining whether or not the trainee has executed the task corresponding to the event is provided based on the record of at least one of the action content and the operation log. It can handle various training scenarios.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, changes, and combinations can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope of the invention described in the claims and the equivalent scope thereof, as are included in the scope and gist of the invention.

1 ... Driving training evaluation system, 2 ... Main monitor device, 3 ... Main operation panel, 4 ... Sub monitor device, 5 ... Sub operation panel, 6 ... Commander operation panel, 7 ... Simulator server, 8 ... Processing server, 9 ... Mobile terminal, 10 ... field monitoring device, 11 ... input unit, 12 ... output unit, 13 ... storage unit, 14 ... communication unit, 15 ... main control unit, 16 ... training scenario setting unit, 17 ... task setting unit, 18 ... Task execution determination unit, 19 ... Task evaluation unit, 20 ... Person position acquisition unit, 21 ... Person position identification unit, 22 ... Speaker management unit, 23 ... Voice recognition unit, 24 ... Speech identification unit, 25 ... Text conversion unit, 26 ... Text mining unit, 27 ... Video analysis unit, 28 ... Line-of-sight acquisition unit, 29 ... Line-of-sight identification unit, 30 ... Line-of-sight layered unit, 31 ... Data extraction unit, 32 ... Line-of-sight evaluation unit, 33 ... Object discrimination unit, 34 ... Evaluation information output unit, 35 ... Training event display unit, 36 ... Operation log display unit, 37 ... Text display unit, 38 ... Voice playback unit, 39 ... Dictionary correction unit, 40 ... Dictionary registration unit, 41 ... Action recording unit, 42 ... Operation log recording unit, 43 ... Process data storage unit, 44 ... Model data storage unit, 45 ... Individual voice recording unit, 46 ... Line-of-sight measurement unit, 47 ... Line-of-sight measurement camera, 48 ... Result layered unit, 49 ... Site image Recording unit, 50 ... On-site voice recording unit, 51 ... Event, 52 ... Operation log, 53 ... Text data, 54, 55 ... Radar chart, 56 ... Line-of-sight trajectory, 57 ... Instruction image.

Claims (15)

A simulator that executes training events according to a training scenario,
An action recording unit that records the behavior of the trainee who is trained in the simulator,
An operation log recording unit that records the operation log of the simulator,
A task execution determination unit that determines whether or not the trainee has executed a task corresponding to the event based on at least one of the action content and the operation log.
An evaluation information output unit that outputs evaluation information that can evaluate the trainee based on the determination,
To prepare
Driving training evaluation system. A task setting unit capable of setting a plurality of types of the tasks in response to one event is provided.
The driving training evaluation system according to claim 1. The task setting unit sets an evaluation index for each type of task.
The driving training evaluation system according to claim 2. The task execution determination unit determines whether or not the trainee has executed the task for each type of the task.
The driving training evaluation system according to claim 2 or 3. A voice recording unit in which the action content includes the utterance of the trainee and records the voice related to the utterance.
A voice recognition unit that recognizes the voice and
A text conversion unit that converts the voice into text data based on a dictionary,
The speaker management unit that manages the trainee who made the utterance in a identifiable manner,
To prepare
The driving training evaluation system according to any one of claims 1 to 4. The task execution determination unit determines whether or not at least one type of the task has been executed by the combination of the utterance and the operation log.
The driving training evaluation system according to claim 5. An audio playback unit that reproduces the audio, and
A text display unit that displays the text data corresponding to the voice on the display, and
A dictionary correction unit that can correct the part corresponding to the erroneous conversion in the contents registered in the dictionary when an erroneous conversion occurs in the text conversion unit.
To prepare
The driving training evaluation system according to claim 5 or 6. A text mining unit is provided that performs text mining based on the text data acquired in the training and extracts the task required of the trainee in response to the event.
The driving training evaluation system according to any one of claims 5 to 7. The evaluation information output unit displays a time-series chart that displays the event, the operation log, and the text data in chronological order on the display.
The driving training evaluation system according to any one of claims 5 to 8. A video recording unit that includes the movements of the trainee and records videos related to the movements.
A video analysis unit that recognizes the movement, analyzes the relationship between the utterance and the event based on the video, and extracts the task required of the trainee in response to the event. When,
To prepare
The driving training evaluation system according to any one of claims 5 to 9. It is provided with an utterance identification unit capable of identifying the utterance of a specific trainee by machine learning from the voice in which the utterances of the plurality of trainees are recorded.
The driving training evaluation system according to any one of claims 5 to 10. The action content includes the position of the trainee, and the person position acquisition unit for acquiring the position and the person position acquisition unit.
A person position identification unit that identifies whether or not the trainee is in the position corresponding to the task when the event occurs.
To prepare
The driving training evaluation system according to any one of claims 1 to 11. The action content includes the line of sight of the trainee, and the line-of-sight acquisition unit for acquiring the line of sight, and the line-of-sight acquisition unit.
A line-of-sight identification unit that identifies whether or not the trainee is looking at a location corresponding to the task when the event occurs.
To prepare
The driving training evaluation system according to any one of claims 1 to 12. The evaluation information output unit is
An individual evaluation chart that graphs the individual evaluation results of the trainee, and
A team evaluation chart that graphs the team evaluation results of a team composed of the plurality of trainees, and
Show at least one of the on the display,
The driving training evaluation system according to any one of claims 1 to 13. The steps in which the simulator executes training events according to the training scenario,
The step of recording the behavior of the trainee who is trained in the simulator, and
The step of recording the operation log of the simulator and
A step of determining whether or not the trainee has performed a task corresponding to the event based on the record of at least one of the action content and the operation log.
A step of outputting evaluation information that can evaluate the trainee based on the determination, and
including,
Driving training evaluation method.

Images