JP7220042B2 - training system and program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a technique for performing work training in virtual space.

Techniques for training using VR (virtual reality) are known. For example, Patent Literature 1 describes that, in a bicycle driving simulator, the time during which the subject checks the rear from the start of driving the simulated bicycle to the end thereof is calculated as evaluation data.

JP 2017-9717 A

VR may be used to train trainees for specific tasks. In such training, the trainee tends to focus only on the object of work. However, if the specific work is dangerous work such as security, disaster response, and rescue activities, there will be events that require caution (approaching suspicious persons, fires, etc.) around the trainee during the work. Since there is a possibility, it is preferable to check the surrounding situation with a certain frequency. In the technique described in Patent Document 1, the time when a specific direction is confirmed is calculated as evaluation data, but from this evaluation data, only confirmation of a specific direction can be recognized, and whether or not the surroundings have been confirmed can be recognized. Therefore, the behavior of checking the surroundings cannot be evaluated.
SUMMARY OF THE INVENTION An object of the present invention is to evaluate a trainee's behavior of checking the surroundings in training of work performed in a virtual space.

The present invention is a training system for evaluating the behavior of a trainee who performs work training in a virtual space, comprising: a storage unit for storing a visual field image from the position of the trainee in the virtual space; a detection unit that detects a visual field direction of a trainee; a specifying unit that specifies the visual field of the trainee in the virtual space based on the visual field direction; and a display unit that displays a portion of the visual field image included in the visual field. and confirming the surroundings of the trainee based on the amount of change in the line-of-sight direction included in the field of view from the reference direction, with the direction from the trainee's position in the virtual space toward the work target as a reference direction. and a calculator for calculating an index value for evaluating behavior.

The calculation unit may increase the index value as the amount of change increases.

When the plurality of change amounts obtained in response to the change of the line-of-sight direction from the reference direction a plurality of times indicate that the line-of-sight direction is biased, the calculation unit calculates, among the plurality of change amounts, At least one may not be reflected in the index value.

The calculation unit may decrease the index value when the state in which the amount of change is greater than the threshold continues for a first predetermined time or longer.

The calculation unit may decrease the index value when the state in which the amount of change is equal to or less than the threshold continues for a second predetermined time or longer.

The calculation unit determines whether, when an event occurs in the virtual space, a difference between a direction of attention, which is a direction from the position to another position where the event occurred, and the line-of-sight direction is equal to or less than a predetermined amount. The index value may be increased or decreased depending on the situation.

The calculation unit increases the index value according to a relationship between one time included in a period in which the amount of change is greater than a threshold and another time when a specific work is performed in the virtual space. Quantities may be weighted.

The amount of change may be an angle formed by the reference direction and the line-of-sight direction in the virtual space.

The amount of change may be the length of a line connecting one point in the reference direction and another point in the line-of-sight direction in the virtual space.

Further, the present invention provides a computer included in a training system that evaluates the behavior of a trainee who performs work training in a virtual space, detecting the direction of the trainee's line of sight in the real space, and specifying a visual field of the trainee in the virtual space; and displaying a portion of the visual field image stored in the storage unit and included in the visual field from the position of the trainee in the virtual space. and a direction from the trainee's position in the virtual space toward the work target as a reference direction, based on the amount of change in the line-of-sight direction included in the visual field from the reference direction, the trainee's surroundings and a step of calculating an index value for evaluating the action of confirming.

Advantageous Effects of Invention According to the present invention, it is possible to evaluate a trainee's behavior of checking the surroundings in training for work performed in a virtual space.

It is a figure showing an example of composition of training system 1 concerning an embodiment. 2 is a diagram showing an example of the hardware configuration of the display device 10; FIG. 3 is a diagram showing an example of a functional configuration of the display device 10; FIG. It is a top view which shows an example of virtual space. 6 is a flowchart showing an example of display processing; 4 is a diagram showing an example of a moving image displayed on the display unit 16; FIG. 6 is a flowchart showing an example of evaluation processing; FIG. 10 is a diagram showing an example of a history of changes in the line-of-sight direction D2;

1. Configuration FIG. 1 is a diagram showing an example of the configuration of a training system 1 according to this embodiment. The training system 1 uses VR to provide training for various types of training, such as security, disaster response, rescue, etc., where dangerous events are assumed to occur around the work site, and it is required to be sufficiently vigilant about the surrounding situation during work. Used for work training. The training system 1 evaluates the behavior of a trainee who trains for work in virtual space. The term “training” refers to educational activities aimed at reaching certain goals. Training does not necessarily require actual work. Training also includes training and learning. The training system 1 includes multiple display devices 10 and a management device 20 . The multiple display devices 10 and the management device 20 are connected via a communication line 30 . Communication line 30 carries communications between these devices. A private network such as a LAN (Local Area Network) may be used for the communication line 30, for example.

FIG. 2 is a diagram showing an example of the hardware configuration of the display device 10. As shown in FIG. The display device 10 provides a virtual space by displaying an image that simulates the field of view of the trainee who performs the work, and records and evaluates the trainee's actions while viewing the content. Note that this content includes, for example, images and sounds. The display device 10 is worn on the trainee's head. For example, a head-mounted wearable terminal may be used as the display device 10 . The display device 10 includes a processor 11 , a memory 12 , a storage 13 , a communication section 14 , an operation section 15 , a display section 16 , a sound output section 17 and a sensor section 18 . These configurations are connected via a bus 19 .

The processor 11 executes various processes by reading programs to the memory 12 and executing them. A CPU (Central Processing Unit) may be used for the processor 11, for example. The memory 12 stores programs executed by the processor 11 . For the memory 12, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory) may be used. The storage 13 stores various data and programs. A flash memory, for example, may be used for the storage 13 . The communication unit 14 is connected to the communication line 30 and performs data communication via the communication line 30 . The operation unit 15 is used for inputting various kinds of information. For example, an operation key may be used for the operation unit 15 . The display unit 16 displays various information. A liquid crystal display, for example, may be used for the display unit 16 . The sound output unit 17 converts the audio signal into sound and outputs the sound. For example, a speaker may be used for the sound output unit 17 .

The sensor unit 18 detects movement of the trainee's head in a coordinate system in real space. This movement includes head orientation. In this embodiment, the orientation of the trainee's head in the real space is detected as the trainee's visual field direction in the real space. The sensor unit 18 may include, for example, a gyroscope, an acceleration sensor, and a geomagnetic sensor. The gyroscope measures the angle or angular velocity of the trainee's head. The acceleration sensor measures the acceleration of the trainee's head. A geomagnetic sensor measures the orientation of the trainee's head.

The display unit 16 displays an image of the virtual space corresponding to the trainee's field of view specified based on the orientation of the head detected by the sensor unit 18 . That is, the trainee can visually recognize the entire virtual space by turning the head left and right in the real space.

In the storage 13, moving image data indicating moving images captured in advance using an imaging device (not shown) is stored in advance as information associated with the coordinate system in the virtual space. In this embodiment, since the trainee does not move in the virtual space and works from a fixed position, a video simulating the view of the trainee at a fixed position in the virtual space, that is, the trainee in the virtual space Moving image data representing a moving image of a range visible from the position is stored. On the other hand, when the position of the trainee changes, such as when the trainee moves, moving image data showing moving images corresponding to each position is stored. In addition, the video includes the state of the work that is the target of the training. This moving image is composed of a plurality of panoramic images called frames. A panoramic image has a wider angle of view than the human field of view. The angle of view of the panoramic image may be 360 degrees, for example. A moving image may be captured using, for example, a single video camera equipped with a wide-angle lens, or may be generated by combining moving images captured using a plurality of video cameras that capture images in different directions. Also, by using an omnidirectional camera or the like to create a moving image from 360-degree panorama images in all directions, the training may be performed in a situation closer to the real space.

In addition, by setting the direction of 0 degrees as the specific direction in the panoramic image forming the virtual space, the range of -180 degrees to 180 degrees in the panoramic image is defined. In this embodiment, a reference direction D1, which will be described later, is used as the specific direction.

Note that the display device 10 may include a housing mounted on the trainee's head and a smartphone attached to the housing. In this case, the smartphone includes the above-described processor 11 , memory 12 , storage 13 , communication section 14 , operation section 15 , display section 16 , sound output section 17 , and sensor section 18 . The smartphone is attached to the housing in such a posture that the trainee can see the display unit 16 when the housing is worn on the trainee's head.

Returning to FIG. 1 , the management device 20 collects and outputs the trainee's evaluation results from each display device 10 . A general-purpose computer, for example, may be used as the management device 20 . The management device 20 may have the same configuration as the processor 11, the memory 12, the storage 13, the communication section 14, the operation section 15, and the display section 16 described above.

FIG. 3 is a diagram showing an example of the functional configuration of the display device 10. As shown in FIG. The display device 10 functions as an identification unit 101 , a calculation unit 102 and an output unit 103 . These functions are realized by cooperation between the program stored in the memory 12 and the processor 11 executing this program, whereby the processor 11 performs calculations or controls communication by the communication unit 14 .

The identifying unit 101 identifies the visual field of the trainee in the virtual space based on the visual field direction of the trainee in the real space. An example of a method for specifying the field of view will be described below. Assuming that the orientation of the trainee's head in real space when the video starts playing is the specific direction in the panoramic image, the orientation of the trainee's head (coordinate system in real space) and the direction in the panoramic image (in virtual space coordinate system). Then, the range between one end forming a predetermined angle (for example, 80 degrees) clockwise from a specific direction in the virtual space and the other end forming a predetermined angle (for example, 80 degrees) counterclockwise is the field of view of the trainee in the virtual space. Identify as That is, the field of view when the reproduction of the moving image is started is in the range of -80 degrees to 80 degrees, and a part of the panorama image (-80 degrees to 80 degrees) corresponding to this field of view is displayed on the display unit 16 . After that, the current field of view of the trainee in the virtual space is identified according to the change in the orientation of the trainee's head detected by the sensor unit 18 . For example, when the range of -80 degrees to 80 degrees is specified as the field of view, when the sensor unit 18 detects that the orientation of the trainee's head has changed counterclockwise by 20 degrees, the specifying unit Based on this detection result, 101 identifies a range of −60 degrees to 100 degrees obtained by changing the entire field of view by 20 degrees in the counterclockwise direction as the trainee's current field of view. For the size of the field of view, for example, the size of the field of view of a general person may be used.

The calculation unit 102 uses a direction from the position of the trainee in the virtual space toward the target of the work to be performed in the virtual space as a reference direction, and based on the amount of change in the line-of-sight direction of the trainee from this reference direction, An index value for evaluating the behavior of checking the surroundings is calculated. In this embodiment, the direction of the center of the visual field of the trainee specified by the specifying unit 101 is set as the line-of-sight direction of the trainee in the virtual space. That is, the change in the orientation of the trainee's head in the real space and the change in the line-of-sight direction in the virtual space are interlocked. Note that the line-of-sight direction of the trainee may be specified more accurately by using eye tracking or the like, as will be described later. Also, the index value is, for example, a value indicating the proficiency level of the trainee's behavior of checking the surroundings. For example, a number may be used as the index value. In this case, a higher index value may indicate a higher evaluation. However, the index value is not necessarily limited to numbers. For example, the index value may be represented by characters or symbols. The reference direction may be, for example, the direction a trainee tends to look at when performing a task. The work target may be, for example, an object on which the work included in the virtual space is performed, or a hand that performs the work included in the virtual space. During the work on the object, the trainee preferably checks his/her surroundings evenly with some frequency. The index value may be calculated, for example, according to the following first to sixth rules.

The first rule is that the more widely the trainee confirms the surroundings, the more the evaluation points are increased. This is because, for example, when looking at the work object during work, an event that requires caution may occur outside the field of view, so the trainee acted to check the surroundings as widely as possible during work. This is because it is considered to be preferable. Whether or not the trainee has widely checked the surroundings may be determined using, for example, the amount of change in the trainee's line-of-sight direction from the reference direction. For this amount of change, for example, an angle formed by the reference direction and the line-of-sight direction may be used. For example, if the amount of change is greater than the first threshold and less than the second threshold, it is determined that the trainee has confirmed the surroundings narrowly, and if the amount of change is greater than or equal to the second threshold, the trainee has confirmed the surroundings. may be determined to have broadly confirmed This is because the larger the amount of change, the more the trainee sees the multidirectional area, and the larger the visually recognized area is considered. For the first threshold value, for example, the minimum value of the amount of change that can be regarded as confirming the surroundings of the trainee may be used. The second threshold is a value greater than the first threshold. For the second threshold, for example, the minimum value of the amount of change that allows the trainee to consider that he or she has extensively checked the surroundings may be used. As a method of increasing the index value, for example, a method of adding a predetermined value to the index value may be used. Then, for example, when the trainee has checked the surroundings narrowly, a first predetermined value is added, and when it is determined that the trainee has checked the surroundings widely, a second predetermined value larger than the first predetermined value is added. may be added.

The second rule is that when the trainee changes the line-of-sight direction multiple times from the reference direction, if these line-of-sight directions are biased, at least one of the changes in the line-of-sight direction is used as an index value. It is a rule not to reflect. This is because, for example, if the trainee only confirms a certain direction, there is a risk of missing an alarming event that occurs outside the field of view when looking in that direction. Whether or not the line-of-sight direction is biased may be determined using, for example, the amount of change in the line-of-sight direction of the trainee from the reference direction. For example, the difference between the amount of change obtained when the trainee changes the line-of-sight direction from the reference direction and the amount of change obtained when the trainee continuously changes the line-of-sight direction from the reference direction is the first If it is equal to or less than a predetermined amount, it may be determined that the line-of-sight direction is biased. For this first predetermined amount, for example, the amount of change in the line-of-sight direction that can be regarded as confirming approximately the same area may be used. In this case, evaluation points do not have to be added for the second and subsequent behaviors among the behaviors of checking the surroundings a plurality of times in which the line-of-sight direction is biased.

The third rule is to decrease the index value when the trainee has not faced the reference direction D1 continuously for a first predetermined time or longer. This is because, for example, if the trainee does not look at the work target for a long time, it is considered that the work will be neglected. Whether or not the trainee faces the reference direction D1 may be determined using, for example, the amount of change in the line-of-sight direction of the trainee from the reference direction. For example, if the amount of change is out of a predetermined range including the reference direction D1, it may be determined that the trainee is not facing the reference direction D1. For this predetermined range, for example, a range of line-of-sight directions in which the trainee can be regarded as facing the reference direction D1 may be used. The upper and lower limits of the predetermined range may be set using, for example, the first threshold used in the first rule. In this case, the fact that the amount of change is out of the predetermined range indicates that the amount of change is greater than the first threshold. For the first predetermined time, for example, a time during which it is considered that the trainee is not looking at the work target to such an extent that the work is neglected may be used. As a method of decreasing the index value, for example, a method of subtracting a predetermined value from the index value may be used. For example, when the trainee has not faced the reference direction D1 continuously for a first predetermined time or longer, the predetermined value may be subtracted.

The fourth rule is to decrease the index value when the trainee faces the reference direction D1 continuously for a second predetermined time or longer. This is because, for example, if the trainee does not check the surroundings for a long period of time, there is a risk of missing an alarming event occurring outside the field of view when looking at the work target. is. Whether or not the trainee faces the reference direction D1 may be determined using, for example, the amount of change in the line-of-sight direction of the trainee from the reference direction, as in the third rule described above. For example, when the amount of change is within the above-described predetermined range, it may be determined that the trainee faces the reference direction D1. As described above, the upper and lower limits of the predetermined range may be set using the first threshold used in the first rule, for example. In this case, the fact that the amount of change is within the predetermined range indicates that the amount of change is equal to or less than the first threshold. For the second predetermined time, for example, a time during which it is considered that confirmation of the surroundings is neglected may be used. The second predetermined time may be shorter than the first predetermined time used in the third rule, for example. As a method of decreasing the index value, for example, a method of subtracting a predetermined value from the index value may be used as in the third rule described above. For example, when the trainee continues facing the reference direction D1 for a second predetermined time or more, the predetermined value may be subtracted.

The fifth rule weights the increment of the index value when the trainee faces the event in response to the occurrence of the event in the virtual space. If not, the rule is to decrease the index value. This is because, for example, when a warning event occurs during work, it is preferable to visually recognize the event. This event is, for example, an event that interferes with the work to be trained. For example, if the work to be trained is security work, the occurrence of the event may be the appearance of a suspicious person. Whether or not the trainee turned toward the event may be determined using, for example, the amount of change in the trainee's line-of-sight direction from the reference direction. For example, when the difference between the attention direction, which is the direction from the trainee's position to another position where the event occurred, and the trainee's line-of-sight direction becomes equal to or less than a second predetermined amount, the trainee can determine the event. It may be determined to have turned. For this second predetermined amount, for example, the difference in the line-of-sight direction from the direction of attention that the trainee can be regarded as visually recognizing the event may be used. The occurrence of an event may be notified to the trainee by emitting a sound indicating the occurrence of the event from the sound output unit 17, for example. As a method of weighting the increment of the index value, for example, a first weighting factor (>1) is used for the first predetermined value or the second predetermined value added to the index value according to the first rule. may be weighted. As a result, compared to the case where the index value is added according to the first rule, the increase in the index value is larger, so that the trainee who has visually recognized the event requiring caution can give a higher evaluation. As a method of decreasing the index value, for example, a method of subtracting a predetermined value from the index value may be used, like the third rule described above. Also, points may be added instead of weighting.

The sixth rule is to weight the increment of the index value when the trainee checks the surroundings immediately before performing a specific task in the virtual space. This specific work is an important work for which, for example, it is considered preferable to check the surroundings immediately before the work. For example, if the tasks to be trained include tasks involving handling cash, the specific tasks may be tasks involving handling cash. This is to prevent, for example, important work from being performed without realizing that an alarming event has occurred. Whether or not the trainee's behavior of checking the surroundings was performed immediately before performing a specific task in the virtual space can be determined, for example, from the time when the trainee performed the behavior of checking the surroundings to another time when the specific task was performed. The determination may be made based on whether or not the time until is within a third predetermined time. For the third predetermined time, for example, a time that can be regarded as immediately before a specific work is performed may be used. The timing at which a specific task is performed can be predicted, for example, from the flow of the story of the moving image. As a method of weighting the increment of the index value, for example, similar to the fifth rule, a weighting factor ( >1) may be used for weighting. As a result, compared to the case where the index value is added according to the first rule, the range of increase in the index value is larger, so that the trainee who performs the important work can be evaluated more carefully after paying attention to the surrounding conditions. can be higher.

The output unit 103 outputs the index value calculated by the calculation unit 102 . The output of this index value may be realized by transmitting it to the management device 20, for example. The timing of outputting the index value may be, for example, after the moving image ends.

2. Operation FIG. 4 is a plan view showing an example of the virtual space. In this virtual space, work training for coping with failures of the ATM 41 (Automated Teller Machine) installed in the convenience store 40 is performed. This activity includes, for example, replenishment and collection of cash. A convenience store 40 is provided with an ATM 41, an entrance 42, a plurality of shelves 43, and a counter 44 on which cash registers are placed. In this virtual space, the trainee is at the reference position P1 in front of the ATM 41 . This reference position P1 is preset. In addition, an invalid area 45 is set in this virtual space. This invalid area 45 is a narrow area between the reference position P1 and the wall of the convenience store 40, and even if a suspicious person moves toward the reference position P1, they will not pass through the invalid area 45.例文帳に追加Therefore, the trainee does not need to confirm the invalid area 45 .

2.1. Display Processing FIG. 5 is a flowchart showing an example of display processing executed by the display device 10 . This display processing is started, for example, when a signal instructing the start of display is input to the display device 10 . This signal may be input by an operation using the operation unit 15, or may be transmitted from the management device 20, for example.

In step S<b>11 , reproduction of moving images is started on the display unit 16 based on the moving image data stored in the storage 13 . Specifically, the angle-of-view portions corresponding to the range of the human visual field are extracted from each panorama image included in the moving image indicated by the moving image data, and displayed on the display unit 16 in order. The video includes work being done to deal with the ATM 41 failure. The moving image may also include an image imitating the hand of the trainee performing the work.

FIG. 6 is a diagram showing an example of a moving image displayed on the display unit 16. As shown in FIG. In the initial state, for example, a portion corresponding to the field of view V1 when the trainee faces the reference direction D1 is extracted from the panorama image included in the moving image and displayed. The reference direction D1 is, for example, the direction from the position P1 toward the work target, as shown in FIG. The work target may be, for example, the ATM 41, a part of the ATM 41, or an image imitating a trainee's hand included in the moving image. For example, when the angle indicating the size of the field of view is 80 degrees to the left and right, the field of view V1 forms an angle of 80 degrees clockwise in FIG. and the other end forming an angle of 80 degrees counterclockwise in FIG. In FIG. 4, for convenience of explanation, the field of view V1 is shown as an arc with a finite radius, but the field of view V1 may extend infinitely.

In step S12, it is determined whether or not the sensor unit 18 has detected movement of the trainee's head. If the trainee does not move his or her head, no movement of the trainee's head is detected, so the determination in step S12 is NO. In this case, the process of step S12 is repeated. On the other hand, as shown in FIG. 4, for example, when the trainee rotates the head counterclockwise by 160 degrees in FIG. is detected. In this case, the determination in step S12 becomes YES, and the process proceeds to step S13.

In step S13, the identification unit 101 identifies the visual field of the trainee based on the movement of the head detected in step S12. As shown in FIG. 4, when the trainee rotates the head counterclockwise by 160 degrees in FIG. 4, the visual field V1 is rotated counterclockwise by 160 degrees around the reference position P1. Then, the field of view V2 after rotational movement is specified.

In step S14, the display on the display unit 16 is changed according to the field of view specified in step S13. For example, a portion corresponding to the field of view V2 after rotational movement is extracted from the panorama image included in the moving image and displayed.

In step S15, it is determined whether or not the moving image has ended. If the moving image has not been reproduced to the end, the determination in step S15 is NO, and the process returns to step S12. On the other hand, for example, when the moving image has been reproduced to the end, the determination in step S15 becomes YES, and the process ends.

2.2. Evaluation Processing FIG. 7 is a flowchart showing an example of evaluation processing executed by the display device 10 . This evaluation process may be started, for example, while the moving image is being reproduced, or may be started after the moving image is finished.

In step S21, based on the field of view specified in step S13 described above, the history of changes in the trainee's line-of-sight direction D2 from the reference direction D1 is stored in the storage 13 as behavior during content viewing. The line-of-sight direction D2 is the direction from the reference position P1 toward the center of the field of view, as shown in FIG. 4, for example. The storage 13 stores, for example, the rotation angle of the line of sight formed by the reference direction D1 and the line of sight direction D2 at predetermined time intervals.

FIG. 8 is a diagram showing an example of a history of changes in the line-of-sight direction D2. The vertical axis in the figure indicates the rotation angle of the line of sight, and the horizontal axis indicates time. As shown in FIG. 4, when the trainee faces the reference direction D1, the reference direction D1 and line-of-sight direction D2 match, so the rotation angle is 0 degrees. If the trainee rotates the head counterclockwise in FIG. 4, the rotation angle has a positive value. On the other hand, if the trainee rotates the head clockwise in FIG. 4, the rotation angle has a negative value. The rotation angle takes values from -180 degrees to 180 degrees. Also, the range of rotation angles from -45 degrees to -90 degrees is set to the invalid range R1 corresponding to the invalid area 45 shown in FIG.

In step S<b>22 , the calculation unit 102 calculates an index value for evaluating the trainee's behavior of checking the surroundings based on the change history of the line-of-sight direction D<b>2 stored in the storage 13 . Here, it is assumed that the first threshold used in the first rule is 30 degrees and the second threshold is 45 degrees. Also, assume that the first predetermined amount used in the second rule is 15 degrees. Furthermore, the predetermined range used in the third rule is from 0 degrees to plus or minus 30 degrees, and the first predetermined time is 10 seconds. Furthermore, the predetermined range used in the fourth rule is from 0 degrees to plus or minus 30 degrees, and the second predetermined time is 30 seconds. Further, assume that the second predetermined amount used in the fifth rule is 15 degrees and the first weighting factor is 1.5. Furthermore, it is assumed that the third predetermined time used in the sixth rule is 5 seconds and the second weighting factor is 2.

In the example shown in FIG. 8, the rotation angle takes an extreme value at time t1. This extreme value is 160 degrees, which is greater than or equal to the second threshold of 45 degrees for the first rule. Since this indicates that the trainee has checked the surroundings widely, 2 points are added to the evaluation score according to the first rule.

During period T11 including time t2, the rotation angle is within the range of plus or minus 30 degrees, which is the predetermined range of the fourth rule. Also, the period T11 is 30 seconds or more, which is the second predetermined time of the fourth rule. Since this indicates that the trainee has continued to face the reference direction D1 for the second predetermined time or longer, one point is subtracted from the evaluation score according to the fourth rule.

At time t3, the rotation angle takes an extreme value. This extreme value is -110 degrees, and its absolute value is greater than or equal to 45 degrees, which is the second threshold of the first rule. This indicates that the trainee has widely checked the surroundings. At time t3, suspicious person S1 appears at a position rotated -100 degrees from a position in reference direction D1 about reference position P1. The appearance of the suspicious person S1 is notified to the trainee by emitting a sound indicating the appearance of the suspicious person S1, such as footsteps, from the sound output unit 17, for example, at time t3. In this case, the direction from the reference position P1 to the position of the suspicious individual S1 is the attention direction. The extreme value at time t3 is -110 degrees, and the attention angle formed by the reference direction D1 and the attention direction is -100 degrees. Their difference is 10 degrees, which is within the second predetermined amount of the fifth rule, 15 degrees. This indicates that the trainee is facing the suspicious person S1 in response to the appearance of the suspicious person S1. In this case, the 2 points added to the evaluation points according to the first rule are weighted using the first weighting factor of 1.5 according to the fifth rule. As a result, 2 points×1.5=3 points are added to the evaluation points.

At time t4, a suspicious person S2 appears at a position rotated 90 degrees from a position in the reference direction D1 about the reference position P1. The appearance of the suspicious individual S2 is notified to the trainee by emitting a sound such as footsteps indicating the appearance of the suspicious individual S2 from the sound output unit 17 at time t4, for example, in the same manner as the appearance of the suspicious individual S1 at time t3. be done. In this case, the direction from the reference position P1 to the position of the suspicious person S2 is the attention direction. However, the rotation angle at time t4 is -5 degrees, and the attention angle formed by the reference direction D1 and the attention direction is 90 degrees. These differences are 95 degrees, which is greater than the second predetermined amount of 15 degrees of the fifth rule. Since this indicates that the trainee is not facing the suspicious person S2 in response to the appearance of the suspicious person S2, one point is subtracted from the evaluation score according to the fifth rule.

At time t5, the rotation angle takes an extreme value. This extreme value is -60 degrees, and its absolute value is greater than or equal to 45 degrees, which is the second threshold of the first rule. However, this extreme value is included in the invalid range R1. Since this indicates that the trainee has confirmed the invalid area 45, the score is not added.

During the period T12 including the time t6, the rotation angle does not include 0 degrees and is continuously larger than 30 degrees, and is out of the range of 0 degrees to plus or minus 30 degrees, which is the predetermined range of the third rule. Also, the period T12 is 10 seconds or more, which is the first predetermined time of the third rule. Since this indicates that the trainee has not faced the reference direction D1 continuously for the first predetermined time or longer, one point is subtracted from the evaluation score according to the third rule.

At time t7, the rotation angle takes an extreme value. This extreme value is 40 degrees, which is greater than the first threshold of 30 degrees of the first rule and less than the second threshold of 45 degrees. Further, at time t8 after time t7, an operation of taking out cash is performed. Time T13 from time t7 to time t8 is within 5 seconds, which is the third predetermined time of the sixth rule. This indicates that the trainee checked the surroundings immediately before taking out the cash. In this case, one point added to the evaluation point according to the first rule is weighted using the second weighting factor of 2 according to the sixth rule. As a result, 1 point×2=2 points are added to the evaluation point.

At time t9, the rotation angle takes an extreme value. This extreme value is 42 degrees, which is greater than the first threshold of 30 degrees of the first rule and less than the second threshold of 45 degrees. However, both the extreme value at time t7 and the extreme value at time t9 have positive values, and there is no extreme value having a negative value between these extreme values. Also, the difference between the extreme value of 40 degrees at time t7 and the extreme value of 42 degrees at time t9 is 2 degrees, which is within 15 degrees, which is the first predetermined amount of the second rule. This indicates that although the trainee changes the line-of-sight direction D2 from the reference direction multiple times, the line-of-sight direction D2 is biased. In this case, the action of checking the surroundings performed by the trainee at time t9 is not scored according to the second rule.

In step S23, the output unit 103 outputs the index value calculated in step S22. For example, the calculated index value is temporarily stored in the storage 13, and after the video ends, the index value stored in the storage 13 is transmitted from the output unit 103 to the management device 20 together with the ID (identification) of the trainee. The management device 20 displays an index value for each trainee, for example.

In the above-described embodiment, the "sensor unit 18", the "storage unit 13", the "video data", the "first threshold", and the "predetermined amount" correspond to the "detection unit" and the "storage unit" according to the present invention, respectively. ”, “field-of-view image”, “threshold”, and “second predetermined amount”.

According to the above-described embodiment, since the evaluation points are calculated according to the trainee's action of checking the surroundings, this action can be evaluated. In addition, since the evaluation points are output, for example, the instructor of the trainee can objectively grasp the proficiency level of the action of checking the surroundings of the trainee. Furthermore, since the index value increases as the amount of change in the line-of-sight direction increases, the more widely the trainee checks the surroundings, the higher the trainee can be evaluated. Furthermore, since the index value is not added when the line-of-sight direction is biased, it is possible to prevent the trainee's evaluation from becoming high when the trainee only confirms a fixed direction. Furthermore, if the trainee does not face the reference direction D1 continuously for the first predetermined time or more, the index value decreases. evaluation can be lowered. Furthermore, if the trainee continues facing the reference direction D1 for the second predetermined time or longer, the index value decreases. You can lower the appraisal of the trainee. Furthermore, according to the occurrence of an event in the virtual space, if the trainee does not face the event, the index value decreases. can lower the rating. Furthermore, if the trainee performs an action to check the surroundings immediately before performing a specific task in the virtual space, the amount of increase in the index value is weighted. Can be rated higher than other trainees' ratings.

3. Modifications The present invention is not limited to the embodiments described above. Various modifications may be made to the embodiments described above. Moreover, the following modifications may be combined and implemented.

In the above-described embodiment, the amount of change in the line-of-sight direction of the trainee is not limited to the rotation angle of the line-of-sight in the virtual space. For example, this amount of change may be the length of a trajectory drawn by a line connecting a point on the reference direction and a point on the line-of-sight direction in the virtual space. In this case, the greater the length of the trajectory, the greater the amount of change. In another example, the amount of change may be an area surrounded by a curve representing the history of changes in line-of-sight direction and a section on the time axis. In this case, the larger the area, the larger the amount of change.

In the above-described embodiments, the amount of change in the line-of-sight direction of the trainee from the reference direction is not limited to the amount of change in the two-dimensional direction. For example, a three-dimensional change amount may be used. In this case, the length of the trajectory described above may be used as the amount of change. Further, when the amount of change is the rotation angle of the line of sight, the horizontal and vertical angles formed by the reference direction and the line of sight direction may be used.

In the embodiment described above, the rules for calculating the index value are not limited to the first to sixth rules described above. Moreover, all of the first to sixth rules may not be used. For example, the index value may be calculated using only the sixth rule.

In the first rule described above, the method of increasing the index value according to the amount of change in the line-of-sight direction is not limited to the method described in the embodiment. For example, the index value may be increased stepwise using three or more thresholds. Alternatively, the index value may gradually increase as the amount of change in the line-of-sight direction increases. That is, the larger the amount of change in the line-of-sight direction, the larger the index value. When using the amount of change in the three-dimensional direction, the horizontal direction is set to 30 degrees for the first threshold and the second threshold is set to 45 degrees, as in the embodiment, and the vertical direction is set to 30 degrees for both thresholds. The condition is that it is within a degree. Here, the reason why the vertical direction is set within 30 degrees is that basically it is not assumed that the trainee will be in danger from above or below while working with the ATM 41, and the peripheral This is because it is desirable to confirm the It should be noted that when training for work such as disaster response where falling objects are expected, it is preferable to set the threshold in accordance with the characteristics of the work, such as within 90 degrees for the vertical direction.

In the second rule described above, the method for determining whether or not the line-of-sight direction is biased is not limited to the method described in the embodiment. For example, it may be determined whether or not the line-of-sight direction is biased based on the degree of dispersion of the extreme values of the rotation angle. This dispersion degree is a value indicating the degree of dispersion. Dispersion includes, for example, dispersion. When looking at the distribution of the extreme values in the example shown in FIG. 8, for example, if the distribution of the extreme values is less than or equal to a predetermined degree, it may be determined that the line-of-sight direction is biased. Moreover, when the line-of-sight direction is biased, the index value may be reduced after reflecting the change in all line-of-sight directions in the index value. Note that when using the amount of change in the three-dimensional direction, 15 degrees, for example, is used as the first predetermined amount in the vertical direction.

In the above-described third rule, even if the trainee faces the reference direction D1 within the first predetermined time, if the time during which the trainee faces the reference direction D1 is less than the minimum time, the index value may be decreased. For this predetermined time, for example, a time during which it can be considered that the worker is paying attention to the work may be used. Here, it is assumed that the minimum time is 2 seconds. For example, even if the rotation angle includes a positive extreme value and a negative extreme value within the first predetermined time, the rotation angle is within the predetermined range of 0 degrees to plus or minus 30 degrees according to the first rule. The index value may be decreased if the range is continuously in the range for less than the minimum time of 2 seconds. Further, for example, when the reference direction D1 is not oriented with a predetermined frequency, the index value may be decreased. For example, when it continues to face the reference direction D1 for a predetermined time or more within a predetermined period, it may be determined that it faces the reference direction D1 at a predetermined frequency. This predetermined period is, for example, 30 seconds, and the predetermined time may be, for example, 20 seconds. Note that when using the amount of change in the three-dimensional direction, a predetermined range of, for example, 0 degrees to plus or minus 30 degrees is used in the vertical direction.

In the fourth rule described above, if the trainee faces the reference direction D1 continuously for the second predetermined time or longer, it is determined that the surrounding confirmation is insufficient, and the training is forcibly terminated. good too.

In the fifth rule described above, events occurring in the virtual space are not limited to the appearance of a suspicious person. For example, this event may be the entry of a person from the entrance 42 of the convenience store 40 . The entry may be notified to the trainee by emitting a sound indicating entry from the sound output unit 17, for example. In this case, for example, the direction from the reference position P1 toward the entrance 42 may be used as the attention direction. Further, for example, the sound output unit 17 may output a sound coming from the position where the event occurred when viewed from the reference position P1. In this case, multi-channel sound may be used in which sounds arrive from a plurality of different directions using a plurality of sound output units 17 . This allows the trainee to estimate the position where the event occurred by listening to the sound. Note that when using the amount of change in the three-dimensional direction, 30 degrees, for example, is used as the second predetermined amount in the vertical direction. Further, when an event occurs, if the difference between the attention direction and the line-of-sight direction does not become equal to or less than the second predetermined amount, it may be determined that the surrounding confirmation is insufficient, and the training may be forcibly terminated.

In the sixth rule described above, the relationship between the time when the specific work is performed in the virtual space and the time when the trainee checks the surroundings is not limited to the example of the embodiment described above. For example, depending on the content of the work, it may be preferable for the trainee to check the surroundings immediately after performing a specific work. In this case, the amount of increase in the index value may be weighted when the trainee takes action to check the surroundings immediately after performing a specific task in the virtual space. That is, the amount of increase in the index value may be weighted according to the relationship between the first time when the trainee performs the action of checking the surroundings and the other time when the specific work is performed in the virtual space. .

In the above-described embodiment, the index value may be calculated based on the speed or acceleration of movement of the trainee's head in addition to the amount of change in the line-of-sight direction. For example, if the speed or acceleration detected by the sensor unit 18 is greater than or equal to a predetermined degree, the index value may not be added. This is because when the trainee rotates the head quickly, it is difficult to say that the trainee is checking the surroundings.

In the above-described embodiment, the description has been given on the assumption that the trainee does not move, but the present invention is not limited to this. For example, the position of the trainee may automatically change according to the passage of time in the video showing the virtual space. In this case, the position of the trainee at each time and the image (moving image) simulating the field of view of the trainee at each time are associated with each other and stored in advance in the storage 13 . Further, movement in the virtual space may be realized by interlocking the movement of the trainee, such as walking, in the real world with the movement of the trainee in the virtual space. For example, the display device 10 may determine the position of the display device 10 using the sensor information detected by the sensor unit 18, and determine the position of the trainee in the virtual space based on this position. . In this case, since the images are distributed according to the movement of the trainee, the storage 13 stores in advance images simulating the trainee's field of view at each position in the virtual space. Note that the method of determining the position of the trainee in the virtual space is not limited to this, and other known techniques can be used.

In the above-described embodiment, the line-of-sight direction of the trainee is not limited to the direction from the reference position P1 toward the center of the trainee's visual field. For example, the display device 10 may detect the line of sight of the trainee using a well-known eye tracking technique, and use the detected direction of the line of sight as the direction of the line of sight. This makes it possible to evaluate the trainee's behavior of paying attention to the surroundings by changing the line of sight without changing the direction of the head, enabling more practical training.

In the embodiments described above, well-known motion capture techniques may be used to detect the trainee's movements. In this case, the trainee may, for example, make a motion to perform a specific task. In this case, for example, in the sixth rule described above, the time when this motion is detected may be used as the time when the specific work is performed.

In the above-described embodiment, the invalid area 45 is not limited to an area through which the suspicious person does not pass when moving to the reference position P1. For example, if a plurality of people are to deal with a failure of the ATM 41 , an area in which other people are present or an area in which other people are responsible for checking the surroundings may be set as the invalid area 45 . Note that the invalid area 45 does not necessarily have to be set, for example, when there is no area that does not require confirmation.

In the embodiments described above, the display unit 16 may be a device that projects an image. For example, the display unit 16 may use a projection unit such as a projector and a display surface on which an image is displayed, or may use a projection unit that directly projects an image onto the retina.

In the above-described embodiments, moving image data may be transmitted from the management device 20 to the display device 10 . In this case, for example, moving image data may be distributed from the management device 20 to a plurality of display devices 10 used in different locations via a wide area network such as the Internet.

In the above-described embodiment, the moving image content is not limited to the one used for the training of dealing with failure of the ATM 41 . Any video content may be used as long as it is used for training for work that requires the trainee to check the surroundings.

In the embodiments described above, the management device 20 may not necessarily be provided. For example, the display device 10 may be used alone. In this case, the index value may be displayed on the display unit 16, for example.

In the above-described embodiment, the training system 1 may be used not only for training using VR, but also for training using AR (Augmented Reality) or MR (Mixed Reality).

In the above-described embodiments, the steps of processing performed in the training system 1 are not limited to the examples described in the above-described embodiments. The steps of this process may be interchanged unless inconsistent. The invention may be provided as a method comprising steps of processing performed in the training system 1 .

The present invention may be provided as a program executed by the display device 10 or the management device 20. FIG. This program may be downloaded via a communication line such as the Internet. In addition, this program is provided in a state recorded on computer-readable recording media such as magnetic recording media (magnetic tapes, magnetic discs, etc.), optical recording media (optical discs, etc.), magneto-optical recording media, semiconductor memories, etc. may

References to "first," "second," and "third," as used herein, do not limit the quantity or order of the elements in which they are used. These descriptions are used to distinguish between two or more elements, and may be used interchangeably with numbers, for example. Numerical values such as each threshold value are examples, and different values may be used according to the characteristics of the work.

1: training system, 10: display device, 11: processor, 12: memory, 13: storage, 14: communication unit, 15: operation unit, 16: display unit, 17: sound output unit, 18: sensor unit, 101: Identification unit, 102: calculation unit, 103: output unit

Claims (9)

A training system that evaluates the behavior of a trainee who performs work training in a virtual space ,
a detection unit that detects the visual field direction of the trainee in real space;
a specifying unit that specifies the visual field of the trainee in the virtual space based on the visual field direction;
a storage unit that stores a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
a display unit that displays a portion included in the field of view in the field of view image;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target , and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. an evaluation unit that evaluates the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation ;
The evaluation unit, in the evaluation when the line-of-sight change occurs a predetermined number of times, lowers the evaluation when the line-of-sight direction is biased toward a part other than the reference direction than the evaluation when the line-of-sight direction is not biased toward the part. training system. A training system that evaluates the behavior of a trainee who performs work training in a virtual space,
a detection unit that detects the visual field direction of the trainee in real space;
a specifying unit that specifies the visual field of the trainee in the virtual space based on the visual field direction;
a storage unit that stores a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
a display unit that displays a portion included in the field of view in the field of view image;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target, and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. and an evaluation unit that evaluates the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation.
with
The training system, wherein the evaluation unit lowers the evaluation when the state in which the angle difference is greater than the threshold continues for a predetermined time or longer. A training system that evaluates the behavior of a trainee who performs work training in a virtual space,
a detection unit that detects the visual field direction of the trainee in real space;
a specifying unit that specifies the visual field of the trainee in the virtual space based on the visual field direction;
a storage unit that stores a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
a display unit that displays a portion included in the field of view in the field of view image;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target, and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. and an evaluation unit that evaluates the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation.
with
The training system, wherein the evaluation unit lowers the evaluation when the state in which the angle difference is equal to or less than the threshold continues for a predetermined time or longer. The training according to any one of claims 1 to 3, wherein the evaluation unit calculates an index value by evaluating the behavior of checking the surroundings of the trainee, and increases the index value as the angle difference increases. system. The evaluation unit calculates an index value by evaluating the behavior of checking the surroundings of the trainee, and when an event occurs in the virtual space, the evaluation unit calculates an index value in a direction from the position to another position where the event occurred. 5. The training system according to any one of claims 1 to 4 , wherein the index value is increased or decreased depending on whether a difference between a direction of attention and the line-of-sight direction is equal to or less than a predetermined amount. The evaluation unit calculates an index value by evaluating the behavior of checking the surroundings of the trainee, and calculates one time included in a period in which the angle difference is greater than the threshold, and a specific task in the virtual space. 6. The training system according to any one of claims 1 to 5 , wherein the amount of increase in the index value is weighted according to the relationship with other times at which the is performed. A computer included in a training system that evaluates the behavior of a trainee who performs work training in a virtual space,
detecting the trainee's viewing direction in real space;
identifying the trainee's field of view in the virtual space based on the viewing direction;
a step of storing a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
displaying a portion of the field-of-view image included in the field of view;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target , and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. A program for executing a step of evaluating the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation ,
In the evaluating step, in the evaluation when the line-of-sight change occurs a predetermined number of times, the evaluation in the case where the line-of-sight direction is biased toward a part other than the reference direction is higher than the evaluation when the line-of-sight direction is not biased in the part. lower
program. A computer included in a training system that evaluates the behavior of a trainee who performs work training in a virtual space,
detecting the trainee's viewing direction in real space;
identifying the trainee's field of view in the virtual space based on the viewing direction;
a step of storing a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
displaying a portion of the field-of-view image included in the field of view;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target, and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. A program for executing a step of evaluating the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation,
In the evaluating step, if the state in which the angular difference is greater than the threshold continues for a predetermined time or longer, the evaluation is lowered.
program. A computer included in a training system that evaluates the behavior of a trainee who performs work training in a virtual space,
detecting the trainee's viewing direction in real space;
identifying the trainee's field of view in the virtual space based on the viewing direction;
a step of storing a visual field image from the position of the trainee in the virtual space and a history of line-of-sight directions included in the visual field;
displaying a portion of the field-of-view image included in the field of view;
Based on the history, an angle difference between a reference direction, which is a direction from the position of the trainee in the virtual space toward the work target, and the line-of-sight direction is obtained, and the number of times the line-of-sight changes occur when the angle difference is greater than a threshold is obtained. A program for executing a step of evaluating the behavior of checking the surroundings of the trainee so that the more the number, the higher the evaluation,
In the evaluating step, if the state in which the angle difference is equal to or less than the threshold continues for a predetermined time or longer, the evaluation becomes low.
program.

Images