JP6838902B2 - Pseudo-experience providing device, simulated experience providing method, simulated experience providing system, and program - Google Patents

Description

The present invention relates to a simulated experience providing device, a simulated experience providing method, a simulated experience providing system, and a program, and more particularly to a technique for providing training using VR (Virtual Reality) technology.

In recent years, VR technology for the purpose of games and training has been rapidly developed. For example, Patent Document 1 discloses a virtual reality simulator capable of performing simulated training in real time in VR.

Japanese Unexamined Patent Publication No. 2000-122520

By using VR, it is possible to easily and safely reproduce an environment that is difficult to reproduce or a dangerous environment for training. On the other hand, there are individual differences in the stimulus received from the virtual environment presented using VR, and even in the same simulated training, the training effect may differ depending on the user. If the stimulus received from the virtual environment is too weak, the training effect will be low, and if it is too strong, it may be difficult to continue the training.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to provide a technique for stabilizing a training effect using virtual reality.

The first aspect of the present invention is a simulated experience providing device. The simulated experience providing device is selected by a selection unit that selects a scene constituting the simulated experience presented to the user based on the stimulus intensity set for the user of the simulated experience providing device, and a selection unit selected by the selection unit. Based on the presentation unit that presents the scene to the user, the biometric information acquisition unit that acquires the biometric information measured by the measuring device that measures the biometric information of the user, and the biometric information acquired by the biometric information acquisition unit. It is provided with an intensity adjusting unit for resetting the stimulation intensity.

When the intensity adjusting unit resets the stimulus intensity, the selection unit may select a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity after the resetting.

The simulated experience providing device may further include a setting unit that sets an initial value of the stimulus intensity based on the foresight information about the user.

The simulated experience providing device includes the results of a questionnaire created by each of a plurality of users using the simulated experience providing device before using the simulated experience providing device, the evaluation results regarding the simulated experience created after using the simulated experience providing device, and the above. A stimulus intensity database that stores the results of the questionnaire and the evaluation results in association with the stimulus intensity based on the stimulus intensity set for each of the plurality of users while using the simulated experience providing device. The setting unit may further include a database generation unit that generates the above-mentioned stimulus intensity by referring to the database based on the result of a questionnaire answered by the user before using the simulated experience providing device. The initial value of may be set.

The biological information acquisition unit may acquire the blood flow rate of the user measured by the measuring device, and the intensity adjusting unit changes the stimulation intensity set for the user based on the change in the blood flow rate. You may.

A second aspect of the present invention is a simulated experience providing method. In this method, a step of selecting a scene constituting a simulated experience to be presented to the user based on a stimulus intensity set for the user of the simulated experience providing device, and presenting the selected scene to the user. The processor executes a step, a step of acquiring the biometric information measured by the measuring device that measures the biometric information of the user, and a step of resetting the stimulus intensity based on the acquired biometric information.

A third aspect of the present invention is a program. This program has a function of selecting a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity set in the user of the simulated experience providing device on the computer, and the selected scene is selected by the user. The function presented to the user, the function of acquiring the biometric information measured by the measuring device that measures the biometric information of the user, and the function of resetting the stimulus intensity based on the acquired biometric information are realized.

A fourth aspect of the present invention is a simulated experience providing system. This system includes one or more simulated experience providing devices that provide simulated experiences to one or more users, one or more head-mounted displays to be attached to each of the one or more users, and the above. Includes one or more measuring devices that measure the biometric information of one or more users. Each of the one or a plurality of simulated experience providing devices selects a selection unit that selects a scene constituting the simulated experience presented to the user based on the stimulus intensity set for the user, and the selection unit. The stimulation intensity is reset based on the presentation unit that presents the scene to the user, the biological information acquisition unit that acquires the biological information measured by the measuring device, and the biological information acquired by the biological information acquisition unit. It is provided with a strength adjusting unit.

Each of the scenes constituting the simulated experience may include a plurality of expression forms corresponding to the stimulus intensity, and the selection unit selects the expression form of the scene to be presented to the user based on the stimulus intensity. You may.

The selection unit (1) changes the order of the scenes presented to the user based on the stimulus intensity when there is one user of the simulated experience providing system, and (2) of the simulated experience providing system. When there are a plurality of users and a simulated experience common to each user is provided at the same time, the order change of the scenes based on the stimulus intensity may be prohibited.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, computer programs, data structures, recording media and the like are also effective as aspects of the present invention.

According to the present invention, the training effect using virtual reality can be stabilized.

It is a figure which shows the outline of the simulated experience providing system which concerns on embodiment of this invention. It is a figure which shows typically the configuration example of the simulated experience providing system which concerns on embodiment of this invention. It is a figure which shows typically the functional structure of the simulated experience providing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the initial value setting table which shows the correspondence relation between the questionnaire result which is the prior information about a user, and the initial value of the training intensity set for a user in a table format. It is a figure which shows the combination of scenes based on a stimulus intensity in a tabular form. It is a figure which shows the flow of the scenario based on the stimulus intensity. It is a figure which shows the relationship of the main functions realized by the simulated experience providing system which concerns on embodiment of this invention. It is a figure which shows typically the functional structure of the strength adjustment part which concerns on embodiment of this invention. It is a flowchart for demonstrating the flow of the simulated experience providing processing executed by the simulated experience providing system which concerns on embodiment of this invention.

<Outline of the embodiment>
An outline of the embodiment of the present invention will be described. The simulated experience providing system according to the embodiment is a system for virtually reproducing a dangerous environment such as an accident situation using VR technology and presenting it to a user who is a subject. Since the situation is reproduced using video, audio, etc., it is possible for the subject to relive various situations while ensuring the safety of the user. In particular, by presenting an image using an HMD (Head Mounted Display) or the like, a strong immersive feeling can be given to the user, so that it can be expected to enhance the training effect of learning a dangerous situation and avoiding it.

The inventor of the present application has found that the reproduction of a virtual situation using VR technology differs in the intensity of stimulation felt by each user even when the same video and audio are presented. When a user feels a weak stimulus from a simulated experience, the training tends to be less effective than when it is strong. On the other hand, if the stimulus felt by the user from the simulated experience is too strong, it may interfere with the continuation of the training.

Therefore, the simulated experience providing system according to the embodiment acquires the biological information of the user who is presenting the simulated experience. Here, the "biological information" is information indicating the time change of the biological signal of the user obtained by measuring the user who is presented with the simulated experience. The "biological signal" is, for example, at least one of a user's blood flow, pulse, blood pressure, heartbeat, body temperature, sweating amount, pupil movement, electroencephalogram, magnetoencephalogram, electrocardiogram, and myoelectric potential. In general, the biological signals are different when the user's psychological state is bored, normal, and when he / she feels a sense of urgency.

The simulated experience providing system according to the embodiment changes the stimulation intensity of the simulated experience presented to the user based on the biological information acquired by measuring the user. As a result, the simulated experience providing system according to the embodiment can change the stimulus intensity of the simulated experience presented to the user according to the psychological state of the user. As a result, it is possible to provide a simulated experience with a stimulus intensity suitable for the user, so that the training effect can be stabilized regardless of the user.

FIG. 1 is a diagram showing an outline of a simulated experience providing system 1 according to an embodiment of the present invention. The outline of the operation of the simulated experience providing system 1 according to the embodiment will be described below with reference to FIG.

An example of the simulated experience providing system 1 shown in FIG. 1 includes a simulated experience providing device 100, a head-mounted display 200, a measuring device 300, a posture detection sensor 400, a camera 500, and a monitor 600.

The simulated experience providing device 100 is a device for presenting a simulated experience to the user U based on the stimulus intensity set for the user U. The simulated experience providing device 100 is, for example, a PC (Personal Computer), a server, or the like, and is an electronic device including various computing resources such as a processor such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit) and a memory.
Hereinafter, the processing procedure will be described with reference to (1), (2), (3) and (4), and the description corresponds to (1), (2), (3) and (4) in FIG. To do.

(1) The simulated experience providing device 100 first selects the stimulus intensity set for each user U. (2) The simulated experience providing device 100 presents the simulated experience to the user U based on the selected stimulus intensity. More specifically, the simulated experience providing device 100 is connected to the head-mounted display 200 worn by the user U by wire or wirelessly, and the video and audio expressing the simulated experience are reproduced on the head-mounted display 200. FIG. 1 shows an example in which a simulated experience of a traffic accident is presented to the user U. The user U wearing the head-mounted display 200 can experience the situation in which the automobile is moving toward the user U.

The user U is equipped with a measuring device 300 for measuring the biological information of the user U. (3) The measuring device 300 measures the biological information of the user U who is presented with the simulated experience and transmits it to the simulated experience providing device 100. (4) The simulated experience providing device 100 adjusts the stimulation intensity based on the biological information of the user U measured by the measuring device 300. By repeating the above, the simulated experience providing device 100 can flexibly change the stimulation intensity while presenting the simulated experience to the user U.

In the example of the simulated experience providing device 100 shown in FIG. 1, the simulated experience is presented to the user U by using the head-mounted display 200. Therefore, a posture detection sensor 400 for detecting the posture of the user U and a camera 500 for photographing the user U are also provided. The simulated experience providing device 100 can change the image presented to the user U according to the posture of the user U detected by the posture detection sensor 400. That is, the video itself to be presented to the user U and the presentation order thereof can be changed according to the reaction of the user U.

Further, since the field of view of the user U wearing the head-mounted display 200 is blocked, it is difficult for the user U to notice even if an object approaches. Therefore, the simulated experience providing device 100 can analyze the image captured by the camera 500 and notify the user U when the user U approaches the object. As a result, the safety of training by the simulated experience providing device 100 can be enhanced.

The simulated experience providing system 1 also includes a monitor 600 for displaying an image presented to the user U. The monitor 600 can also display a GUI (Graphical User Interface) for controlling the simulated experience presented by the simulated experience providing device 100. The operator of the simulated experience providing device 100 can manually change the stimulation intensity, change the simulated experience, or interrupt the simulated experience while looking at the monitor 600.

<Configuration of simulated experience provision system 1>
FIG. 2 is a diagram schematically showing a configuration example of the simulated experience providing system 1 according to the embodiment of the present invention. FIG. 2 illustrates a system in which a plurality of simulated experience providing devices 100 are linked to present a simulated experience to a plurality of users U, unlike the example of the simulated experience providing system 1 shown in FIG.

In the example of the simulated experience providing system 1 shown in FIG. 2, the two simulated experience providing devices 100, the first simulated experience providing device 100a and the second simulated experience providing device 100b, can communicate with each other via the relay device 800. It is connected with. The simulated experience providing system 1 can also connect two or more simulated experience providing devices 100 in a manner capable of communicating with each other.

The first simulated experience providing device 100a provides the first user U1 with a simulated experience, and the second simulated experience providing device 100b provides the second user U2 with a simulated experience. Therefore, the first head-mounted display 200a, the first measuring device 300a, the first posture detection sensor 400a, the first camera 500a, and the first monitor 600a are connected to the first simulated experience providing device 100a. Further, the first simulated experience providing device 100a is also connected to the first hearing reproducing device 250a for the first user U1 to wear.

The second simulated experience providing device 100b is also the same as the first simulated experience providing device 100a, and is the second head-mounted display 200b, the second auditory reproduction device 250b, the second measuring device 300b, the second posture detection sensor 400b, and the second camera. A 500b and a second monitor 600b are connected.

In the example of the simulated experience providing system 1 shown in FIG. 2, a tactile stimulus can be presented to the user U in addition to the video and audio as a simulated experience. Therefore, the simulated experience providing system 1 includes both the first tactile reproduction device 700a for presenting the tactile stimulus to the first user U1 and the second tactile reproduction device 700b for presenting the tactile stimulus to the second user U2. Be prepared.

The first hearing reproduction device 250a and the second hearing reproduction device 250b are realized by using, for example, headphones. Further, the first tactile reproduction device 700a and the second tactile reproduction device 700b can be realized by, for example, a vibration generator and a discharge device, respectively.

In this way, in the simulated experience providing system 1, a plurality of simulated experience providing devices 100 can cooperate to present a simulated experience to a plurality of users U. As a result, the simulated experience providing system 1 can provide an opportunity for a plurality of users U to collaborate and experience the same simulated experience, and can provide more practical training.

The simulated experience providing system 1 shown in FIG. 2 further includes a stop switch for urgently stopping the presentation of the simulated experience. More specifically, the simulated experience providing system 1 includes a first stop switch 900a for the operator to stop the presentation of the simulated experience by the first simulated experience providing device 100a. Similarly, the simulated experience providing system 1 includes a second stop switch 900b for the operator to stop the presentation of the simulated experience by the second simulated experience providing device 100b.

Hereinafter, in the present specification, the first simulated experience providing device 100a and the second simulated experience providing device 100b, the first head mount display 200a and the second head mount display 200b, the first hearing reproducing device 250a and the second hearing reproducing device 250b, First measuring device 300a and second measuring device 300b, first posture detection sensor 400a and second posture detection sensor 400b, first camera 500a and second camera 500b, first monitor 600a and second monitor 600b, first tactile reproduction Unless the device 700a and the second tactile reproduction device 700b, the first stop switch 900a and the second stop switch 900b, and the first user U1 and the second user U2 are particularly distinguished, they are simply simulated experience providing devices, respectively. 100, head mount display 200, auditory reproduction device 250, measurement device 300, attitude detection sensor 400, camera 500, monitor 600, tactile reproduction device 700, stop switch 900, and user U.

<Functional configuration of the simulated experience providing device 100>
FIG. 3 is a diagram schematically showing a functional configuration of the simulated experience providing device 100 according to the embodiment of the present invention. The simulated experience providing device 100 includes a control unit 110 and a storage unit 120. The control unit 110 includes a selection unit 111, a presentation unit 112, a biological information acquisition unit 113, a strength adjustment unit 114, a setting unit 115, and a database generation unit 116. Further, the storage unit 120 includes a scenario storage unit 121, a stimulus intensity storage unit 122, and a stimulus intensity database 123.

FIG. 3 shows a functional configuration for realizing the simulated experience providing device 100 according to the embodiment, and other configurations are omitted. In FIG. 3, each element described as a functional block that performs various processes can be configured by a CPU, a main memory, and other LSI (Large Scale Integration) in terms of hardware. In terms of software, it is realized by a program loaded in the main memory. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and the present invention is not limited to any of them.

The simulated experience is composed of a combination of multiple "scenes". The "scene" includes one or more moving image frames, audio data reproduced in conjunction with the moving image frames, and stimulus information to be reproduced by the tactile reproduction device 700. The simulated experience providing system 1 provides the user U with a simulated experience by presenting a plurality of scenes to the user U in order.

Different scenes that make up a simulated experience give different stimuli to user U. For example, in a simulated experience of traffic accident prevention training, the former gives a stronger stimulus to the user U than the latter in the scene where the car actually hits a person and the scene where the car stops before hitting a person. ..

Further, even if the combination of scenes constituting the simulated experience is the same, if the order of presenting each scene is different, the stimulus given to the user U may be different. For example, at the beginning of a simulated experience, a distant car gradually approaches and collides with a person, and a scene in which a car collides with a person immediately after the start of the simulated experience is reproduced, and then the first stage is presented. Then, it is considered that the former gives a weaker stimulus to the user U than the latter. In the former case, the user U can predict what kind of simulated experience will be presented.

Therefore, the selection unit 111 selects a scene constituting the simulated experience presented to the user U based on the stimulus intensity set for the user U of the simulated experience providing device 100. Specifically, the scenario storage unit 121 stores the scenes constituting the simulated experience in association with the stimulus intensity. The selection unit 111 selects a scene matching the stimulus intensity set for the user U from the scenario storage unit 121. The stimulus intensity set in the user U is stored in the stimulus intensity storage unit 122. The details of scene selection based on the stimulus intensity will be described later.

The presentation unit 112 presents the scene selected by the selection unit 111 to the user U. Specifically, the presentation unit 112 generates information to be reproduced by the head-mounted display 200, the auditory reproduction device 250, or the tactile reproduction device 700, and causes the head-mounted display 200, the auditory reproduction device 250, or the tactile reproduction device 700 to reproduce the information. ..

The biological information acquisition unit 113 acquires the biological information measured by the measuring device 300 that measures the biological information of the user U. Although not limited, the measuring device 300 measures the blood flow rate of the user U, and the biological information acquisition unit 113 acquires the blood flow rate of the user U measured by the measuring device 300.

The intensity adjusting unit 114 changes the stimulus intensity based on the change in the biological information acquired by the biological information acquisition unit 113, and stores the stimulus intensity in the stimulus intensity storage unit 122. The intensity adjusting unit 114 is obtained by learning the biological information measured when the scenes at various stimulus intensities are presented to a plurality of users U by using a known machine learning method such as deep learning. By using the stimulus intensity selection discriminator, the stimulus intensity to be reset for the user U is acquired. This stimulus intensity selection discriminator inputs biological information measured from the user U and outputs the stimulus intensity to be set for the user U.

When the intensity adjusting unit 114 resets the stimulus intensity, the selection unit 111 creates a scene that constitutes a simulated experience presented to the user U based on the stimulus intensity after the resetting stored in the stimulus intensity storage unit 122. Select from the scenario storage unit 121. As a result, the simulated experience providing system 1 can adjust the stimulus intensity of the simulated experience according to the change in the psychological state of the user U who is presented with the simulated experience.

<Setting the initial value of stimulus intensity>
Here, the initial setting of the stimulation intensity of the simulated experience will be described.
The simulated experience providing system 1 according to the embodiment selects the stimulus intensity of the simulated experience presented to the user U based on the biological information of the user U. Therefore, since the biological information of the user U cannot be acquired before the simulated experience is presented to the user U, the simulated experience providing system 1 cannot select the stimulus intensity of the simulated experience. Therefore, the simulated experience providing system 1 includes a setting unit 115 that sets an initial value of the stimulus intensity based on the foresight information about the user U.

More specifically, the setting unit 115 obtains the stimulus intensity by referring to the stimulus intensity database 123 based on the result of the questionnaire created by the user U before receiving the presentation of the simulated experience from the simulated experience providing system 1. Then, it is stored in the stimulus intensity storage unit 122.

Here, the "questionnaire" is prior information of the user U referred to by the setting unit 115 in order to determine the stimulus intensity of the user U, and is composed of a plurality of questions and the answers of the user U to the questions. There is. Examples of questionnaire questions include "whether or not you have experienced so-called VR sickness", "whether or not you are not good at high places", "whether or not you are not good at electrical stimulation", etc. This is a question directly or indirectly related to the simulated experience presented by the system 1 to the user U.

FIG. 4 is a diagram showing an example of an initial value setting table showing the correspondence relationship between the questionnaire result, which is prior information about the user U, and the initial value of the training intensity set in the user U in a tabular format. The initial value setting table of FIG. 4 shows the relationship between the score and the stimulus intensity when the questionnaire result of the user U is scored out of 100 points.

In the initial value setting table shown in FIG. 4, the stimulation intensity is represented in 10 steps from 1 to 10. Here, the larger the numerical value indicating the stimulus intensity, the stronger the stimulus given to the user U by the simulated experience. The initial value setting table is stored in the stimulus intensity storage unit 122. The setting unit 115 sets the initial value of the stimulus intensity by referring to the initial value setting table stored in the stimulus intensity database 123 based on the result of the questionnaire answered by the user U before using the simulated experience providing system 1. Set. As a result, the setting unit 115 can select the stimulus intensity determined based on the subjective evaluation of the user U based on the response items of the user U.

The stimulus intensity database 123 includes the results of a questionnaire created before the use of the simulated experience providing system 1 by each of the plurality of users U using the simulated experience providing system 1, the evaluation results regarding the simulated experience created after the use, and the simulated experience. It stores an evaluation database that associates the relationship with the stimulus intensity set for each of the plurality of users U while using the provided system 1.

Here, the "evaluation regarding the simulated experience" includes the answers of each user U to the same questions as the above-mentioned questionnaire. The database generation unit 116 acquires the evaluation result by the user U using the simulated experience providing system 1, and generates and updates the evaluation database stored in the stimulus intensity database 123. The database generation unit 116 also updates the initial value setting table stored in the stimulus intensity storage unit 122 based on the evaluation database.

In this way, the database generation unit 116 maintains and manages the stimulus intensity database and the initial value setting table, so that the evaluation of the user U who actually used the simulated experience providing system 1 can be reflected in the initial value setting table. it can. As a result, the accuracy of the initial value setting table is improved, and the simulated experience providing system 1 can provide the user U with a simulated experience that is more suitable for the user U.

<Scene selection based on stimulus intensity>
Subsequently, the selection of the scene based on the stimulus intensity executed by the selection unit 111 will be described.
FIG. 5 is a diagram showing the combination of scenes based on the stimulation intensity in a tabular format. Hereinafter, in the present specification, the subject of the simulated experience presented by the simulated experience providing system 1 to the user U will be described as a “scenario”. For example, FIG. 5 illustrates a scene constituting "Scenario 1" with the theme of traffic safety training. The table shown in FIG. 5 is stored in the scenario storage unit 121.

In the example shown in FIG. 5, scenario 1 is composed of five scenes from scene 1 to scene 5. Moreover, even in the same scene, there are a plurality of types depending on the stimulation intensity. For example, there are four types of scenes 1 to scenes 1-1 to 1-4, and the stimulation intensity becomes stronger in this order.

According to the example shown in FIG. 5, when the stimulus intensity set for the user U is 1, the selection unit 111 performs "scene 1-1", "scene 2-1", "scene 3-1", and "scene". Select scenario 1 composed of "4-1" and "scene 5-1". When the stimulus intensity set for the user U is 10, the selection unit 111 is set to "Scene 1-4", "Scene 2-4", "Scene 3-4", "Scene 4-4", and "Scene 4-4". Select scenario 1 composed of "5-4".

6 (a)-(b) are diagrams showing the flow of the scenario based on the stimulation intensity. Specifically, FIG. 6A is a diagram showing in tabular form the correspondence between the stimulus intensity and the flow type of the scenario for each scenario. Further, FIG. 6B is a diagram schematically showing the flow of the scenario for each type. Both the tabular correspondence shown in FIG. 6A and the flow shown in FIG. 6B are stored in the scenario storage unit 121.

In the example shown in FIG. 6A, there are four types of flows in scenario 1 from type 1 to type 4. For example, when the stimulus intensity is 1 and 2, a type 1 flow is assigned to both. On the other hand, when the stimulation altitude is 10, scenario 1 is assigned a type 4 flow.

As shown in FIG. 6B, when the flow types are different, the order in which the scenes constituting the scenario are presented is different. For example, in the example shown in FIG. 6B, in the type 1 flow, the scenes constituting the scenario appear in order from scene 1 to scene 5. Further, depending on the judgment of the user U, the scene 3 may not appear. On the other hand, in the type 4 flow, scene 5 appears at the beginning, and then scene 4 also appears. Although the order of appearance of the scenes changes depending on the judgment of the user U, all the scenes are presented as a whole.

As described above, each of the scenes constituting the simulated experience includes a plurality of expression forms corresponding to the stimulus intensity set for the user U. The selection unit 111 selects the expression form of the scenario presented to the user U based on the stimulus intensity set in U.

More specifically, the selection unit 111 acquires the stimulus intensity set for the user U from the stimulus intensity storage unit 122. Based on the acquired stimulus intensity, the selection unit 111 acquires a scene that constitutes a scenario with reference to the scenario storage unit 121. Subsequently, the selection unit 111 acquires the type of the scenario flow and the scenario flow corresponding to the type with reference to the scenario storage unit 121 based on the acquired stimulus intensity. The presentation unit 112 presents the simulated experience to the user U according to the flow of the scenario acquired by the selection unit 111. As a result, the simulated experience providing system 1 can present the scene based on the stimulus intensity set in the user U to the user U.

<Update of stimulus intensity>
FIG. 7 is a diagram showing the relationship between the main functions realized by the simulated experience providing system 1 according to the embodiment of the present invention, and is a diagram for explaining the update of the stimulus intensity. Specifically, the simulated experience providing system 1 includes a stimulus intensity initial value setting function F0 mainly executed by the setting unit 115, a simulated experience presentation function F1 mainly executed by the selection unit 111 and the presentation unit 112, and a biological information acquisition unit. It has at least a biological information measurement function F2 mainly executed by 113, a stimulus intensity adjustment function F3 mainly executed by the intensity adjusting unit 114, and an emergency stop function F4.

After executing the stimulus intensity initial value setting function F0 once, the simulated experience providing system 1 executes the simulated experience presenting function F1, the biological information measurement function F2, and the stimulus intensity adjusting function F3 in parallel. When the simulated experience providing system 1 executes the simulated experience presenting function F1, the simulated experience is presented to the user U, and the biological information measurement function F2 of the simulated experience providing system 1 determines the change in the biological information of the user U due to the intensity of the stimulus. measure. Based on the biological information of the user U measured by the biological information measurement function F2 of the simulated experience providing system 1, the stimulus intensity adjusting function F3 of the simulated experience providing system 1 should be set to the user U at that time. To adjust.

<Emergency stop of simulated experience>
FIG. 8 is a diagram schematically showing a functional configuration of the strength adjusting unit 114 according to the embodiment of the present invention, and in particular, a diagram showing a functional configuration for urgently stopping the presentation of a simulated experience. The intensity adjusting unit 114 includes a biological information analysis unit 1140, a stop threshold storage unit 1141, a comparison unit 1142, and an OR unit 1143.

The biological information analysis unit 1140 analyzes the biological information of the user U acquired from the biological information acquisition unit 113. Specifically, the biological information analysis unit 1140 analyzes the acquired biological information and classifies the psychological stress felt by the user U into 10 stages from, for example, stress level 1 to stress level 10. The larger the stress level, the stronger the stress the user U feels.

The stop threshold storage unit 1141 stores the stop threshold referred to by the intensity adjusting unit 114 in order to determine whether or not to stop the presentation of the simulated experience to the user U. The stop threshold storage unit 1141 stores information indicating, for example, the stress level 8. This means that when the stress level of the user U analyzed by the biological information analysis unit 1140 is equal to or higher than the stop threshold value, the presentation of the simulated experience to the user U is stopped. The stop threshold is not limited to the stress level 8, and may be determined in consideration of the type of simulated experience, the questionnaire result of the user U, and the like.

The comparison unit 1142 compares the stress level of the user U analyzed by the biological information analysis unit 1140 with the stop threshold value stored in the stop threshold storage unit 1141. The comparison unit 1142 outputs the comparison result to the OR unit 1143. More specifically, the comparison unit 1142 outputs 1 when the stress level of the user U analyzed by the biological information analysis unit 1140 is equal to or higher than the stop threshold value, and outputs 0 otherwise.

The OR unit 1143 calculates the OR of the output of the stop switch 900 and the output of the comparison unit 1142 and outputs it to the presentation unit 112. Here, the stop switch 900 is configured to output 1 when pressed by the operator and 0 otherwise. Therefore, the OR 1143 sets 1 when the stop switch 900 is pressed or at least one of the cases where the stress level of the user U analyzed by the biological information analysis unit 1140 is equal to or higher than the stop threshold value. Output. Further, the OR unit 1143 outputs 0 when the stop switch 900 is not pressed and the stress level of the user U analyzed by the biological information analysis unit 1140 is less than the stop threshold value.

When the output of the OR unit 1143 is 1, the presentation unit 112 stops the simulated experience presented to the user U. In this sense, the signal 1 output by the OR unit 1143 functions as an emergency stop signal. In this way, the simulated experience providing device 100 stops the simulated experience presented to the user U by an artificial judgment by the operator or a mechanical judgment by analyzing the biological information of the user U. As a result, it is possible to suppress giving unnecessary psychological stress to the user U.

By continuing the cycle of the simulated experience presentation function F1, the biological information measurement function F2, and the stimulus intensity adjusting function F3, the simulated experience providing system 1 has a stimulus suitable for the user U according to the stimulus felt by the user U. The intensity can be adjusted in real time. As a result, the training effect of the user U by the simulated experience provided by the simulated experience providing system 1 can be enhanced. Further, when the stimulus of the simulated experience presented to the user U is too strong and the simulated experience should be stopped, the presentation of the simulated experience is stopped by the intensity adjusting unit 114, which is a psychological load on the user U. Can also be reduced.

<Processing flow of simulated experience providing processing executed by simulated experience providing system 1>
FIG. 9 is a flowchart for explaining the flow of the simulated experience providing process executed by the simulated experience providing system 1 according to the embodiment of the present invention. The process in this flowchart starts, for example, when the power of the simulated experience providing device 100 is turned on.

The setting unit 115 sets an initial value of the stimulus intensity of the simulated experience presented to the user U (S2). The selection unit 111 selects a scene that constitutes a simulated experience scenario to be presented to the user U based on the stimulus intensity (S4).

The presentation unit 112 presents the scene selected by the selection unit 111 to the user U (S6). While the scenario presented to the user U continues (No in S8), the biometric information acquisition unit 113 acquires the biometric information of the user U measured by the measuring device 300 (S10). The intensity adjusting unit 114 resets the stimulus intensity of the simulated experience based on the biological information of the user U acquired by the biological information acquisition unit 113 (S12). After that, the control unit 110 returns to step S4 and continues the above process.

When the scenario presented to the user U ends (Yes in S8), the process in this flowchart ends.

As described above, according to the simulated experience providing system 1 according to the embodiment, the training effect using VR can be stabilized.
In particular, the simulated experience providing system 1 changes the stimulus intensity of the simulated experience presented to the user U in real time based on the biological information of the user U who is presented with the simulated experience. A simulated experience can be presented to the user U.

Further, since the simulated experience providing system 1 sets the stimulus intensity of the user U based on the questionnaire answered by the user U before the start of the training, the user has a stimulus intensity suitable for the user U from the start of the training. A simulated experience can be presented to U.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various changes or improvements can be made to the above embodiments. It is clear from the description of the claims that such modified or improved forms may also be included in the technical scope of the present invention.

<First modification>
For example, in addition to the traffic safety training described above, the simulated experience provision system 1 includes structural understanding / function explanation, learning manual, facility maintenance / maintenance management, danger prediction prevention learning, response procedure learning, security training, skill improvement / response. It can be used for various training and learning such as training.

<Second modification>
In the above, the case where the user U of the simulated experience providing system 1 is one is mainly described. That is, when there is only one user U of the simulated experience providing system 1, the selection unit 111 has described the case of changing the order of the scenes presented to the user U based on the stimulus intensity set in the user U. ..

The simulated experience providing system 1 according to the second modification is assumed to be used by a plurality of users U, and simultaneously provides a simulated experience common to each user U. When a plurality of users U use the simulated experience providing system 1, the stimulus intensities set for each of the plurality of users U may differ from each other. Therefore, if the order of the scenes presented to the user U is changed according to the stimulus intensity of any one of the plurality of users U, a simulated experience common to each user U can be provided at the same time. Can't be done.

Therefore, when the user U of the simulated experience providing system 1 has a plurality of users U and simultaneously provides a simulated experience common to each user U, the selection unit 111 prohibits the change of the order of the scenes based on the stimulus intensity. As a result, one common scenario can be provided to a plurality of users U at the same time.

<Third modification example>
In the above, the case where the intensity adjusting unit 114 adjusts the stimulation intensity based on the biological information of the user U has been mainly described. In addition to this, the operator of the simulated experience providing system 1 observes the biological information and facial expressions of the user U displayed on the monitor 600, manually adjusts the stimulus intensity, and changes the order of the presented scenes. You may do it. This allows the operator of the simulated experience providing system 1 to operate the presentation unit 112 and the setting unit 115 via, for example, a user interface (not shown) such as a keyboard or a pointing device connected to the simulated experience providing device 100. Can be realized with.

1 ... Pseudo-experience providing system 100 ... Pseudo-experience providing device 110 ... Control unit 111 ... Selection unit 112 ... Presenting unit 113 ... Biological information acquisition unit 114 ... Strength adjusting unit 115 ... Setting unit 116 ... Database generation unit 120 ... Storage unit 121 ... Scenario storage unit 122 ... Stimulation intensity storage unit 123 ... Stimulation intensity database 200 ... Head mount display 250 ... -Hearing reproduction device 300 ... Measuring device 400 ... Attitude detection sensor 500 ... Camera 600 ... Monitor 700 ... Tactile reproduction device 800 ... Relay device 900 ... Stop switch 1140 ... Biometric information analysis unit 1141 ... Stop threshold storage unit 1142 ... Comparison unit 1143 ... Logical sum calculation unit

Claims (8)

It is a simulated experience providing device that presents a virtual scene to the user.
The results of a questionnaire created before the use of the simulated experience providing device by each of the plurality of users using the simulated experience providing device, the evaluation results regarding the simulated experience created after the use, and the simulated experience providing device during use. The stimulus intensity set for each of the plurality of users, based on the stimulus intensity indicating the intensity of the stimulus that the user receives from the virtual scene and affects the psychological state of the user. A database generation unit that generates a stimulus intensity database that stores the results of the questionnaire and the evaluation results in association with the stimulus intensity.
By referring to the stimulus intensity database set for the user of the simulated experience providing device based on the result of the questionnaire answered by the user before using the simulated experience providing device. Setting part to set and
A selection unit that selects a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity.
A presentation unit that presents the scene selected by the selection unit to the user, and a presentation unit.
A biometric information acquisition unit that acquires biometric information measured by the measuring device that measures the biometric information of the user, and a biometric information acquisition unit.
A stimulus intensity selection discriminator that inputs biometric information measured by a user and outputs the stimulus intensity that should be set for that user, and presents scenes at various stimulus intensities to multiple users. The stimulus intensity obtained by inputting the biological information acquired by the biological information acquisition unit into the stimulus intensity selection discriminator obtained by learning the measured biological information using a machine learning method. Strength adjustment unit to reset
A simulated experience providing device equipped with. When the intensity adjusting unit resets the stimulus intensity, the selection unit selects a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity after the resetting.
The simulated experience providing device according to claim 1. The selection unit further selects a flow that determines the order in which the scenes are presented based on the stimulation intensity set for the user of the simulated experience providing device.
The presenting unit presents the scene selected by the selecting unit to the user according to the flow.
The simulated experience providing device according to claim 1 or 2. The biological information acquisition unit acquires the blood flow rate of the user measured by the measuring device, and obtains the blood flow rate of the user.
The intensity adjusting unit changes the stimulation intensity set for the user based on the change in blood flow rate.
The simulated experience providing device according to any one of claims 1 to 3. The processor of the simulated experience providing device that presents a virtual scene to the user
The results of a questionnaire created before the use of the simulated experience providing device by each of the plurality of users using the simulated experience providing device, the evaluation results regarding the simulated experience created after the use, and the simulated experience providing device during use. The stimulus intensity set for each of the plurality of users, based on the stimulus intensity indicating the intensity of the stimulus that the user receives from the virtual scene and affects the psychological state of the user. A step of generating a stimulus intensity database that stores the result of the questionnaire and the relationship between the evaluation result and the stimulus intensity in association with each other.
By referring to the stimulus intensity database set for the user of the simulated experience providing device based on the result of the questionnaire answered by the user before using the simulated experience providing device. Steps to set and
A step of selecting a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity, and
A step of presenting the selected scene to the user, and
The step of acquiring the biological information measured by the measuring device that measures the biological information of the user, and
A stimulus intensity selection discriminator that inputs biometric information measured by a user and outputs the stimulus intensity that should be set for that user, and presents scenes at various stimulus intensities to multiple users. A step of resetting the stimulus intensity with the stimulus intensity obtained by inputting the acquired biological information into the stimulus intensity selection discriminator obtained by learning the measured biological information using a machine learning method. When,
How to provide a simulated experience to execute. To the computer of the simulated experience providing device that presents a virtual scene to the user
The results of a questionnaire created before the use of the simulated experience providing device by each of the plurality of users using the simulated experience providing device, the evaluation results regarding the simulated experience created after the use, and the simulated experience providing device during use. The stimulus intensity set for each of the plurality of users, based on the stimulus intensity indicating the intensity of the stimulus that the user receives from the virtual scene and affects the psychological state of the user. A function to generate a stimulus intensity database that stores the result of the questionnaire and the relationship between the evaluation result and the stimulus intensity in association with each other.
By referring to the stimulus intensity database set for the user of the simulated experience providing device based on the result of the questionnaire answered by the user before using the simulated experience providing device. Functions to set and
A function to select a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity, and
A function of presenting the selected scene to the user and
The function of acquiring the biological information measured by the measuring device that measures the biological information of the user, and
A stimulus intensity selection discriminator that inputs biometric information measured by a user and outputs the stimulus intensity that should be set for that user, and presents scenes at various stimulus intensities to multiple users. A function of resetting the stimulus intensity with the stimulus intensity obtained by inputting the acquired biological information into the stimulus intensity selection discriminator obtained by learning the measured biological information using a machine learning method. When,
A program that realizes. One or more simulated experience providing devices that present virtual scenes to one or more users, respectively.
One or more head-mounted displays to be attached to each of the one or more users, and
One or more measuring devices that measure the biological information of each of the one or more users, and
It is a simulated experience providing system including
The results of a questionnaire created before the use of the simulated experience providing device by each of the plurality of users using the simulated experience providing device, the evaluation results regarding the simulated experience created after the use, and the simulated experience providing device during use. The stimulus intensity set for each of the plurality of users, based on the stimulus intensity indicating the intensity of the stimulus that the user receives from the virtual scene and affects the psychological state of the user. A database generation unit that generates a stimulus intensity database that stores the results of the questionnaire and the evaluation results in association with the stimulus intensity.
By referring to the stimulus intensity database set for the user of the simulated experience providing device based on the result of the questionnaire answered by the user before using the simulated experience providing device. Setting part to set and
A selection unit that selects a scene that constitutes a simulated experience to be presented to the user based on the stimulus intensity.
A presentation unit that presents the scene selected by the selection unit to the user, and a presentation unit.
A biometric information acquisition unit that acquires biometric information measured by the measuring device that measures the biometric information of the user, and a biometric information acquisition unit.
A stimulus intensity selection discriminator that inputs biometric information measured by a user and outputs the stimulus intensity that should be set for that user, and presents scenes at various stimulus intensities to multiple users. The stimulus intensity obtained by inputting the biological information acquired by the biological information acquisition unit into the stimulus intensity selection discriminator obtained by learning the measured biological information using a machine learning method. It is equipped with a strength adjustment unit that resets the
Pseudo experience providing system. Each of the scenes constituting the simulated experience includes a plurality of expression forms corresponding to the stimulus intensity.
The selection unit selects the expression form of the scene to be presented to the user based on the stimulus intensity.
The simulated experience providing system according to claim 7.

Images