JP2018072950A - Simulation device and simulation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a simulation device and a simulation system capable of equalizing feelings of presence which users have.SOLUTION: The simulation device has a processing unit which executes processing of: receiving biometric data of a user viewing a video; estimating his or her sensibility to the video on the basis of the biometric data; and controlling the operation of a seating part which he or she sits on, on the basis of his or her sensibility. The simulation system also has: an immersive HMD including a measurement instrument for measuring the biometric data of the user viewing the video; and a controller which estimates his or her sensibility to the video on the basis of the biometric data and controls the operation of the seating part which the user sits on, on the basis of his or her sensibility estimated.SELECTED DRAWING: Figure 1

Description

  This case relates to a simulation apparatus and a simulation system.

  A bodily sensation generating system that gives a predetermined stimulus corresponding to a video to a user (hereinafter referred to as a user) in conjunction with a scene (hereinafter referred to as a video) projected on a monitor by a computer game or video software is known. (For example, refer to Patent Document 1).

JP 2002-239213 A

  However, depending on the sensibility that the user feels, some users feel the predetermined stimulus weak, and conversely, some feel strong. That is, even if a predetermined stimulus is given to the user, there is a problem that the sense of reality that the user feels varies depending on the stimulus and the video.

  Therefore, an object of one aspect is to provide a simulation apparatus and a simulation system that can homogenize the realistic sensation felt by a user.

  In one embodiment, the simulation apparatus receives biometric data of a user who views a video, estimates the user's sensitivity to the video based on the biometric data, and the user is seated based on the sensitivity. It is a simulation apparatus which has a process part which controls operation | movement of a seating part and performs a process.

  In one embodiment, the simulation system estimates an sensibility of the user with respect to the video based on an immersive HMD including a measurement device that measures biological data of the user viewing the video and the biometric data. And a control device that controls the operation of the seating portion on which the user is seated based on the estimated sensitivity.

  The sense of reality that the user feels can be homogenized.

FIG. 1 is an example of a simulation system. FIG. 2 is an example of an HMD. FIG. 3 shows an example of the hardware configuration of the HMD and the control device. FIG. 4 is an example of a functional block diagram of the HMD and the control device. FIG. 5A is an example of a viewing data storage unit. FIG. 5B is an example of the sensation data storage unit. FIG. 5C shows an example of the stimulus data storage unit. FIG. 6 is a flowchart showing an example of the operation of the HMD. FIG. 7 is a flowchart showing an example of the operation of the control device. FIG. 8 is an example of a flowchart showing a part of the operation of the control device.

  Hereinafter, an embodiment for carrying out this case will be described with reference to the drawings.

(First embodiment)
FIG. 1 shows an example of a simulation system S. FIG. 2 is an example of a head mounted display (HMD) 100. The simulation system S includes a HMD 100 and a simulation device (hereinafter referred to as a control device) 300 that controls the operation of the seating unit 200 on which the user 10 is seated. The seating surface 210 of the seating unit 200 and the control device 300 are connected via the link mechanism 20. Therefore, when the control device 300 drives the link mechanism 20 according to the image displayed on the HMD 100, the seating portion 200 including the seat surface 210 connected to the link mechanism 20 swings back and forth, left and right, and up and down. Or vibrate. Thereby, a predetermined stimulus is given to the user 10.

  The HMD 100 is attached to the head of the user 10. As shown in FIG. 2, the HMD 100 includes a display 100A and a speaker 100B. The display 100A displays an image, and the speaker 100B outputs sound. The HMD 100 is an immersive type that completely covers both eyes. For this reason, the realistic sensation obtained from the image of the display 100A is higher than that of the monocular type or the transmission type.

  Although details will be described later, the HMD 200 includes a heart rate sensor and a wireless communication unit (not shown). The heart rate sensor measures the heart rate of the user 10, and the wireless communication unit wirelessly transmits the heart rate measured by the heart rate sensor to the control device 300. Examples of wireless include radio waves and infrared rays. The control device 300 operates the seating unit 200 while adjusting the driving strength of the link mechanism 20 based on the heart rate.

  FIG. 3 shows an example of the hardware configuration of the HMD 100 and the control device 300.

  First, the HMD 100 will be described. The HMD 100 includes a display 100A, a speaker 100B, and a heart rate sensor 100C. The HMD 100 includes a central processing unit (CPU) 100D, a random access memory (RAM) 100E, a read only memory (ROM) 100F, and a radio frequency (RF) circuit 100G. An antenna 100G ′ is connected to the RF circuit 100G. A CPU that realizes a communication function may be used instead of the RF circuit 100G.

  The display 100A to the RF circuit 100G are connected to each other by an internal bus 100H. A computer is realized by cooperation of at least the CPU 100D and the RAM 100E. In the above-described RAM 100E, the program stored in the ROM 100F is stored by the CPU 100D. When the CPU 100D executes the stored program, various functions in the HMD 100 to be described later are realized, and various processes are executed. In addition, what is necessary is just to make a program according to the flowchart mentioned later.

  Next, the control device 300 will be described. The control device 300 includes a CPU 300A, a RAM 300B, a ROM 300C, an RF circuit 300D, and an actuator 300E. An antenna 300D ′ is connected to the RF circuit 300D. A CPU that realizes a communication function may be used instead of the RF circuit 300D. On the other hand, the link mechanism 20 is connected to the actuator 300E.

  The CPU 300A to the actuator 300E are connected to each other by an internal bus 300F. At least the CPU 300A and the RAM 300B cooperate to realize a computer. In the above-described RAM 300B, the program stored in the ROM 300C is stored by the CPU 300A. When the stored program is executed by the CPU 300A, various functions in the control device 300 described later are realized, and various processes are executed. In addition, what is necessary is just to make a program according to the flowchart mentioned later.

  Next, functions of the HMD 100 and the control device 300 will be described with reference to FIGS. 4 and 5.

  FIG. 4 is an example of a functional block diagram of the HMD 100 and the control device 300. FIG. 5A is an example of the viewing data storage unit 110. FIG. 5B is an example of the sensation data storage unit 310. FIG. 5C is an example of the stimulus data storage unit 320.

  First, the HMD 100 will be described. The HMD 100 includes a viewing data storage unit 110, a video display unit 120, an audio output unit 130, and a heart rate measurement unit 140. Note that the viewing data storage unit 110 is realized by the ROM 100F described above, for example. The video display unit 120 is realized by, for example, the display 100A, the CPU 100D, the RAM 100E, and the RF circuit 100G described above. The audio output unit 130 is realized by the speaker 100B described above, for example. The heart rate measurement unit 140 is realized by, for example, the above-described heart rate sensor 100C and the RF circuit 100G.

  The viewing data storage unit 110 stores viewing data. More specifically, as shown in FIG. 5A, the viewing data storage unit 110 stores viewing data (for example, a roller coaster) regarding attractions and viewing data (for example, a drive or a construction site) regarding safety education. The viewing data includes video data to be displayed on the video display unit 120 and audio data to be output from the audio output unit 130.

  The video display unit 120 displays video. More specifically, when the user 10 performs a predetermined selection operation on the HMD 100, the video display unit 120 specifies viewing data corresponding to the selection operation from the viewing data storage unit 110, and the video data included in the specified viewing data The video corresponding to the is displayed. When the video ends, the video display unit 120 wirelessly transmits a stop signal to the control device 300. The stop signal is a signal for stopping the operation of the control device 300. The stop signal reaches the control device 300 via the wireless network NW.

  The audio output unit 130 outputs audio. More specifically, when the user 10 performs the predetermined selection operation described above, the audio output unit 130 specifies the viewing data corresponding to the selection operation from the viewing data storage unit 110 and outputs the audio data included in the specified viewing data. To do.

  The heart rate measuring unit 140 measures the heart rate of the user 10. More specifically, when the video display unit 120 starts displaying an image, the heart rate measurement unit 140 starts measuring the heart rate and periodically transmits the heart rate to the control device 300 by radio. The heart rate reaches the control device 300 via the wireless network NW. In this embodiment, the heart rate is described as an example. However, the biological data of the user 10 is not limited to the heart rate, and for example, body temperature, blood pressure, myoelectricity, or the like may be used. In this case, a sensor corresponding to each is used. Body temperature, blood pressure, myoelectricity, etc. may be used in combination with heart rate. Thereby, the estimation precision of the sensitivity of the user 10 improves. For example, if the body temperature rises and the myoelectricity indicates high stiffness, it is estimated with high accuracy that the user 10 is sensitive to the stimulus.

  Next, the control device 300 will be described. The control device 300 includes a sensation data storage unit 310, a stimulus data storage unit 320, a data processing unit 330 as a processing unit, and a link mechanism driving unit 340. Note that the sensation data storage unit 310 and the stimulus data storage unit 320 are realized by the ROM 300C described above, for example. The data processing unit 330 is realized by, for example, the CPU 300A, the RAM 300B, and the RF circuit 300D described above. The link mechanism driving unit 340 is realized by the actuator 300E described above, for example.

  The sensation data storage unit 310 stores sensation data. More specifically, as shown in FIG. 5B, the sensation data storage unit 310 stores sensation data in which the heart rate and the sensation level are associated with each other. For example, the heart rate “rising” and the sensation level “sensitive” of the user 10 are associated with each other. That is, even if a slight stimulus is given to the user 10 by video and audio, if the user 10 beats quickly, the heart rate increases. In this case, it is estimated that the user 10 is sensitive to the stimulus. That is, it is estimated that the user 10 is likely to feel fear and excitement.

  The stimulus data storage unit 320 stores stimulus data. More specifically, as shown in FIG. 5C, the stimulus data storage unit 320 stores stimulus data in which the sensation level is associated with the stimulus. The stimulus represents the intensity of the stimulus given to the user 10. For example, when a stimulus with intensity “medium” is given to the user 10 with the sensation level “sensitive”, there is a high possibility that the user 10 feels the stimulation strongly. A weak stimulus is given. As a result, it is possible to provide the same level of realism as when a stimulus of “medium” intensity is given to the user 10 with the “normal” experience level.

  The data processing unit 330 receives the heart rate and stop signal transmitted from the HMD 100. When receiving the heart rate, the data processing unit 330 analyzes the heart rate variability from both the absolute value of the heart rate and the difference between the heart rates. For example, the data processing unit 330 calculates the rate of change in heart rate from the difference in heart rate and compares it with a preset threshold rate of change (for example, 100%). If the calculated change rate is greater than the threshold change rate, the data processing unit 330 determines that the heart rate has increased. Conversely, if the calculated rate of change is smaller than the threshold rate of change, it is determined that the heart rate has decreased. When the data processing unit 330 finishes analyzing the heart rate, the data processing unit 330 accesses the sensation data storage unit 310 and estimates the sensation level according to the analysis result. When the data processing unit 330 estimates the sensation level, the data processing unit 330 outputs the estimated sensation level to the link mechanism driving unit 340.

  The link mechanism drive unit 340 drives the link mechanism 20 according to the video displayed on the HMD 100. More specifically, when the link mechanism driving unit 340 receives the sensation level output from the data processing unit 330, the link mechanism driving unit 340 accesses the stimulation data storage unit 320, acquires the intensity of stimulation according to the sensation level, and sets it. As a result, the link mechanism drive unit 340 drives the link mechanism 20 according to the image and the intensity set for itself.

Next, operations of the HMD 100 and the control device 300 will be described with reference to FIGS. 6 and 7.
FIG. 6 is a flowchart showing an example of the operation of the HMD 100. FIG. 7 is a flowchart showing an example of the operation of the control device 300.

  First, as shown in FIG. 6, the video display unit 120 of the HMD 100 displays a video (step S101). More specifically, when a predetermined selection operation is performed from the user 10 to the HMD 100, the video display unit 120 specifies viewing data corresponding to the selection operation, and displays a video corresponding to the video data included in the specified viewing data. indicate. For example, when the user 10 performs a selection operation for selecting viewing data of a roller coaster, the video display unit 120 displays a video before the roller coaster starts to move.

  When the process of step S101 is completed, the heart rate measuring unit 140 then measures the heart rate of the user 10 (step S102). The heart rate measuring unit 140 may measure and use a waveform together with the heart rate. As described above, when the video before the roller coaster starts moving is displayed on the video display unit 120, it is assumed that the heart rate of the user 10 is stable.

  When the process of step S102 is completed, the video display unit 120 then updates the video (step S103). For example, the image display unit 120 updates the image before the roller coaster starts moving to an image in which the roller coaster starts moving and rises. As a result, some users 10 maintain a stable heart rate and others have an increased heart rate. On the other hand, some users 10 are relaxed and their heart rate decreases.

  When the process of step S103 is completed, the heart rate measuring unit 140 then transmits the heart rate to the control device 300 (step S104), and the video display unit 120 determines whether the video is finished (step S105). ). If the video has not ended (step S105: NO), the processes of steps S103 and S104 are repeated. Therefore, the video is periodically updated until the video ends, and the heart rate is transmitted. On the other hand, when the video is finished (step S105: YES), the video display unit 120 transmits a stop signal to the control device 300 (step S106). Although details will be described later, when the control device 300 receives the stop signal, the control device 300 stops its operation.

  Next, as shown in FIG. 7, the data processing unit 330 of the control device 300 receives the heart rate (step S201) and analyzes the heart rate (step S202). When the process of step S202 is completed, the data processing unit 330 then estimates the sensation level based on the analysis result (step S203). For example, when the data processing unit 330 determines that the heart rate has increased based on the analysis result, the data processing unit 330 accesses the sensation data storage unit 310 and estimates that the sensation level is sensitive. Conversely, when the data processing unit 330 determines that the heart rate has decreased based on the analysis result, the data processing unit 330 accesses the sensation data storage unit 310 and estimates that the sensation level is insensitive.

  When the process of step S203 is completed, the link mechanism driving unit 340 then sets a stimulus based on the sensation level estimated by the data processing unit 330 (step S204). More specifically, the link mechanism driving unit 340 accesses the stimulus data storage unit 320, specifies a stimulus according to the sensation level estimated by the data processing unit 330, and sets the specified stimulus to itself. For example, if the sensation level is sensitive, the link mechanism driving unit 340 sets the stimulus intensity “weak”. On the contrary, if the sensation level is insensitive, the link mechanism drive unit 340 sets the stimulus intensity “strong”.

  When the process of step S204 is completed, the link mechanism driving unit 340 changes the operation of the seating unit 200 via the link mechanism 20 (step S205). For example, if the user 10 is sensitive, the link mechanism driving unit 340 weakens the operation of the seating unit 200. Thereby, even if it is a slight stimulus for the user 10 who is sensitive to the sensation level, the sensation is the same as the sensation felt by the user 10 who is normal.

  When the process of step S205 is completed, the data processing unit 330 then determines whether or not a stop signal has been received (step S206). If the data processing unit 330 has not received a stop signal (step S206: NO), the processing from step S201 to S205 is repeated. Therefore, every time the data processing unit 330 receives a heart rate, the data processing unit 330 analyzes the heart rate, and estimates a sensation level based on the analysis result. Then, the link mechanism driving unit 340 sets the intensity of stimulation based on the sensation level estimated by the data processing unit 330, and changes the operation of the seating unit 200 based on the set intensity. On the other hand, when the data processing unit 330 receives the stop signal (step S206: YES), the data processing unit 330 ends the processing. Thereby, the control apparatus 300 stops operation | movement and the seating part 200 stops.

  As described above, according to the first embodiment, the control device 300 includes the data processing unit 330 and the link mechanism driving unit 340. The data processing unit 330 receives biometric data of the user 10 who views the video, and estimates the sensation level of the user 10 with respect to the video based on the received biometric data. Further, the link mechanism driving unit 340 controls the operation of the seating unit 200 on which the user 10 is seated based on the sensation level estimated by the data processing unit 330. Thereby, the realistic sensation felt by the user can be homogenized.

  As described above, although the user 10 has a different sensation level, high realistic sensation is reproduced by linking the image of the HMD 100 and the analysis of the heart rate and the control of the seating unit 200 by the control device 300. In particular, since the immersive HMD 100 is employed in the first embodiment, a higher immersive feeling with respect to virtual reality (VR) video than when the user 10 watches video displayed on a monitor or a screen. Is obtained.

(Second Embodiment)
Next, a second embodiment of the present case will be described with reference to FIG.
FIG. 8 is an example of a flowchart showing a part of the operation of the control device 300. As shown in FIG. 8, in the process of step S203, when the data processing unit 330 estimates the sensation level, the data processing unit 330 determines whether or not the sensation level is sensitive (step S301). If the sensation level is sensitive (step S301: YES), the data processing unit 330 transmits a change signal for changing the video to the HMD 100 (step S302).

  When the video display unit 120 of the HMD 100 receives the change signal, the video display unit 120 changes the video. For example, when receiving the change signal, the video display unit 120 skips or replaces the video. That is, when the stimulus given to the user 10 by the video is strong, the stimulus may be too strong for the user 10 whose sensitivity level is sensitive. Therefore, in such a case, an increase in the heart rate of the user 10 can be suppressed by changing the video. When the process of step S302 is completed, the data processing unit 330 executes the process of step S206. If the sensation level is not sensitive (step S301: NO), the link mechanism driving unit 340 executes the processes of steps S204 and S205. As described above, according to the second embodiment, it is possible to suppress an increase in the heart rate of the user 10 according to the sensation level while homogenizing the realistic sensation felt by the user.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific embodiments according to the present invention, and various modifications are possible within the scope of the gist of the present invention described in the claims.・ Change is possible. For example, in the first embodiment and the second embodiment described above, the heart rate sensor 100C is incorporated in the HMD 100, but the heart rate sensor 100C or the heart rate monitor may be separate from the HMD 100. Further, the heart rate sensor 100C or the heart rate monitor may be detachable from the HMD 100.

In addition, the following additional notes are disclosed regarding the above description.
(Appendix 1) Accepting biometric data of a user viewing a video, estimating the user's sensitivity to the video based on the biometric data, and controlling the operation of a seating portion on which the user is seated based on the sensitivity A simulation apparatus having a processing unit for executing processing.
(Additional remark 2) The said processing part receives the biometric data of the user who views the said image | video by immersive HMD, The simulation apparatus of Additional remark 1 characterized by the above-mentioned.
(Supplementary note 3) The simulation apparatus according to supplementary note 2, wherein the HMD includes a measurement device that measures the biological data, and the processing unit receives the biological data from the measurement device.
(Additional remark 4) The said process part compares the 1st change rate of the said biometric data with the 2nd preset change rate, and changes the operation | movement of the said seating part according to a comparison result. 4. The simulation apparatus according to any one of supplementary notes 1 to 3, which is a feature.
(Additional remark 5) The said process part compares the 1st change rate of the said biometric data with the preset 2nd change rate, and changes the image | video which the said user views according to a comparison result, Additional remarks The simulation apparatus according to any one of 1 to 4.
(Supplementary Note 6) Based on the immersive HMD including a measuring device that measures the biometric data of the user viewing the video and the biometric data, the user's sensitivity to the video is estimated, and based on the estimated sensitivity And a control device for controlling the operation of the seating portion on which the user is seated.
(Additional remark 7) The said control apparatus compares the 1st change rate of the said biometric data with the 2nd change rate set beforehand, and changes the operation | movement of the said seating part according to a comparison result. The simulation system according to appendix 6, which is characterized.
(Additional remark 8) The said control apparatus compares the 1st change rate of the said biometric data with the 2nd predetermined change rate, and changes the image | video which the said user views according to a comparison result, Additional remarks 8. The simulation system according to 6 or 7.

S simulation system 100 HMD
DESCRIPTION OF SYMBOLS 110 Viewing data memory | storage part 120 Image | video display part 130 Audio | voice output part 140 Heart rate measurement part 200 Seating part 300 Control apparatus 310 Sensory data memory | storage part 320 Stimulus data memory | storage part 330 Data processing part 340 Link mechanism drive part

Claims (6)

Accept biometric data of users who watch videos,
Based on the biometric data, estimate the user's sensitivity to the video,
Based on the sensibility, control the operation of the seating portion on which the user is seated,
A simulation apparatus having a processing unit for executing processing. The processing unit receives biometric data of a user who views the video with an immersive HMD.
The simulation apparatus according to claim 1. The HMD includes a measuring device that measures the biological data,
The processing unit receives the biological data from the measuring device.
The simulation apparatus according to claim 2. The processing unit compares a first change rate of the biological data with a preset second change rate, and changes the operation of the seating unit according to a comparison result.
The simulation apparatus according to any one of claims 1 to 3, wherein The processing unit compares a first change rate of the biological data with a preset second change rate, and changes a video to be viewed by the user according to a comparison result.
The simulation apparatus according to any one of claims 1 to 4. An immersive HMD including a measuring device for measuring biometric data of a user viewing the video;
A control device that estimates the sensibility of the user with respect to the video based on the biometric data, and controls the operation of a seating unit on which the user is seated based on the estimated sensibility;
A simulation system.

Images