JP2021081864A - Display program, display method, head mount display and information processor - Google Patents

Abstract

To achieve improvement of physical and cognitive functions without limiting physical activities of a user.SOLUTION: A computer executes a virtual motion display processing E repeating a predetermined number of the following processing: first display processing for displaying a viewpoint image of a virtual body of a user when the virtual body of the user is exercising at a first exercise intensity in a virtual reality space on a head mount display worn by the user for a first time period; and second display processing for displaying a viewpoint image of the virtual body of the user when the virtual body of the user is exercising at a second exercise intensity that is smaller than the first exercise intensity in the virtual reality space on the head mount display worn by the user for a second time period.SELECTED DRAWING: Figure 19

Description

The present invention relates to a display program, a display method, a head-mounted display, and an information processing device.

A certain level of exercise is one of the key factors in general health and is known to delay age-related disorders.

Aerobic exercise is also known to have beneficial consequences not only for physical function but also for cognitive function.

Kujach S, Byun K, Hyodo K, Suwabe K, Fukuie T, Laskowski R, et al. A transferable high-intensity intermittent exercise improves executive performance in association with dorsolateral prefrontal activation in young adults. Neuroimage. 2018

However, constant physical exercise or short-term high-impact physical exercise may be difficult for, for example, non-exercising people, busy people, or elderly people in a frail state, for example. , Patients with heart disease, patients who have recovered after long-term illness, or patients with movement disorders, etc., may not be able to perform this procedure.

The lifestyles of these people with limited physical activity adversely affect the improvement of physical function and may limit or contribute to the deterioration of cognitive function.

In one aspect, the present invention aims to realize improvement of physical function and cognitive function regardless of the limitation of physical activity of the user.

It should be noted that the other object of the present invention is not limited to the above-mentioned object, but is an action and effect derived by each configuration shown in the embodiment for carrying out the invention described later, and exerts an action and effect which cannot be obtained by the conventional technique. It can be positioned as one of.

In one aspect, the display program of the present invention displays the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity in the virtual reality space over a first time period. The first display process to be displayed on the head-mounted display worn by the user, and when the virtual body of the user exercises at a second exercise intensity smaller than the first exercise intensity in the virtual reality space. A computer is made to execute a virtual motion display process of repeating a second display process of displaying the viewpoint image of the virtual body on the head-mounted display worn by the user for a second time period and a predetermined number of times.

In one aspect, improvement of physical function and cognitive function can be realized regardless of the limitation of the physical activity of the user.

It is a figure which shows an example of HMD (Head Mounted Display) which concerns on one Embodiment. It is a figure explaining the procedure example of a crossover randomized controlled trial. It is a figure which shows an example of the Stroop task. It is a figure which shows an example of the online questionnaire about SoBO (Sense of Body Ownership) and SoA (Sense of Agency). It is a figure which shows the display example of the display when the avatar runs in 1PP (First Person Perspective). It is a figure which shows the display example of the display when the avatar walks in 1PP. It is a figure which shows the display example of the scenery in a virtual space in 1PP. It is a figure which shows an example of the offline questionnaire about SoBO and SoA. It is a figure which shows an example of the heart rate measurement result obtained in this test. It is a figure which shows an example of the online and offline questionnaire results obtained in this test. It is a figure which shows an example of the analysis result of the Stroop task obtained in this test. It is a figure which shows an example of the change of O2Hb which is associated with the Stroop interference in lDLPFC (left DorsoLateral PreFrontal Cortex: left dorsolateral prefrontal cortex) obtained by this test. It is a figure which shows an example of the change of O2Hb which is associated with the Stroop interference in lDLPFC obtained by this test. It is a figure which shows an example of the analysis result of the TDMS (Two Dimension Mood Scale) data obtained by this test. It is a figure which shows the configuration example of the display system as an example of one Embodiment. It is a block diagram which shows the hardware configuration example of a computer. It is a block diagram which shows the functional structure example of HMD and PC (Personal Computer). It is a sequence diagram which shows the operation example of the display system which concerns on one Embodiment. It is a figure explaining an example of the session which concerns on 1st Example. It is a block diagram which shows the functional structure example of the PC of the display system which concerns on 1st Example. It is a flowchart which shows the operation example of the PC which concerns on 1st Example. It is a block diagram which shows the functional structure example of the PC of the display system which concerns on 2nd Example. It is a figure which shows an example of a parameter information. It is a block diagram which shows the functional structure example of the HMD and PC of the display system which concerns on other examples.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and there is no intention of excluding the application of various modifications and techniques not specified below. For example, the present embodiment can be variously modified and implemented without departing from the spirit of the present embodiment. In the drawings used in the following embodiments, the parts having the same reference numerals represent the same or similar parts unless otherwise specified.

[1] One Embodiment [1-1] Effect of physical exercise in virtual reality on physical function and cognitive function In a situation where physical activity is limited, the physical reaction and the cognitive reaction become equivalent to the actual reaction. Avoiding the actual performance of the exercise by creating conditions can be a potential solution.

As an effective technology for that purpose, virtual reality (VR) that makes it possible to display a virtual world can be considered. In particular, in one embodiment, it is possible to display a realistic and highly relevant Virtual Body (which may be referred to as a "virtual body" or "avatar") as an example of VR. Focus on Immersive VR (IVR).

For example, a VR virtual environment can be realized by attaching an HMD (Head Mounted Display) 1 as shown in FIG. 1 to the user's head. Reference numeral A in FIG. 1 indicates the front surface (front surface) of the HMD 1, and reference numeral B indicates the back surface (mounting side) of the HMD 1. HMD1 may be referred to as "VR goggles", "VR visor", or "3D headset".

As shown in FIG. 1, the HMD 1 includes two displays 1a (see reference numerals B) and cables 1b (see reference numerals A and B) facing each other at the positions of both eyes of the user when worn.

The two displays 1a display images according to the left and right viewpoints, respectively. For example, in displaying the virtual environment of IVR, an image (viewpoint image) from the viewpoint of a human body, for example, an avatar that imitates the body of a user is displayed on the display 1a.

The cable 1b is a communication line for transmitting a data signal such as a video signal, and is connected to a computer (not shown) that outputs a video signal.

Further, the HMD1 has one of a head tracking function that detects the angle and movement of the user's head by a sensor such as an angle sensor and an eye tracking function that detects the movement of the eyeball (line of sight) by a camera or the like. Both tracking functions may be provided. The movement detected by the tracking function is fed back to the virtual environment, and the viewpoint image linked to the movement of the user's head and line of sight is displayed on the display 1a.

By giving the user a visual stimulus different from that that occurs in other multisensory illusions, such HMD1 can remind the user of a strong feeling of "being there".

The simple display of the avatar in the first person perspective (1PP) not only creates the illusion of Sense of Body Ownership (SoBO) on the avatar, but the movement of the avatar in 1PP also moves on the avatar. Creates the illusion of a sense of agency (SoA).

Actions performed by one's own avatar in a virtual environment are subjectively perceived by the user as what they have done, and in the actual user's body, subjectively similar to what happens in the physical world. , Behavioral and physiological reactions occur.

For example, if the movement of one's own avatar does not match the actual movement, the SoBO on the avatar makes an unconscious adjustment of the actual performance.

However, even in the absence of actual movement, if the avatar moves while the actual body is completely stationary, the illusion causes a measurable response that is perfectly comparable to the ongoing response of the actual movement. be able to. For example, when the avatar displayed in 1PP climbs a hill, the heart rate (HR) increases in the actual body. This is exactly what happens in the real world, but in this case it can be said to be a physiological response to the visual illusion of SoBO.

From the above, IVR can provide an effective research method for better understanding the relationship between SoBO and SoA, and can provide a potential solution to the clinical population to complement the existing one.

Through the following experiment (crossover randomized controlled trial), the inventor of the present application has shown that the illusion shows beneficial results not only for the body but also for executive functions and its neural substrate by this IVR. discovered.

<Randomized controlled trials of crossovers>
<< Overview >>
In this experiment, the effectiveness of physical high-intensity intermittent aerobic exercise (HIE), which is one of the aerobic exercises known to show improvement in cognitive executive function, Combines with the possibility of manipulating SoBO and SoA provided by IVR.

In this experiment, 30 healthy young adults were used as subjects, and a virtual HIE-based intervention (Virtual HIE: vHIE) performed exclusively by the avatar while the subject's body was completely stationary was actually performed. We have demonstrated whether it has acute cognitive and physically beneficial effects on the body compared to what occurs after actual physical training.

In vHIE, the avatar repeatedly performs a predetermined sequence of aerobic exercises a predetermined number of times (a predetermined number of sets) while the subject wearing the HMD1 is completely stationary. In this test, the predetermined sequence is a set of 30 seconds of running (running) and 30 seconds of slow walking (walking), and the predetermined number of times is 8. In this test, Facebook (registered trademark) Oculus Rift (registered trademark) was used as HMD1.

In this study, subjects were allowed to experience vHIE training with avatars by the following two patterns of manipulating visual viewpoints known to induce or inhibit SoBO and SoA on avatars.

In the first pattern, the avatar is displayed at 1PP. That is, the subject is made to experience the vHIE training in the IVR described above. On the other hand, in the second pattern, the avatar is displayed in the third person perspective (3PP) for comparison with 1PP.

To confirm that a 1PP subject had SoBO and SoA on the avatar, the inventor recorded the subject's heart rate in addition to a subjective questionnaire to the subject.

In addition, the inventor performs a color word matching stroop task (hereinafter, simply referred to as "stroop task") before and after each IVR session (1PP), and during the execution of the stroop task, functional near-red Cortical hemodynamic changes in the prefrontal cortex (PFC) of the subject were measured using a functional near-infrared spectroscopy (fNIRS) device.

From a theoretical point of view, the above test was able to prove that SoBO and SoA can regulate physical and cognitive parameters without actual movement. Also, from a clinical point of view, these results have been found to be useful for training cognitive and physical functions at the same time in a completely safe environment.

<< Procedure >>
FIG. 2 is a diagram illustrating an example of a test procedure. FIG. 2 shows the protocol of one experimental session with a timeline. As shown in FIG. 2, the study includes three phases: a pre-phase (BEFORE), a VR training phase (INTERVENTIONS), and a post-phase (AFTER). The protocol of the experimental session shown in FIG. 2 has the same procedure, but the intervention conditions (patterns) are changed, for example, the pattern of the VR training phase is switched between 1PP and 3PP at weekly intervals. Will be implemented.

In the pre-intervention phase, pre-intervention measurements (TDMS (Two Dimension Mood Scale), Stroop task and fNIRS recording) are performed. In the VR training phase, vHIE intervention is performed. In vHIE, the moving (running) speed of the avatar changes from slow walking to fast running every 30 seconds. Have the subject complete an online questionnaire about SoBO and SoA at a given point in time (timing) of the speed change. In the post-phase, post-vHIE intervention measurements are performed in a manner similar to the pre-phase. The details of each phase will be described below.

<<< Pre-Phase and Post-Phase >>
・ TDMS
TDMS is an effective scale for recording changes in psychological mood state, and is a psychometric scale using eight mood expression items that express mood states of pleasure and arousal in a complex manner. In TDMS, when a subject answers a question, the psychological state at the time of measurement can be quantified, and "activity", "stability" and the like can be measured.

In reference numerals A and G in FIG. 2, TDMS is executed immediately before the stroop task before the VR training phase and immediately before the stroop task after the VR training phase for each session (1PP and 3PP), thereby performing the VR training phase. Detect changes in the subject's psychological state (mood) before and after the experience. For example, the subject is asked to evaluate the current psychological state (at the time point A and the time point G in FIG. 2) using a 6-step Likert scale.

-Stroop task and fNIRS
Reference numerals B and H of FIG. 2 cause the subject to perform a color word matching Stroop task to measure the effectiveness of VR training for a period of 6 minutes and 30 seconds. Also, during the performance of the Stroop task, fNIRS devices are used to record cortical hemodynamic changes in the subject's PFC before and after each IVR session (either 1PP or 3PP).

The Stroop task is an example of a subject's cognitive function test, 10 times "neutral" (see reference numeral A), 10 times "match" (see reference numeral B), and 10 times, as illustrated in FIG. Consists of 30 trials, including the "mismatch" (see sign C), presented in random order. Two words are displayed above and below for all attempts.

In the case of a "neutral" trial, red, white, blue, pink or yellow letters are displayed in the upper row, and black letters such as "red", "white", "blue", "pink" or "yellow" are displayed in the lower row. Words are displayed. In the case of a "match" trial, the words "red", "white", "blue", "pink" or "kiiro" are displayed in the matching color in the upper row (for example, "red" is displayed in red. ), Words of the same color are displayed in black at the bottom. In the case of "mismatch" enforcement, the upper word is displayed in a dissonant color (for example, "red" is displayed in yellow) and cognitive interference between the color word and the color name (ie, Stroop interference). Causes. In FIG. 3, the colored characters in the upper row are shown in bold.

The Stroop task is displayed on the monitor of a computer such as a PC (Personal Computer). The lower row is presented 100 ms later than the upper row for continuous visual attention. Between each trial, a Fixation Cross with an Inter-Stimulus Interval of 9 to 13 seconds is displayed to avoid timing prediction. Regardless of the subject's response, the stimulus remains on the screen for 2 seconds.

Whether the color of the upper word (or letter) corresponds to the color name of the lower word by having the subject press button 1 "yes" or button 2 "no" on the keypad with the index finger and give an answer. To train to decide. Fifty percent of the stimuli presented were correct (the correct answer was "yes"). The inventor recorded the response time (RT) and the error ratio (Error Ratio) as variables.

fNIRS is an example of a device for measuring brain function, more specifically for monitoring cortical hemodynamics. In this test, wearable optical topography (registered trademark) WOT-HS manufactured by NeU Co., Ltd. is used as fNIRS. The fNIRS headset sends a detection signal to the computer via the control box.

The system comprises 35 capsules located at a distance of 3 cm from each other, in which a microprocessor, near-infrared radiation or sensitive receiving sensor is packaged. Subjects place the device on their forehead at FpZ (10% of the distance between Nasion and Inion), centered on a specific mark on the bottom of the probe, according to the International 10-20 System.

This device applies near-infrared light (850 nm, 730 nm) of two short-range wavelengths to monitor changes in cortical hemodynamics in PFC during stroop task performance, resulting in mmol-millimeters (mM · mm). The concentration of oxygenated hemoglobin (O2Hb), deoxygenated hemoglobin (HHb) and total hemoglobin calculated in units is detected.

<<< VR training phase >>>
In both the first (1PP) and second (3PP) patterns, the subject comfortably sat in a chair with both feet on the ground and arms relaxed toward the body, and wore the HMD1.

The virtual environment, along with avatars and animations, will be modeled in 3D Studio Max® 2015 and implemented in Unity® 3D. The implemented virtual environment represents an open space with a green floor (simulating a meadow) and a naturally illuminated sky, with a horizon displayed. The subject observes a gender-matched life-sized humanoid virtual body.

The first pattern consists of displaying the avatar in 1PP. The avatar replaces the virtual body and spatially matches the real body, that is, the subject directs his or her point of view to his or her body to observe the avatar.

The second pattern consists of displaying the same avatar in 3PP. The avatar is placed near 1.5 meters to the left of the actual subject's body, otherwise the subject rotates his head and points his point of view to the left side of the avatar to observe the avatar.

-Baseline As shown by reference numeral C in FIG. 2, the VR training phase (intervention) is referred to as a 4-minute proficiency phase (hereinafter referred to as "baseline") in both the first and second patterns (conditions). ). In this phase, avatars whose gender matches the subject are displayed, but there is no animation yet (that is, the 1PP or 3PP avatar is stationary).

At baseline, an online survey on SoBO and SoA (see Figure 4) to avoid or (within 1PP) the illusion of ownership on the avatar (within 3PP), heart rate. The baseline recording of the illness and the confirmation of the final disease problem due to the virtual display are performed.

(Online Questionnaire)
FIG. 4 is a diagram showing an example of an online questionnaire regarding SoBO and SoA. In an online questionnaire, subjects are asked to verbally assess the level of consent for four questions about the subject's subjective emotions during the intervention. The level of consent may be, for example, a 7-step (1-7) Likert scale in which "1" means "totally disagree" and "7" means "totally agree".

As shown in FIG. 4, the online questionnaire may include questions s1 and s2 related to SoBO (Ownership) and questions s3 and s4 related to SoA (Agency). Questions s1 and s3 are "real statements" that check the actual existence of the illusion, and questions s2 and s4 are "control statements."

The online questionnaire was conducted at multiple (eg, 5) time points (eg, 5) in the VR training phase to control the final change in subjective emotions during the intervention at different time points for the subject's subjective experience with SoBO and SoA. Timing). One of these timings is within the baseline period as described above. For example, an online survey will be conducted 2 minutes after the start of the VR training phase (baseline) in the case of an IVR session.

The remaining (4 times) timings are all within the vHIE period. The timing of conducting the online questionnaire during the vHIE period will be described later.

(Heart rate)
During the VR training phase (intervention), a heart rate monitor records the subject's heart rate (HR) to see if the subject actually recognizes the avatar as himself (1PP). In this test, a Polar H10 heart rate monitor is used as the heart rate monitor. The H10 heart rate monitor is controlled by a specific application installed on the computer via Bluetooth®. The subject wears an elastic strip with a heart rate monitor attached around the chest and positions the heart rate monitor closer to the heart.

Heart rate measurements are performed during the VR training phase (eg, for the entire period). That is, the heart rate monitor shows the baseline (4 minutes) heart rate under each intervention condition (1PP and 3PP), the total 8 minutes (4 minutes running + 4 minutes slow walking) heart rate during the intervention, and the post-intervention. Record the heart rate for 30 seconds (with the subject wearing HMD1 and with his eyes closed). The heart rate measured by the heart rate monitor is also used after the intervention to confirm the final delay in heart rate (decrease near HR at baseline) and to wait for heart rate synchronization.

-Speed test The speed test shown by reference numeral D in FIG. 2 is a phase for optimizing the running speed of the avatar. After baseline, display 1a shows a plurality (eg, 4) different options on the avatar to select the speed of the avatar's running animation in vHIE. These options may have multiple running animations with different speeds from each other. Subjects verbally report which option is subjectively perceived as "quite fast but feasible (quite fast but not infeasible)".

This is to maximize the possibility of actual bioactivity, and at the same time to display a subjectively valid running animation so as not to break the illusion of possession in 1PP. The slow walking animation may be the same for all subjects.

・ VHIE (1PP or 3PP)
As shown by reference numeral E in FIG. 2, after the speed test, an actual intervention, that is, a 1PP or 3PP vHIE exclusively performed by the avatar is performed. The subject sits down, keeps an eye on the avatar, and listens to the images of the avatar running for 30 seconds and the slow walking for 30 seconds alternately for a total of 8 minutes. ..

5 and 6 are diagrams showing a display example of the display 1a in the case of 1PP, and show a display example when the avatar runs and walks, respectively. In addition, since FIG. 5 and FIG. 6 show a display example in which the subject wearing HMD1 faces downward (down), a part of the hands and feet of the avatar in vHIE is displayed. .. When the subject wearing the HMD1 faces the front while in the sitting posture, the display 1a may display, for example, a landscape (landscape) in the virtual space as shown in FIG. 7.

After viewing the vHIE for 8 minutes, the subject is asked to keep his eyes closed for 30 seconds, and the vHIE period (see reference numeral E in FIG. 2) ends 8 minutes and 30 seconds after the start of the vHIE.

During the period of 8 minutes and 30 seconds of vHIE (including the last 30 seconds), the heart rate is measured and recorded by the heart rate monitor.

Also, as mentioned above, online questionnaires are conducted at multiple time points during the vHIE period. For example, an online questionnaire is conducted at the following four time points, as shown by reference numeral E'in FIG.

1 minute after the start of vHIE (during running).
2 minutes and 30 seconds after the start of vHIE (during walking).
5 minutes after the start of vHIE (during running).
6 minutes and 30 seconds after the start of vHIE (during walking).

・ Offline Questionnaire
As shown by reference numeral F in FIG. 2, after completion of vHIE (in the case of both 1PP and 3PP), the subject completes an offline questionnaire regarding SoBO and SoA. The offline questionnaire already contains detailed questions about vHIE motion sensation, motor control and physical effort, and subjects self-filled the offline questionnaire regarding subjective emotions during the previous vHIE intervention. To do.

FIG. 8 is a diagram showing an example of an offline questionnaire regarding SoBO and SoA. The second column of the offline survey is a list of underlying explored domains, not shown to the subject. Subjects assess the level of consent to the description in the third column. The level of consent may be, for example, a 7-step (1-7) Likert scale in which "1" means "totally disagreement" and "7" means "totally agreeable".

<< Statistical analysis >>
The information obtained from the experimental session shown in FIG. 2 was analyzed to clarify that vHIE intervention performed on one's avatar had a beneficial effect on cognitive executive function.

Current crossover randomized controlled trials are designed to determine if vHIE has a beneficial effect on executive function, but this determination is the embodiment of the avatar in 1PP during the intervention itself. It is possible if some assumptions such as are met. Therefore, data analysis is divided into the following two phases.

The first phase reviews the results collected in the intervention itself (ie, heart rate, online and offline surveys) to see if the vHIE intervention is actually effective. In the second phase, measurements are analyzed to check hypotheses (ie, Stroop tasks and fNIRS data) and their correlations.

<<< First Phase >>>
-Heart rate data For heart rate, data for three recording periods (baseline, running and slow walking) are averaged for 4 minutes each. The average baseline (HRb) result is subtracted from the average running (HBr) and average walking (HRw) results, as in equations (1) and (2) below.

dHRr = (HRr --HRb) (1)
dHRw = (HRw --HRb) (2)

Next, a two-step HR rate (dHRr means "fast" and dHRw means "slow") and a two-step state (CONDITION) corresponding to the intervention condition (1PP, 3PP) are obtained, respectively. Perform an analysis of variance (ANOVA 2x2) as a factor. The inventor predicted to find a significant increase in dHRr under 1PP conditions for all other measurements to confirm the physiological response of avatar embodied in 1PP alone.

FIG. 9 is a diagram showing an example of the heart rate measurement result obtained in this test. Regarding the objective heart rate measurement results in the VR training phase, there is a significant difference between 1PP and 3PP when the heart rate and vHIE over 8 minutes and 30 seconds are taken into consideration. In addition, the graph of 1PP shows that the subject has a high peak as in the case of actually running and walking.

-Online and offline questionnaire data As a subjective response, analyze the questionnaire data. The online survey was repeated at five different time points (baseline, vHIE 1 minute, 2 minutes 30 seconds, 5 minutes, 6 minutes 30 seconds). In order to exclude the time effect, first, analysis of variance (ANOVA 5x4), time (TIME) (corresponding to the time when the online questionnaire is conducted) and question (QUESTION) (corresponding to the four descriptions of the online questionnaire). Compare factors.

Eventually, the mean of the descriptions crossed the point in time, and in ANOVA 4x2, four descriptions (specifically, the result of the "actual description" and the result of the "contrast description") and the state (CONDITION). Can be compared with (1PP, 3PP).

The inventor predicted that he would find a higher level of physical ownership and motor subjectivity with respect to the actual and contrasting descriptions in 1PP, as well as the description regarding 3PP.

In addition, for the offline questionnaire (conducted at the end of each IVR condition), the data comparing the descriptions between the two conditions (1PP and 3PP) will be analyzed.

FIG. 10 is a diagram showing an example of online and offline questionnaire results obtained in this test. As shown in FIG. 10, in the online questionnaire conducted in the VR training phase, the subject has a sense of physical possession and exercise subjectivity of the avatar in 1PP, but this tendency is not seen in 3PP. In addition, the offline questionnaire conducted immediately after the VR training phase has the same pattern as the online questionnaire, and the subjects experienced the sensation of where the avatar is located, walking sensation, and motor control sensation in 1PP. There is. On the other hand, this tendency is not seen in 3PP.

Therefore, it can be concluded that the subject subjectively perceived the 1PP avatar as himself. Also, by combining the heart rate measurement results and the questionnaire results, it can be assumed that the subject effectively experienced the illusion of 1PP instead of 3PP.

<<< Second Phase >>>
The second phase of data analysis concerns the data collected before and after all IVR interventions, which means the results to confirm the main hypothesis (ie, the effectiveness of IVR training on cognition).

・ RT and ER of Stroop task
As described above, in the Stroop task, two measured values of RT (Response Time) and ER (Error Rate) are obtained. Calculate the discrepancy / neutral difference (contrast) that is assumed to represent Stroop interference. Both RT and ER are analyzed by repeated measures analysis of variance using TIME (before, after) and CONDITION (1PP, 3PP) as intra-subject factors.

As a supplementary analysis, only the Stroop task before vHIE is considered (predicted no difference) and the final difference between the two sessions is checked. According to the inventor's hypothesis, it is predicted that RT after vHIE is shorter and ER is lower in 1PP than in 3PP.

FIG. 11 is a diagram showing an example of the analysis result of the Stroop task obtained in this test. As shown in FIG. 11, from the analysis results of the Stroop task, we found a significant decrease in reaction time (RT) after vHIE at 1PP instead of 3PP. This means that the reaction time of the subject after vHIE at 1PP was increased. There was no significant difference in error rate (ER) between 1PP and 3PP. Actually, the error rate improved after 1PP and 3PP, which is considered to be due to the learning effect.

-FNIRS data The optical data of fNIRS is analyzed based on the modified Beer-Lambert's law. The sampling rate is set to 10 Hz, and the difference between the O2Hb and the Hb signal is analyzed.

To identify the hemodynamic brain response between O2Hb and Hb, a general linear model is adopted with reference to experimental factors. If desired, there is little need to combine adjacent channels to create a Region of Interest (ROI) according to the LBPA40 anatomical labeling system.

Next, changes in O2Hb and Hb concentrations in each channel are processed according to the following steps.

(I) Exclude cutaneous blood flow (ie, heartbeat) from raw data by using the specific software provided.

(Ii) Each channel is preprocessed using a 0.01-0.5 Hz bandpass filter to account for the effects of Mayer waves, high frequency fluctuations and baseline drift.

(Iii) A moving average for 3 seconds is performed to smooth the raw O2Hb concentration.

(Iv) The Stroop task picks up changes associated with each condition of O2Hb concentration from each channel (or combined ROI) because it is used in an event-related design that presents each neutral, matching, and inconsistent condition in a random order. .. In particular, the average value of the amount of change in the O2Hb concentration 2 seconds before the start of the task is defined as the "baseline", and 10 seconds after the start of the interval between stimuli is defined as the "vascular reaction", and is calculated under each Stroop task condition. This is because the NIRS signal is delayed with respect to the subject's response.

(V) The average value of the O2Hb concentration of each channel is converted into a normalized value using a linear conversion so that the average ± standard deviation of the O2Hb concentration in 2 seconds of the baseline period becomes 0 ± 1 (AU). To do. This method appears to be useful because it avoids the effects of different pathway length factors between subjects and between cortical regions.

(Vi) Finally, the channels on the target ROI region are finally averaged for each Stroop task condition.

12 and 13 are diagrams showing an example of changes in O2Hb associated with Stroop interference in the left Dorsolateral Prefrontal Cortex (lDLPFC) obtained in this test. As shown in FIG. 12, higher activation of lDLPFC is observed after vHIE at 1PP. Compared with before vHIE (pre), the O2Hb signal is about 17 times higher after vHIE (post). This is the target area for executive function. The target area corresponds to a significant increase in cortical activation of lDLPFC observed after the subject's actual HIE session in the physical world. Also, as shown in the graph of FIG. 13, it is observed that there is a higher peak activation at 1PP relative to 3PP a few seconds after the stimulus appears.

-TDMS data Calculate the level of pleasure and arousal from the TDMS score. First, make sure that the sample represents a normal population, that is, shows a normal distribution (all p-values for pleasure and arousal levels must be greater than or equal to 0.05 on the Shapiro-Wilk test under any intervention condition. ).

Similarly, check that the levels of pleasure and arousal do not differ between sessions so that all subjects start at the same baseline level (compare pre-PP and pre-3PP results). In other words, it is possible to proceed by subtracting the post-intervention result from the pre-intervention result (after-before).

By performing ANOVA comparing 1PP and 3PP, it is predicted that an increase in arousal level (not necessarily pleasure level) will be observed after vHIE, especially after intervention with 1PP.

FIG. 14 is a diagram showing an example of the analysis result of the TDMS data obtained in this test. As shown in FIG. 14, an increase in pleasure and arousal levels is observed.

<< Correlation >>
The decisive correlation to the inventor's hypothesis concerns the relationship between the Stroop task and fNIRS data. As a baseline criterion, the inventor first investigates the cortical regions that were predominantly activated during the pre-vHIE Stroop task in both intervention conditions. Next, the inconsistency / neutral contrast between the two intervention conditions is averaged as a substrate for ROI analysis. RT and ER discrepancies / neutral differences with O2Hb changes in all ROIs are treated with repeated measures ANOVA with intervention conditions (1PP and 3PP) and time (before and after vHIE) as factors.

In addition to this, the relationship between the Stroop task data and the results of cortical activation or TDMS is examined by a binomial formula. For each variable, the following differences are calculated. Both values are subjected to a McNemar test to determine the correspondence between the two incidences.

{[Post-vHIE (mismatch-neutral)]-[Before vHIE (mismatch-neutral)] in 1PP condition}
-{[Post-vHIE (mismatch-neutral)]-[Before vHIE (mismatch-neutral)] in 3PP conditions}}

<< Conclusion >>
This crossover randomized controlled trial shows that HIE has acute beneficial consequences for executive function and its neural basis, and that the movement of its avatar is measurable in substance, comparable to when it actually moves. It starts with the assumption that it can produce results. As a result, the inventor hypothesized that vHIE interventions performed by his avatar could have the same beneficial acute effects on executive function.

By testing according to the procedure described above, the inventor recorded the subject's performance on the Stroop task before and after the two vHIE intervention conditions to test hypotheses about acute beneficial effects on executive function, and at the same time fNIRS. Cortical hemodynamic changes across PFC were recorded in. This led the inventor to find better results in the Stroop task with respect to reaction time and accuracy, and to find higher activation of the neural correlation region (ie PFC) after vHIE at 1PP.

As described above, this test proved that SoBO and SoA can regulate physical and cognitive parameters from a theoretical point of view without actual movement. Also, from a clinical point of view, these results have been found to be useful for training cognitive and physical functions at the same time in a safe environment.

[1-2] Configuration example of display system according to one embodiment Therefore, in one embodiment, based on a hypothesis proved through a crossover randomized controlled trial, physical function is not restricted by the physical activity of the user. And to provide a display system that realizes improvement of cognitive function, and its method will be described.

FIG. 15 is a diagram showing a configuration example of the display system 10 as an example of one embodiment. As shown in FIG. 15, the display system 10 may optionally include the HMD1 and the PC2 shown in FIG.

HMD1 is an example of a virtual reality head-mounted display. The HMD 1 may include the two displays 1a and the cable 1b described above, as illustrated in FIG.

The cable 1b is connected to the PC2 that outputs a video signal. Examples of the cable 1b include cables conforming to standards such as HDMI (High-Definition Multimedia Interface) (registered trademark) and DisplayPort. The HMD1 may display the virtual environment of the IVR on the display 1a by the display control from the PC2 via the cable 1b.

The HMD 1 may generate and display a video signal based on the video content stored by itself. In this case, the PC 2 and the cable 1b may be omitted from the display system 10. For example, the HMD1 may be connectable to a server (including a cloud server) (not shown) by wire or wireless communication, or may be connectable to a recording medium for storing video content by wire or wireless communication. ..

The PC 2 is an example of an information processing device or a computer that controls the display of the HMD 1, and may be connected to the HMD 1 via a cable 1b. The PC2 generates a video signal in a virtual reality space (VR space) based on the video content stored in the PC2, and transmits the generated video signal to the HMD1 to display the video on the HMD1. The PC2 controls the video signal according to the control signal transmitted from the HMD1 by the tracking function of the HMD1 so that, for example, an image linked to the movement of the user (HMD1) is displayed on the display 1a. Good.

[1-3] Computer Hardware Configuration Example According to One Embodiment FIG. 16 is a block diagram showing a hardware (HW) configuration example of the computer 3. PC2 shown in FIG. 15 is an example of computer 3. Further, when the HMD1 corresponds to the acquisition and storage of video contents and the generation of video signals, the HMD1 itself or the control device included in the HMD1 is an example of the computer 3.

As shown in FIG. 16, the computer 3 has an HW configuration, for example, a processor 3a, a memory 3b, a storage device 3c, an IF (Interface) unit 3d, an I / O (Input / Output) unit 3e, and a reading unit. 3f may be provided.

The processor 3a is an example of an arithmetic processing unit that performs various controls and operations. The processor 3a may be connected to each block in the computer 3 so as to be able to communicate with each other by the bus 3i. The processor 3a may be a multiprocessor including a plurality of processors, a multicore processor having a plurality of processor cores, or a configuration having a plurality of multicore processors.

Examples of the processor 3a include integrated circuits (ICs) such as CPUs, MPUs, GPUs, APUs, DSPs, ASICs, and FPGAs. A combination of two or more of these integrated circuits may be used as the processor 3a. CPU is an abbreviation for Central Processing Unit, and MPU is an abbreviation for Micro Processing Unit. GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

The memory 3b is an example of an HW that stores information such as various data and programs. Examples of the memory 3b include a volatile memory such as a DRAM (Dynamic Random Access Memory).

The storage device 3c is an example of an HW that stores information such as various data and programs. Examples of the storage device 3c include a magnetic disk device such as an HDD (Hard Disk Drive), a semiconductor drive device such as an SSD (Solid State Drive), and various storage devices such as a non-volatile memory. Examples of the non-volatile memory include flash memory, SCM (Storage Class Memory), ROM (Read Only Memory) and the like.

Further, the storage device 3c may store a program 3g that realizes all or a part of various functions of the computer 3. For example, the processor 3a of the PC2 can realize the function as the PC2 by expanding and executing the program 3g (for example, a display program) stored in the storage device 3c in the memory 3b. Further, the processor 3a of the HMD1 can realize the function as the HMD1 by expanding and executing the program 3g (for example, a display program) stored in the storage device 3c in the memory 3b.

The IF unit 3d is an example of a communication IF that controls connection and communication with a network or other devices. For example, the IF unit 3d may include one or more adapters compliant with LAN (Local Area Network) such as Ethernet (registered trademark), HDMI (registered trademark), DisplayPort, and the like. These adapters may support one or both wireless and wired communication methods. For example, the program 3g may be downloaded to the computer 3 from the network or other device via any of the communication IFs and stored in the storage device 3c.

The I / O unit 3e may include one or both of an input device and an output device. Examples of the input device include a keyboard, a mouse, a touch panel, and the like. Examples of the output device include a monitor, a projector, a printer and the like. The display 1a of the HMD1 is an example of an output device of the I / O unit 3e.

The reading unit 3f is an example of a reader that reads data and program information recorded on the recording medium 3h. The reading unit 3f may include a connection terminal or device to which the recording medium 3h can be connected or inserted. Examples of the reading unit 3f include an adapter compliant with USB (Universal Serial Bus) and the like, a drive device for accessing a recording disk, a card reader for accessing a flash memory such as an SD card, and the like. The program 3g may be stored in the recording medium 3h, or the reading unit 3f may read the program 3g from the recording medium 3h and store it in the storage device 3c.

Examples of the recording medium 3h include a non-temporary computer-readable recording medium such as a magnetic / optical disk or a flash memory. Examples of magnetic / optical disks include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, HVDs (Holographic Versatile Discs), and the like. Examples of the flash memory include semiconductor memories such as USB memory and SD card.

The HW configuration of the computer 3 described above is an example. Therefore, the increase / decrease of HW (for example, addition or deletion of arbitrary blocks), division, integration in any combination, addition or deletion of buses, etc. in the computer 3 may be performed as appropriate. For example, the PC 2 does not have to include at least one of the I / O unit 3e and the reading unit 3f. Further, the HMD1 does not have to include the reading unit 3f. Further, the HMD1 may further include devices of various sensors (angle sensor, position sensor, inertial sensor, radio wave sensor, camera, etc.) for realizing the tracking function.

[1-4] Functional Configuration Example of Display System According to One Embodiment FIG. 17 is a block diagram showing a functional configuration example of HMD1 and PC2. In FIG. 17, a case where the PC2 generates a video signal based on the video content and outputs the video signal to the HMD1 will be described as an example, but a case where the HMD1 generates and displays the video signal based on the video content (PC2 is unnecessary). In the case), the HMD1 may have the functional configuration of the PC2 shown in FIG.

As illustrated in FIG. 17, the HMD 1 may include a communication unit 11, a memory unit 12, a display control unit 13, a display unit 14, a sensor unit 15, and an operation unit 16. The communication unit 11 is an example of the IF unit 3d shown in FIG. 16, and the display unit 14 and the operation unit 16 are examples of the I / O unit 3e shown in FIG.

The communication unit 11 communicates with the PC 2. For example, the communication unit 11 may receive a video signal from the PC 2 and transmit a control signal to the PC 2 via the communication path 4. As the communication path 4, the cable 1b illustrated in FIG. 15 can be mentioned.

The memory unit 12 stores various information used to realize the function of the HMD1. For example, the memory unit 12 may store video signal information (video information) received from the PC 2, sensing information detected by the sensor unit 15, operation information input via the operation unit 16, and the like. The memory unit 12 may be realized by, for example, a storage area included in at least one of the memory 3b and the storage device 3c (see FIG. 16) included in the HMD1.

The display control unit 13 reads the video information received from the PC 2 from the memory unit 12, and controls the display unit 14 to display the IVR video (IVR video) based on the read video information. Examples of the display unit 14 include the display 1a illustrated in FIG.

The display control unit 13 according to one embodiment may display, for example, an IVR session similar to that described in the crossover randomized controlled trial described above on the display unit 14.

For example, the display control unit 13 causes the display unit 14 to display the vHIE IVR session in the viewpoint image (1PP) of the avatar that imitates the appearance (at least gender) of the user of the HMD1. In the IVR session, the display control unit 13 may repeat the first display process and the second display process a predetermined number of times in the VR space.

In the first display process, the display control unit 13 displays the viewpoint image of the avatar when the user's avatar exercises at the first exercise intensity on the display unit 14 of the HMD1 over the first time period. You can do it. Further, in the second display process, the display control unit 13 displays the viewpoint image of the avatar when the user's avatar exercises at a second exercise intensity smaller than the first exercise intensity in the VR space. It may be displayed on the display unit 14 of the HMD1 over a period of 2 hours.

Examples of the exercise of the first exercise intensity include high-intensity aerobic exercise such as running (running), cycling, swimming, mountain climbing (trail), and dance (aerobics). The intensity of high-intensity aerobic exercise may be, for example, 50% to 70% of the user's maximum aerobic force, for example, about 60%. This is because in the physical world, there was a significant increase in cortical activation of lDLPFC after about 60% of HIE sessions.

As the exercise of the second exercise intensity, for example, the cardiopulmonary load such as walking (slow walking, slow walking) is small or close to zero (for example, 0% to 20% of the user's maximum aerobic force, for example, 5%. (Degree) intensity of exercise. Further, as another example, when the exercise of the first exercise intensity is cycling, swimming, mountain climbing (trail), dance (aerobics), etc., the second exercise intensity is the exercise intensity of these exercises as described above. The exercise intensity may be reduced to 0% to 20%, for example, about 5%.

In one embodiment, the display control unit 13 has a 1PP IVR image when the avatar is running (see FIGS. 5 and 7) and a 1PP IVR image when the avatar is walking (FIGS. 6 and 7). (See) and may be repeatedly displayed on the display unit 14.

The first time period and the second time period may be, for example, 30 seconds, respectively, according to the crossover randomized controlled trials described above. Note that these time periods are not limited to 30 seconds, and may be appropriately changed in the range of several seconds to several minutes. Further, the first time period and the second time period may be different from each other.

The display control unit 13 may repeat the first display process and the second display process described above 8 times (8 sets) according to a predetermined number of times, for example, the crossover randomized controlled trial described above. The number of repetitions of the first display process and the second display process is not limited to 8, and may be changed as appropriate.

For example, the display control unit 13 may display the vHIE shown by the reference numeral E (E') in FIG. 2 on the display unit 14 (at 1PP) as the viewpoint image of the avatar.

Further, before the display of vHIE, the display control unit 13 displays the viewpoint image of the avatar when the avatar is stationary in the IVR space on the display unit 14 of the HMD1 over a third time period. You can.

"The avatar is stationary" may mean that the avatar is not performing any exercise such as running or walking, for example, the avatar is upright, sitting, or lying down, the avatar (in other words, the user). ) May be in a relaxed (neutral) state. The exercise intensity (third exercise intensity) in the state where the avatar is stationary may be sufficiently smaller (for example, close to 0) than the first and second exercise intensity of vHIE.

The third time period is, for example, an example of the baseline (see reference numeral C) period in the crossover randomized controlled trial described above, and may be, for example, about 4 minutes. The third time period is not limited to 4 minutes, and may be appropriately changed in the range of several seconds to several minutes.

The baseline period described above is important for making the user feel as if he / she is the avatar (in other words, increase SoBO). The time it takes to feel an avatar as if you were yourself varies from person to person. Therefore, by providing a baseline period during which the avatar does not operate before vHIE, it is possible to ensure that the user feels the ownership of the avatar. During the baseline period, the user's stable heart rate can be measured. The heart rate measured at baseline can be the reference heart rate relative to the heart rate measured during the vHIE period.

The sensor unit 15 includes at least one of an angle sensor, a position sensor, an inertial sensor, a radio wave sensor, a camera, and the like that detect the angle and movement of the head of the user wearing the HMD 1, the movement of the eyeball (line of sight), and the like. It is a module equipped with various sensors. The sensor unit 15 may record the sensing information detected by the sensor in the memory unit 12. The sensing information may be used by the PC 2 as control information for performing head tracking or eye tracking.

The operation unit 16 may record the operation information input by the user via the operation means provided or connected to the HMD 1 in the memory unit 12. Examples of the operating means include at least one of an operating button, an operating controller, a mouse, a keyboard, and the like. Of the operation information, the operation request to the display control unit 13 may be output from the operation unit 16 to the display control unit 13.

As illustrated in FIG. 17, the PC 2 may include a communication unit 21, a memory unit 22, a storage unit 23, a video output unit 24, and an operation unit 25. The communication unit 21 is an example of the IF unit 3d shown in FIG. 16, and the operation unit 25 is an example of the I / O unit 3e shown in FIG.

The communication unit 21 communicates with the HMD1. For example, the communication unit 21 may transmit a video signal to the HMD 1 and receive a control signal from the HMD 1 via the communication path 4.

The memory unit 22 stores various information used for realizing the functions of the PC 2. For example, the memory unit 22 may store video signal information (video information) transmitted to the HMD 1, control information received from the HMD 1, and the like. The memory unit 22 may be realized by, for example, a storage area included in at least one of the memory 3b and the storage device 3c (see FIG. 16) included in the PC 2.

The storage unit 23 stores the video content of the IVR video to be displayed on the HMD 1. The video content may be, for example, a video file (data) created by a development environment such as Unity (registered trademark), or a file in the process of creating a video file, such as a model such as an avatar or a VR space. It may be an editable project file. In addition, the storage unit 23 may store a program in a development environment capable of processing or editing a video file or a project file.

For example, the video content may include an avatar, an animation, a virtual environment, and the like. The avatar may be selectable and switchable by gender, age, body shape, appearance, and the like. The animation may include movements of the avatar, such as movements of first and second movement intensity. The animation may be switchable by selecting and switching the type of vHIE (eg, running, cycling, swimming, etc.). The virtual environment may include an environment in which the avatar performs vHIE, for example, a background displayed on the display unit 14. As an example, the virtual environment may include a ground (floor), a landscape (indoor or outdoor), and the like.

The video output unit 24 outputs a video signal to be transmitted to the HMD 1 based on the video content recorded in the storage unit 23. For example, the video output unit 24 generates a video signal to be transmitted to the HMD 1 (display unit 14) based on the display capability, setting, profile, etc. of the IVR video of the HMD 1 (display unit 14). The video signal output by the video output unit 24 may be temporarily stored in the memory unit 22, read from the memory unit 22 by the communication unit 21, and transmitted to the HMD 1.

For example, the video output unit 24 may output a video signal of an IVR video for displaying the vHIE indicated by the reference numeral E (E') in FIG. 2 on the display unit 14 (at 1PP) as the viewpoint video of the avatar. As an example, the video output unit 24 has a first output process of outputting a viewpoint video when the avatar exercises at the first exercise intensity in the IVR space over a first time period based on the video content. , The second output process of outputting the viewpoint image when exercising at the second exercise intensity over the second time period, and the virtual motion display process of repeating the process a predetermined number of times may be executed.

Further, the video output unit 24 may output a video signal based on the control information received from the HMD1. For example, when the control signal includes sensing information, the video output unit 24 outputs a video signal based on the video content so that the IVR video (avatar's viewpoint video) moves (follows) according to the sensing information. To do. As an example, when the sensing information indicates a change from the state in which the user's head is depressed to the state in which the user's head is facing forward, the video output unit 24 faces the front from the state in which the viewpoint image of the avatar is depressed (see FIG. 5). The video signal may be output (changed) based on the video content so as to change to the state (see FIG. 7).

Further, the video output unit 24 executes a stationary state display process of outputting the viewpoint image of the avatar when the avatar is stationary in the IVR space for a third time period before the virtual motion display process. You can do it. As described above, the rest state display process is a process for securing a baseline period for the HMD1 before displaying the vHIE.

Further, for example, when the control signal includes operation information, the video output unit 24 changes the IVR image (avatar's viewpoint image) or changes various information to the IVR image according to the operation content indicated by the operation information. Overlay display and the like may be performed.

The operation unit 25 receives the operation content input by the user via the operation means provided or connected to the PC 2, and outputs the operation content to the video output unit 24 or the like. Examples of the operating means include at least one of a mouse, a keyboard, and the like.

As described above, according to the display system 10 according to the embodiment, 1PP, which is proved through the above-mentioned crossover randomized controlled trial, is as effective as the actual HIE in improving motor function and cognitive function. The vHIE in can be provided to the user.

As a result, for example, a person who does not exercise, a person who is busy, or an elderly person who is in a frail state, a person who has a heart disease, a patient who has recovered after a long-term illness, or a patient who has a movement disorder, etc. For people with limited physical activity, it is possible to provide virtual physical activity in a safe and comfortable manner that has the same beneficial effects as physical activity.

It is also possible to maximize the effectiveness of both from a physical and cognitive point of view by combining vHIE with physical exercise that exerts a load on the heart or muscle strength.

As described above, according to the display system 10 according to the embodiment, it is possible to improve the physical function and the cognitive function regardless of the limitation of the physical activity of the user.

[1-5] Operation Example of Display System According to One Embodiment FIG. 18 is a sequence diagram showing an operation example of the display system 10 according to one embodiment. As illustrated in FIG. 18, the communication unit 11 of the HMD1 transmits a vHIE start notification to the PC2 in response to a start request from the operation unit 16 by a user operation (process P1).

In response to the start notification, the video output unit 24 of the PC2 generates a vHIE video signal corresponding to the HMD1 based on the video content of the storage unit 23, and transmits the vHIE video signal to the HMD1 via the communication unit 21 (process P2). The display control unit 13 of the HMD 1 causes the display unit 14 to display the vHIE video based on the video signal received by the communication unit 11 (process P3).

The communication unit 11 transmits the sensing information detected by the sensor unit 15 to the PC 2 while displaying the vHIE image (process P4). The video output unit 24 generates (corrects) a vHIE video signal based on the sensing information and transmits it to the HMD1 (process P5). The display control unit 13 causes the display unit 14 to display the vHIE image based on the corrected image signal (process P6). The processes P4 to P6 may be executed each time the sensor unit 15 detects the sensing information during the display of the vHIE image.

When the communication unit 11 transmits the vHIE end notification to the PC 2 in response to the end request from the operation unit 16 by the user operation or in response to the end of the video content (process P7), the video output unit 24 sends the vHIE video. The output of the signal is terminated (process P8).

[2] First Example [2-1] About the First Example The display system 10 according to one embodiment can be applied to various usage situations in order to improve or prevent deterioration of cognitive function. As an example, the display system 10 may be used regularly in facilities such as elderly facilities, home care, hospitals, and rehabilitation centers.

By using the display system 10 in these facilities together with various tests on motor function and cognitive function, it is possible to confirm and evaluate the improvement effect of motor function and cognitive function by vHIE, and the cognitive function of the user is deteriorated. Prevention and mental and physical rehabilitation can be realized efficiently.

Hereinafter, a first embodiment of IVR training using the display system 10 will be described. In the first embodiment, the display system 10 is based on the experimental session shown in FIG. 2 and operates according to the session shown in FIG.

As illustrated in FIG. 19, the session may include a pre-phase, a VR training phase, and a post-phase.

In the pre-phase, TDMS (see reference numeral A) for measuring the psychological state (mood) of the user before vHIE, and the first stroop task before vHIE (see reference numeral B) for measuring the effectiveness of vHIE. And the first measurement of cortical hemodynamic changes by fNIRS before vHIE (see reference numeral B) is performed.

In the VR training phase, an online questionnaire about SoBO and SoA and a baseline recording of heart rate are performed at baseline (see reference numeral C). Further, in the speed test (reference numeral D), the first exercise intensity is adjusted. In vHIE (1PP) (reference numerals E and E'), vHIE is performed with the user sitting and gazing at the avatar without moving the body (other than the head). During the vHIE period, online questionnaires will be executed at multiple points in time. Also, during the vHIE period, heart rate measurements are taken. In the offline questionnaire (see reference numeral F), an offline questionnaire regarding SoBO and SoA is performed.

In the post-phase, TDMS (see code G) to measure the user's psychological state (mood) after vHIE, and a second post-vHIE stroop task (see code H) to measure the effectiveness of vHIE. A second measurement of cortical hemodynamic changes by fNIRS after vHIE (see reference numeral H) is performed.

The period of each phase, the period / content / order of each sequence (reference numerals A to H) in each phase, and the processing executed in each sequence (TDMS, Stroop task and fNIRS, heart rate measurement, vHIE, questionnaire, etc.) The period / content / order of the above may be the same as that of the experimental session shown in FIG. 2 described above, but at least a part thereof may be different.

[2-2] Functional Configuration Example of First Example FIG. 20 is a block diagram showing a functional configuration example of PC2A of the display system 10A according to the first embodiment. PC2A may function as a data collection server (computer) input or measured in tests performed in each phase (TDMS, stroop task, fNIRS, heart rate measurement, avatar speed test, online and offline questionnaire, etc.). ..

As shown in FIG. 20, the PC2A according to the first embodiment may include an output unit 26, a collection unit 27, and an analysis unit 28 in addition to the functions of the PC2 shown in FIG. Further, the memory unit 22 may store a plurality of collected data 22a and the analysis result 22b. The HMD1 communicating with the PC2A may have the same functional configuration as the HMD1 shown in FIG.

The output unit 26, together with the operation unit 25, is an example of the I / O unit 3e shown in FIG. 15, and is, for example, an output device such as a monitor or a speaker. In the first embodiment, the operation unit 25 may include a microphone in addition to the mouse and the keyboard.

The output unit 26 is an example of the I / O unit 3e shown in FIG. 15 together with the operation unit 25, and may include, for example, one or both of the monitor and the speaker. For example, the output unit 26 may output a question display screen to the monitor and / or output a voice of the question from the speaker for the question of the test performed in each phase.

As an example, the output unit 26 may display the display screen of the question (see FIGS. 4 and 8) on the monitor for the TDMS, the Stroop task, and the offline questionnaire, respectively. On the other hand, the output unit 26 may output the voice of the question from the speaker for the speed test and the online questionnaire performed while the user is wearing the HMD1.

The collecting unit 27, together with the communication unit 21, is an example of the IF unit 3d shown in FIG. 15, and may be connected to the heart rate sensor by wire or wirelessly, for example. The collection unit 27 collects (acquires) various types of data input or measured in the tests performed in each phase, and stores the collected data 22a in the memory unit 22.

As an example, the collecting unit 27 may collect the answers of the TDMS, the Stroop task, and the offline questionnaire input by the user (or his / her assistant) who wears the HMD1 via the operation unit 25 such as a keyboard or a mouse. .. Further, for example, the collecting unit 27 may collect the answers of the speed test and the online questionnaire, which are voice-input by the user (or an assistant thereof), as voice information via the operation unit 25 such as a microphone. Further, the collecting unit 27 may collect the user's heart rate data received from the heart rate sensor.

The analysis unit 28 analyzes the collected data 22a collected by the collection unit 27, and stores the analysis result as the analysis result 22b in the memory unit 22. Further, the analysis unit 28 may output a screen for displaying the analysis result 22b to a monitor or the like of the PC2A.

For example, the analysis unit 28 may analyze the effect of vHIE on the improvement of motor function and cognitive function based on the collected data 22a. As an example, the analysis unit 28 outputs the evaluation result of the influence of vHIE on the cognitive function of the user based on the result of the Stroop task and the measurement result of fNIRS. The evaluation result may be, for example, the result shown in FIGS. 11 to 13, and in this case, the analysis unit 28 may execute the above-mentioned derivation process of the result of FIGS. 11 to 13.

The analysis unit 28 aggregates the collected data 22b for each type (TDMS, stroop task, fNIRS, heart rate measurement, speed test, online questionnaire, offline questionnaire, etc.) (see FIGS. 9 to 14), and At least one of the comprehensive analysis results using the aggregated results of each type may be output as the analysis result 22b.

As described above, according to the display system 10A according to the first embodiment, it is possible to confirm and evaluate the improvement effect of executive function and cognitive function by vHIE, prevent deterioration of cognitive function of the user, and rehabilitate the mind and body. It can be realized efficiently.

In addition, each function provided in PC 2A may be distributed and arranged in a plurality of computers 3. As an example, the functions of the video output unit 24 and the like, the function of the output unit 26, the function of the collection unit 27, and the function of the analysis unit 28 included in the PC 2 shown in FIG. 17 are distributed and arranged in two or more computers 3. You can do it.

[2-3] Operation Example of PC According to First Example FIG. 21 is a flowchart showing an operation example of PC2A according to the first embodiment. As illustrated in FIG. 21, the output unit 26 of the PC 2A determines whether or not the output timing of the screen or audio to be output to the user for collecting data in each phase of the session (see FIG. 19) has arrived. (Step S1).

When it is determined that the output timing has not arrived (NO in step S1), the process proceeds to step S3. On the other hand, when it is determined that the output timing has arrived (YES in step S1), the output unit 26 outputs the screen or sound according to the sequence from the monitor or the speaker (step S2), and the process shifts to step S3. ..

In step S3, the collecting unit 27 determines whether or not input data or measurement data has been received from the operating unit 25, the heart rate sensor, or fNIRS.

When it is determined that the input data or the measurement data has not been received (NO in step S3), the process proceeds to step S5. On the other hand, when it is determined that the input data or the measurement data has been received (YES in step S3), the collection unit 27 saves the received data as the collection data 22a in the memory unit 22 (step S4), and the process proceeds to step S5. Transition.

In step S5, PC2A determines whether or not the session has ended. For example, the PC2A may determine whether or not it has received the data input and measured with reference numeral H in FIG.

If it is determined that the session has not ended (NO in step S5), the process proceeds to step S1. On the other hand, when it is determined that the session has ended (YES in step S5), the analysis unit 28 analyzes the collected data 22a stored in the memory unit 22 (step S6), and saves the analysis result 22b in the memory unit 22. ..

Then, the analysis unit 28 outputs the analysis result 22b to a monitor or the like (step S7), and the process ends.

[3] Second Example For example, the SoBO (body ownership) recognized by the user by vHIE at 1PP can excite aspects such as the user's physical function, and as a result, further aspects such as cognitive function can be excited. Can be excited.

In this way, SoBO can be positioned as a variable that is manipulated through the user's point of view (or the avatar's point of view in 1PP). In other words, the degree of improvement of the user's physical function and cognitive function by vHIE can be controlled according to a variable called SoBO.

Further, since the SoBO can change according to the displayed avatar, it can be said that the SoBO is a variable that can greatly affect the avatar (for example, the motion of the avatar). In other words, it is possible to define a variable called SoBO in the theoretical model of motion control used to control the motion of the avatar.

It should be noted that SoA (sense of exercise subjectivity) may also be regarded as a variable in the same manner as SoBO.

Therefore, in the second embodiment, the SoBO and / or SoA is changed based on the analysis result 22b analyzed by the display system 10A according to the first embodiment, so that the physical function and the cognitive function of the user by vHIE can be determined. An example of optimizing the improvement will be described.

FIG. 22 is a block diagram showing a functional configuration example of PC2B of the display system 10B according to the second embodiment. As shown in FIG. 22, the PC2B according to the second embodiment may include a parameter changing unit 29 in addition to the functional configuration included in the PC2A shown in FIG. Further, the memory unit 22 may further store the parameter information 22c.

The parameter changing unit 29 changes the parameters used for outputting the video signal by the video output unit 24 based on the analysis result 22b. For example, the parameter changing unit 29 may update the parameter information 22c stored in the memory unit 22.

FIG. 23 is a diagram showing an example of parameter information 22c. As shown in FIG. 23, the parameter information 22c may optionally include the user's identification information, exercise intensity (High, Low), and display parameter items.

Exercise intensity (High) and exercise intensity (Low) are examples of the first and second exercise intensity, respectively, and indicate the exercise intensity of vHIE to be executed by the user's avatar. For example, information indicating exercise intensity such as a moving speed (m / s) and a ratio (%) with respect to a reference exercise intensity may be set in High and Low. The exercise intensity (High) may be set as an initial value, for example, the exercise intensity selected by the user in the session speed test (see reference numeral D in FIG. 19).

The display parameter indicates the display parameter of the display unit 14 (display 1a) of the HMD1. For example, various parameters that affect the user's immersive feeling, such as the viewing angle (FOV) and the resolution, may be set as the display parameters.

The parameter information 22c may include parameters related to the avatar such as gender, age, body shape, appearance, and motion of the avatar.

The video output unit 24 may generate a video signal based on the video content and the parameter information 22c.

For example, the parameter changing unit 29 may gradually change at least one parameter included in the parameter information 22c for each session of the same user, and manage the changed parameter in association with the analysis result 22b. Then, the parameter changing unit 29 sets the parameter associated with the analysis result 22b in which the SoBO and / or SoA is optimal (for example, the maximum) from the analysis results 22b obtained from the plurality of sessions as the optimum parameter, and sets the parameter information 22c. May be recorded in. The parameter changing unit 29 may optimize the parameter information 22c by executing such optimization for other parameters included in the parameter information 22c.

The method for optimizing the parameter information 22c by the parameter changing unit 29 is not limited to this, and various known optimization methods may be adopted.

Further, the parameter information 22c may include parameters such as, for example, the period of the session, the period of the phase, the period of processing within the phase, each period of running and walking in vHIE, and the number of executions. The parameter changing unit 29 may also determine such a parameter by an optimization method so that SoBO and / or SoA becomes optimum (for example, maximum).

[4] Others The technology according to the above-described embodiment can be modified or modified as follows.

For example, in the HMD 1 shown in FIG. 17, the functional blocks 11 to 16 may be merged in any combination or may be divided respectively. Further, in the PC 2 shown in FIG. 17, the functional blocks 21 to 25 may be merged in any combination or may be divided respectively. Further, in the PC 2A shown in FIG. 20, the functional blocks 21 to 28 may be merged in any combination or may be divided respectively. Further, in the PC 2B shown in FIG. 22, the functional blocks 21 to 29 may be merged in any combination or may be divided respectively.

Further, the HMD1 may include, for example, the functions of the storage unit 23 and the video output unit 24 among the functions of the PC2, PC2A, or PC2B. In this case, these functions in PC2, PC2A, or PC2B may be omitted. For example, as shown in FIG. 24, the display system 10C may include an HMD1C further including a storage unit 23 and a video output unit 24 in addition to the functions of the HMD1 shown in FIG. In the example of FIG. 24, the display system 10C may include a PC2C in which the storage unit 23 and the video output unit 24 are deleted from the PC2B shown in FIG. 22. As shown in FIG. 24, since the HMD1C has the function of the video output unit 24, the above-described rest state display process and virtual motion display process can be executed on the HMD1C.

Further, the output unit 26, the collection unit 27, the analysis unit 28, and the parameter change unit 29 of the PC2B of the PC2A and the PC2B are distributed to one or more computers 3 (including a cloud computer) different from the PC2A or the PC2B. Alternatively, it may be provided in a redundant manner. In this case, these functions in PC2A and PC2B may be omitted.

[5] Additional Notes The following additional notes will be further disclosed with respect to the above-described embodiment and the first and second embodiments.

(Appendix 1)
In the virtual reality space, the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity is displayed on the head-mounted display worn by the user for the first time period. Display processing of 1 and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises at a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. A second display process to be displayed on the head-mounted display worn by the user, and
A display program that causes a computer to execute a virtual motion display process that repeats the above steps a predetermined number of times.

(Appendix 2)
The result of the first cognitive function test of the user wearing the head-mounted display, which was executed before the virtual motion display process, and the user's first cognitive function test performed during the first cognitive function test. The result of the brain function measurement of 1 and the result are stored in the storage area,
The result of the user's second cognitive function test executed after the virtual movement display process, and the result of the user's second brain function measurement performed during the second cognitive function test. , Is stored in the storage area,
Based on the results of the first and second cognitive function tests and the results of the first and second brain function measurements, an analysis result regarding the user's cognitive function in the virtual motion display process is output.
The display program according to Appendix 1, which causes the computer to execute the process.

(Appendix 3)
The evaluation results regarding the subjective emotions of the user acquired from the user before, during, and after the execution of the virtual motion display process are stored in the storage area.
Based on the evaluation result, the analysis result regarding one or both of the physical possession feeling and the movement subject feeling of the user in the virtual movement display processing is output.
The display program according to Appendix 2, which causes the computer to execute the process.

(Appendix 4)
The user's heart rate measured before and during the execution of the virtual motion display process is stored in the storage area.
Output the analysis result of the heart rate,
The display program according to Appendix 2 or Appendix 3, which causes the computer to execute the process.

(Appendix 5)
The evaluation results regarding the psychological state of the user obtained from the user in each of the pre-first cognitive function test and the pre-second cognitive function test are stored in the storage area. And
Based on the evaluation result regarding the psychological state, the analysis result of the arousal level of the user in the virtual motion display process is output.
The display program according to any one of Supplementary note 2 to Supplementary note 4, which causes the computer to execute the process.

(Appendix 6)
Based on the output of one or more of the analysis results, the exercise intensity of one or both of the first exercise intensity and the second exercise intensity is changed.
The display program according to any one of Supplementary note 2 to Supplementary note 5, which causes the computer to execute the process.

(Appendix 7)
The display parameters of the head-mounted display are changed based on the output one or more of the analysis results.
The display program according to any one of Supplementary note 2 to Supplementary note 6, which causes the computer to execute the process.

(Appendix 8)
Prior to the virtual motion display process, the head mount worn by the user for a third time period is a viewpoint image of the virtual body when the virtual body is stationary in the virtual reality space. Still state display processing to be displayed on the display,
The display program according to any one of Supplementary note 1 to Supplementary note 7, wherein the computer is executed.

(Appendix 9)
The computer is a computer that is wired or wirelessly connected to the head-mounted display and controls the display of the head-mounted display.
The display program according to any one of Supplementary note 1 to Supplementary note 8.

(Appendix 10)
The computer is a computer provided in the head-mounted display.
The display program according to any one of Supplementary note 1 to Supplementary note 8.

(Appendix 11)
The computer
In the virtual reality space, the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity is displayed on the head-mounted display worn by the user for the first time period. Display processing of 1 and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises at a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. A second display process to be displayed on the head-mounted display worn by the user, and
A display method that executes a virtual motion display process that repeats a predetermined number of times.

(Appendix 12)
The result of the first cognitive function test of the user wearing the head-mounted display, which was executed before the virtual motion display process, and the user's first cognitive function test performed during the first cognitive function test. The result of the brain function measurement of 1 and the result are stored in the storage area,
The result of the user's second cognitive function test executed after the virtual movement display process, and the result of the user's second brain function measurement performed during the second cognitive function test. , Is stored in the storage area,
Based on the results of the first and second cognitive function tests and the results of the first and second brain function measurements, an analysis result regarding the user's cognitive function in the virtual motion display process is output.
The display method according to Appendix 11, wherein the computer executes the process.

(Appendix 13)
The evaluation results regarding the subjective emotions of the user acquired from the user before, during, and after the execution of the virtual motion display process are stored in the storage area.
Based on the evaluation result, the analysis result regarding one or both of the physical possession feeling and the movement subject feeling of the user in the virtual movement display processing is output.
The display method according to Appendix 12, wherein the computer executes the process.

(Appendix 14)
The user's heart rate measured before and during the execution of the virtual motion display process is stored in the storage area.
Output the analysis result of the heart rate,
The display method according to Appendix 12 or Appendix 13, wherein the computer executes the process.

(Appendix 15)
The evaluation results regarding the psychological state of the user obtained from the user in each of the pre-first cognitive function test and the pre-second cognitive function test are stored in the storage area. And
Based on the evaluation result regarding the psychological state, the analysis result of the arousal level of the user in the virtual motion display process is output.
The display method according to any one of Supplementary note 12 to Supplementary note 14, wherein the processing is executed by the computer.

(Appendix 16)
Based on the output of one or more of the analysis results, the exercise intensity of one or both of the first exercise intensity and the second exercise intensity is changed.
The display method according to any one of Supplementary note 12 to Supplementary note 15, wherein the processing is executed by the computer.

(Appendix 17)
The display parameters of the head-mounted display are changed based on the output one or more of the analysis results.
The display method according to any one of Supplementary note 12 to Supplementary note 16, wherein the processing is executed by the computer.

(Appendix 18)
Prior to the virtual motion display process, the head mount worn by the user for a third time period is a viewpoint image of the virtual body when the virtual body is stationary in the virtual reality space. Still state display processing to be displayed on the display,
The display method according to any one of Supplementary note 11 to Supplementary note 17, wherein the computer executes the above.

(Appendix 19)
A display that displays virtual reality space and
A display control unit that controls the display of the display is provided.
The display control unit
In the virtual reality space, the first display process of displaying the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity on the display over the first time period. ,
In the virtual reality space, a viewpoint image of the virtual body when the virtual body of the user exercises with a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. The second display process to be displayed on the display and
A head-mounted display that executes a virtual motion display process that repeats a predetermined number of times.

(Appendix 20)
A communication unit that communicates with a head-mounted display that displays virtual reality space,
A video output unit that outputs a video signal via the communication unit is provided.
The video output unit
In the virtual reality space, the first output process of outputting the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity over the first time period, and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises with a second exercise intensity smaller than the first exercise intensity is output over a second time period. Second output processing to be done
An information processing device that executes virtual motion display output processing that repeats a predetermined number of times.

(Appendix 21)
The result of the first cognitive function test of the user wearing the head-mounted display, which was executed before the virtual motion display process, and the user's first cognitive function test performed during the first cognitive function test. The result of the brain function measurement of 1, the result of the user's second cognitive function test executed after the virtual movement display process, and the user's second cognitive function test performed during the second cognitive function test. The result of the second brain function measurement, the storage area for storing, and
Based on the results of the first and second cognitive function tests and the results of the first and second brain function measurements, an analysis unit that outputs an analysis result regarding the user's cognitive function in the virtual movement display processing, and an analysis unit. 20. The information processing apparatus according to Appendix 20.

(Appendix 22)
The storage area stores the evaluation results regarding the subjective emotions of the user acquired from the user before, during, and after the execution of the virtual motion display process.
Based on the evaluation result, the analysis unit outputs an analysis result regarding one or both of the user's physical possession feeling and the movement subject feeling in the virtual movement display processing.
The information processing device according to Appendix 21.

(Appendix 23)
The storage area stores the user's heart rate measured before and during the execution of the virtual motion display process.
The analysis unit outputs the analysis result of the heart rate.
The information processing device according to Appendix 21 or Appendix 22.

(Appendix 24)
The storage area stores the evaluation results regarding the psychological state of the user obtained from the user in each of the pre-first cognitive function test and the pre-second cognitive function test. And
The analysis unit outputs the analysis result of the arousal level of the user in the virtual motion display process based on the evaluation result regarding the psychological state.
The information processing device according to any one of Supplementary note 21 to Supplementary note 23.

(Appendix 25)
A parameter changing unit that changes the exercise intensity of one or both of the first exercise intensity and the second exercise intensity based on the output one or more of the analysis results.
The information processing apparatus according to any one of Supplementary note 21 to Supplementary note 24.

(Appendix 26)
The parameter changing unit changes the display parameter of the head-mounted display based on the output one or more of the analysis results.
25. The information processing apparatus according to Appendix 25.

1,1C Head Mounted Display (HMD)
1a Display 1b Cable 10, 10A, 10B, 10C Display system 11, 21 Communication unit 12, 22 Memory unit 13 Display control unit 14 Display unit 15 Sensor unit 16, 25 Operation unit 2, 2A, 2B, 2C PC
22a Collected data 22b Analysis result 22c Parameter information 23 Storage unit 24 Video output unit 26 Output unit 27 Collection unit 28 Analysis unit 29 Parameter change unit 3 Computer 3a Processor 3b Memory 3c Storage device 3d IF unit 3e I / O unit 3f Reading unit 3g Program 3h Recording medium 4 Communication path

Claims (11)

In the virtual reality space, the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity is displayed on the head-mounted display worn by the user for the first time period. Display processing of 1 and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises at a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. A second display process to be displayed on the head-mounted display worn by the user, and
A display program that causes a computer to execute a virtual motion display process that repeats the above steps a predetermined number of times. The result of the first cognitive function test of the user wearing the head-mounted display, which was executed before the virtual motion display process, and the user's first cognitive function test performed during the first cognitive function test. The result of the brain function measurement of 1 and the result are stored in the storage area,
The result of the user's second cognitive function test executed after the virtual movement display process, and the result of the user's second brain function measurement performed during the second cognitive function test. , Is stored in the storage area,
Based on the results of the first and second cognitive function tests and the results of the first and second brain function measurements, an analysis result regarding the user's cognitive function in the virtual motion display process is output.
The display program according to claim 1, wherein the computer executes the process. The evaluation results regarding the subjective emotions of the user acquired from the user before, during, and after the execution of the virtual motion display process are stored in the storage area.
Based on the evaluation result, the analysis result regarding one or both of the physical possession feeling and the movement subject feeling of the user in the virtual movement display processing is output.
The display program according to claim 2, wherein the computer executes the process. The user's heart rate measured before and during the execution of the virtual motion display process is stored in the storage area.
Output the analysis result of the heart rate,
The display program according to claim 2 or 3, wherein the processing is executed by the computer. The evaluation results regarding the psychological state of the user obtained from the user in each of the pre-first cognitive function test and the pre-second cognitive function test are stored in the storage area. And
Based on the evaluation result regarding the psychological state, the analysis result of the arousal level of the user in the virtual motion display process is output.
The display program according to any one of claims 2 to 4, which causes the computer to execute the process. Based on the output of one or more of the analysis results, the exercise intensity of one or both of the first exercise intensity and the second exercise intensity is changed.
The display program according to any one of claims 2 to 5, which causes the computer to execute the process. The display parameters of the head-mounted display are changed based on the output one or more of the analysis results.
The display program according to any one of claims 2 to 6, wherein the computer executes the process. Prior to the virtual motion display process, the head mount worn by the user for a third time period is a viewpoint image of the virtual body when the virtual body is stationary in the virtual reality space. Still state display processing to be displayed on the display,
The display program according to any one of claims 1 to 7, wherein the computer is executed. The computer
In the virtual reality space, the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity is displayed on the head-mounted display worn by the user for the first time period. Display processing of 1 and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises at a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. A second display process to be displayed on the head-mounted display worn by the user, and
A display method that executes a virtual motion display process that repeats a predetermined number of times. A display that displays virtual reality space and
A display control unit that controls the display of the display is provided.
The display control unit
In the virtual reality space, the first display process of displaying the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity on the display over the first time period. ,
In the virtual reality space, a viewpoint image of the virtual body when the virtual body of the user exercises with a second exercise intensity smaller than the first exercise intensity is displayed over a second time period. The second display process to be displayed on the display and
A head-mounted display that executes a virtual motion display process that repeats a predetermined number of times. A communication unit that communicates with a head-mounted display that displays virtual reality space,
A video output unit that outputs a video signal via the communication unit is provided.
The video output unit
In the virtual reality space, the first output process of outputting the viewpoint image of the virtual body when the user's virtual body exercises at the first exercise intensity over the first time period, and
In the virtual reality space, the viewpoint image of the virtual body when the virtual body of the user exercises with a second exercise intensity smaller than the first exercise intensity is output over a second time period. Second output processing to be done
An information processing device that executes virtual motion display output processing that repeats a predetermined number of times.

Images