JP6948220B2 - Methods, programs and computers for providing virtual experiences to users - Google Patents

Description

The present disclosure relates to methods, programs and computers for providing virtual experiences to users.

Patent Document 1 describes sickness in a virtual reality (VR) space (so-called "VR") when a user's visual field image is updated in synchronization with the movement of a head-mounted device (HMD). A technique for reducing "sickness") is disclosed. According to this technology, when the virtual camera is moved by an input from an external controller, a line-of-sight guidance area for guiding the user's line of sight is generated. By hiding a part of the user's visual field image by this line-of-sight guidance region, VR sickness when the virtual camera moves is reduced.

In the future, it is expected that various technologies for reducing VR sickness will be developed without impairing the virtual experience of the user in the virtual space.

Japanese Patent No. 6087453

An object of the present disclosure is to provide a novel method for reducing VR sickness without impairing the user's virtual experience when providing the user with a virtual experience regarding an event occurring in the virtual space.

According to one embodiment of the present disclosure, a computer for providing a virtual experience to a user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A program is provided for executing a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

According to another embodiment of the present disclosure, it is a computer.
Controlled by a processor included in the computer to provide a virtual experience to the user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A computer is provided in which, while the action is being performed, a step of controlling the field of view is performed so that the degree of visibility of the event in the field of view satisfies the condition.

According to another embodiment of the present disclosure, a method performed by a computer to provide a virtual experience to a user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A method is provided that includes a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

According to the embodiment of the present disclosure, when providing a virtual experience regarding an event occurring in a virtual space to a user, it is possible to provide a novel method for reducing VR sickness without impairing the virtual experience of the user.

Other features and advantages of the present disclosure will become apparent from the description of embodiments described below, the accompanying drawings and the claims.

It is a figure which shows the structure of the HMD system schematicly. It is a block diagram which shows the example of the basic hardware composition of the computer by one Embodiment of this disclosure. It is a figure that conceptually represents the uvw field-of-view coordinate system set in the HMD according to one embodiment. It is a figure which conceptually represents one aspect which expresses a virtual space according to one Embodiment. It is the figure which showed the head of the user who wears HMD according to one Embodiment from the top. It is a figure which shows the yz cross section which looked at the visual area from the x direction in a virtual space. It is a figure which shows the xz cross section which looked at the visual area in a virtual space from the y direction. It is a figure which shows the schematic structure of the controller according to one Embodiment. It is a figure which shows the schematic structure of the controller according to one Embodiment. FIG. 3 is a block diagram showing a function of a computer for providing a virtual experience to a user via an HMD system according to an embodiment of the present disclosure. It is a flow diagram of a general process for displaying an image of a virtual space in which a user is immersed on a display unit. It is a flowchart of the method by one Embodiment of this disclosure. It is a flowchart which shows the specific example of the process executed in steps 1110 and 1112. It is a figure which shows roughly the appearance that the event occurred during the play of a game. It is a figure which shows the example of the field of view image provided to the user with respect to the scene shown in FIG. It is a flowchart which shows the specific example of the process executed in step 1116. It is a figure which shows the state of the process performed in step 1502 schematically. It is a figure which shows the state of the process performed in step 1502 schematically. It is a figure which shows the state of the process performed in step 1502 schematically. It is a figure which shows the state of the process performed in step 1504 schematically. It is a figure which shows the state of the process performed in step 1504 schematically. It is a figure which shows the state of the process performed in step 1504 schematically. It is a flowchart which shows the specific example of the process executed in step 1118. It is a figure explaining how to move a virtual viewpoint of a user. It is a figure which shows the field of view image after moving a virtual camera so that the center point of the field of view image coincides with a reference point. It is a flowchart which shows the specific example of the process executed in steps 1110 and 1112. It is a figure explaining the relationship between a feature point set in an object, and a field of view image. It is a flowchart which shows the specific example of the process executed in steps 1110 and 1112. It is a figure which shows the example of the number of objects in the field of view obtained within a predetermined period, and the value based on this.

[Explanation of Embodiments of the present disclosure]
First, the configurations of exemplary embodiments of the present disclosure will be listed and described. The methods, programs and computers according to the embodiments of the present disclosure may have the following configurations.

(Item 1)
To a computer for providing a virtual experience to the user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A program for executing a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

(Item 2)
The step of executing the process is
Including the step of outputting a voice suggesting that the action is performed.
The program according to item 1, wherein the one or more character objects are one or more character objects associated with the voice.

(Item 3)
The step of executing the process is
As the action, a first action of covering the field of view with the one or more character objects and a second action of triggering the field of view to be controlled so that the degree of visibility of the event is satisfied. , The program according to item 1 or 2, which executes a process to be performed by the one or more character objects.

(Item 4)
The event is an event in which one or more objects appear.
When a predetermined number or more of the one or a plurality of objects are not included in the field of view, the computer determines that the degree of visibility of the event in the field of view does not satisfy the condition. The program according to any one of items 1 to 3 for further execution by the computer.

(Item 5)
The event is an event in which one or more objects appear.
When a predetermined feature point among the one or more feature points preset for the one or more objects is not included in the field of view, the degree of visibility of the event in the field of view determines the condition. The program according to any one of items 1 to 3, for causing the computer to further perform a step of determining that the condition is not satisfied.

(Item 6)
The event is an event in which one or more objects appear.
In the determination step, when the value based on the number of the one or more objects appearing in the field of view within a predetermined period is less than the predetermined number, the degree of visibility of the event in the field of view is the condition. The program according to any one of items 1 to 3, which includes a step of determining that the above conditions are not satisfied.

(Item 7)
The event is an event in which at least one of the one or more character objects appears.
The character object that performs the action among the one or more character objects and the character object that appears in the event among the one or more character objects have common attributes.
The program according to any one of items 1 to 6, wherein the common attribute includes at least one of gender, clothing, affiliation group and size.

(Item 8)
It ’s a computer,
Controlled by a processor included in the computer to provide a virtual experience to the user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A computer that performs a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

(Item 9)
A method performed by a computer to provide a virtual experience to a user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A method comprising the step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

[Details of Embodiments of the present disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, similar elements are designated by the same reference numerals. The same applies to their names and functions. Duplicate description of such elements is omitted.

The configuration of the head-mounted device (HMD) system 100 will be described with reference to FIG. FIG. 1 schematically shows the configuration of the HMD system 100. In one example, the HMD system 100 is provided as a home system or a business system. The HMD may be a so-called head-mounted display provided with a display unit, or may be a head-mounted device to which a terminal such as a smartphone having a display unit can be attached.

The HMD system 100 includes an HMD 110, an HMD sensor 120, a controller 160, and a computer 200. The HMD 110 includes a display unit 112 and a gaze sensor 140. The controller 160 may include a motion sensor 130.

In one example, the computer 200 may be able to connect to a network 192 such as the Internet, or may be able to communicate with a computer such as a server 150 connected to the network 192. In another embodiment, the HMD 110 may include a sensor 114 instead of the HMD sensor 120.

The HMD 110 may be worn on the head of the user 190 and provide the user with virtual space during operation. More specifically, the HMD 110 displays an image for the right eye and an image for the left eye on the display unit 112, respectively. When each eye of the user visually recognizes the image, the user can recognize the image as a three-dimensional image based on the parallax of both eyes.

The display unit 112 is realized as, for example, a non-transparent display device. In one example, the display unit 112 is arranged on the main body of the HMD 110 so as to be located in front of both eyes of the user. Therefore, the user can immerse himself in the virtual space by visually recognizing the three-dimensional image displayed on the display unit 112. In certain embodiments, the virtual space includes, for example, a background, user-operable objects, user-selectable menu images, and the like. In certain embodiments, the display unit 112 can be realized as a liquid crystal display unit or an organic EL (Electroluminescence) display unit included in an information display terminal such as a smartphone.

In one example, the display unit 112 may include a sub-display unit for displaying an image for the right eye and a sub-display unit for displaying an image for the left eye. In another aspect, the display unit 112 may be configured to display the image for the right eye and the image for the left eye as a unit. In this case, the display unit 112 includes a high-speed shutter. The high-speed shutter operates so that the image for the right eye and the image for the left eye can be alternately displayed so that the image is recognized by only one of the eyes.

In one example, the HMD 110 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 120 has a position tracking function for detecting the movement of the HMD 110. More specifically, the HMD sensor 120 may read a plurality of infrared rays emitted by the HMD 110 to detect the position and inclination of the HMD 110 in the real space.

In some embodiments, the HMD sensor 120 may be implemented by a camera. In this case, the HMD sensor 120 can detect the position and tilt of the HMD 110 by executing the image analysis process using the image information of the HMD 110 output from the camera.

In another aspect, the HMD 110 may include a sensor 114 as a position detector instead of the HMD sensor 120. The HMD 110 can detect the position and tilt of the HMD 110 itself by using the sensor 114. For example, when the sensor 114 is an angular velocity sensor, a geomagnetic sensor, an acceleration sensor, a gyro sensor, or the like, the HMD 110 detects its position and inclination by using any of these sensors instead of the HMD sensor 120. obtain. As an example, when the sensor 114 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around the three axes of the HMD 110 in real space over time. The HMD 110 calculates the temporal change of the angle around the three axes of the HMD 110 based on each angular velocity, and further calculates the inclination of the HMD 110 based on the temporal change of the angle. Further, the HMD 110 may be provided with a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transparent display device by adjusting its transmittance. Further, the field of view image may include a configuration for presenting the real space as a part of the image constituting the virtual space. For example, an image taken by a camera mounted on the HMD 110 may be superimposed on a part of the view image and displayed, or by setting a high transmittance of a part of the transmissive display device, the view image can be displayed. The real space may be visible from a part.

The gaze sensor 140 detects the direction (line of sight) to which the eyes of the right eye and the left eye of the user 190 are directed. The detection of the direction is realized by, for example, a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In some embodiments, the gaze sensor 140 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 140 may be, for example, a sensor that irradiates the right eye and the left eye of the user 190 with infrared light and detects the angle of rotation of each eyeball by receiving the reflected light from the cornea and the iris with respect to the irradiation light. .. The gaze sensor 140 can detect the line of sight of the user 190 based on each of the detected rotation angles.

The server 150 may send the program to the computer 200. In another aspect, the server 150 may communicate with another computer 200 to provide virtual reality to the HMD used by other users. For example, in an amusement facility, when a plurality of users play a participatory game, each computer 200 communicates a signal based on the operation of each user with another computer 200, and the plurality of users are common in the same virtual space. Allows you to enjoy the game.

The controller 160 is connected to the computer 200 by wire or wirelessly. The controller 160 receives an instruction input from the user 190 to the computer 200. In some embodiments, the controller 160 is configured to be grippable by the user 190. In another embodiment, the controller 160 is configured to be wearable on a part of the body or clothing of the user 190. In another aspect, the controller 160 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In another aspect, the controller 160 receives from the user 190 an operation for controlling the position and movement of an object arranged in the virtual space.

In some embodiments, the motion sensor 130 is attached to the user's hand to detect the movement of the user's hand. For example, the motion sensor 130 detects the rotation speed, the number of rotations, and the like of the hand. The detected signal is sent to the computer 200. The motion sensor 130 is provided on, for example, a glove-shaped controller 160A. In certain embodiments, for safety in real space, it is desirable that the controller 160A be attached to something that does not easily fly by being attached to the user 190's hand, such as a glove type. In another embodiment, a controller 160B having a general structure having a plurality of operation buttons may be used. In another embodiment, a sensor not attached to the user 190 may detect the movement of the user 190's hand. Optical sensors may be used to detect the position, orientation, direction of movement, distance of movement, etc. of various parts of the body of the user 190. For example, the signal of the camera that captures the user 190 may be input to the computer 200 as a signal representing the operation of the user 190. As an example, the motion sensor 130 and the computer 200 are wirelessly connected to each other. In the case of wireless communication, the communication mode is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication method is used.

The computer 200 according to the embodiment of the present disclosure will be described with reference to FIG. FIG. 2 is a block diagram showing an example of a basic hardware configuration of the computer 200 according to the embodiment of the present disclosure. The computer 200 includes a processor 202, a memory 204, a storage 206, an input / output interface 208, and a communication interface 210 as main components. Each component is connected to bus 212, respectively.

The processor 202 executes a series of instructions included in the program stored in the memory 204 or the storage 206 based on the signal given to the computer 200 or when a predetermined condition is satisfied. In some embodiments, the processor 202 is realized as a device such as a CPU (Central Processing Unit), an MPU (Micro Processor Unit), or an FPGA (Field-Programmable Gate Array).

Memory 204 temporarily stores programs and data. The program is loaded from storage 206, for example. The data includes data input to the computer 200 and data generated by the processor 202. In some embodiments, the memory 204 is realized as a volatile memory such as a RAM (Random Access Memory).

Storage 206 permanently holds programs and data. The storage 206 is realized as, for example, a non-volatile storage device such as a ROM (Read-Only Memory), a hard disk device, or a flash memory. The program stored in the storage 206 includes a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, a program for realizing communication with another computer 200, and the like. The data stored in the storage 206 includes data, objects, and the like for defining the virtual space.

In another aspect, the storage 206 may be realized as a removable storage device such as a memory card. In yet another embodiment, a configuration using programs and data stored in an external storage device may be used instead of the storage 206 built into the computer 200. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used such as an amusement facility, it is possible to update programs and data at once.

In certain embodiments, the input / output interface 208 communicates signals with the HMD 110, the HMD sensor 120, and the motion sensor 130. In some embodiments, the input / output interface 208 is implemented using terminals such as USB (Universal Serial Bus, USB), DVI (Digital Visual Interface), HDMI® (High-Definition Multimedia Interface), and the like. The input / output interface 208 is not limited to the above.

In certain embodiments, the input / output interface 208 may further communicate with the controller 160. For example, the input / output interface 208 receives input of signals output from the controller 160 and the motion sensor 130. In another embodiment, the input / output interface 208 sends an instruction output from the processor 202 to the controller 160. The command instructs the controller 160 to vibrate, output voice, emit light, and the like. Upon receiving the command, the controller 160 executes vibration, voice output, light emission, and the like in response to the command.

The communication interface 210 is connected to the network 192 and communicates with another computer (for example, the server 150) connected to the network 192. In some embodiments, the communication interface 210 is realized as, for example, a wired communication interface such as LAN (Local Area Network), or a wireless communication interface such as WiFi (Wireless Field), Bluetooth (registered trademark), NFC (Near Field Communication), etc. Will be done. The communication interface 210 is not limited to the above.

In some embodiments, the processor 202 accesses storage 206, loads one or more programs stored in storage 206 into memory 204, and executes a series of instructions contained in the programs. The one or more programs may include an operating system of a computer 200, an application program for providing a virtual space, game software that can be executed in the virtual space, and the like. The processor 202 sends a signal for providing virtual space to the HMD 110 via the input / output interface 208. The HMD 110 displays an image on the display unit 112 based on the signal.

In the example shown in FIG. 2, the computer 200 is provided outside the HMD 110. However, in another aspect, the computer 200 may be built into the HMD 110. As an example, a portable information communication terminal (for example, a smartphone) including the display unit 112 may function as the computer 200.

Further, the computer 200 may have a configuration commonly used for a plurality of HMD 110s. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so that each user can enjoy the same application as other users in the same virtual space.

In one embodiment, the HMD system 100 has a preset global coordinate system. The global coordinate system has three reference directions (axises) that are parallel to the vertical direction in the real space, the horizontal direction orthogonal to the vertical direction, and the front-back direction orthogonal to both the vertical direction and the horizontal direction. In this embodiment, the global coordinate system is one of the viewpoint coordinate systems. Therefore, the horizontal direction, the vertical direction (vertical direction), and the front-back direction in the global coordinate system are defined as the x-axis, the y-axis, and the z-axis, respectively. More specifically, in the global coordinate system, the x-axis is parallel to the horizontal direction of real space. The y-axis is parallel to the vertical direction in real space. The z-axis is parallel to the front-back direction of the real space.

In some embodiments, the HMD sensor 120 includes an infrared sensor. When the infrared sensor detects infrared rays emitted from each light source of the HMD 110, the presence of the HMD 110 is detected. The HMD sensor 120 further detects the position and inclination of the HMD 110 in the real space according to the movement of the user 190 wearing the HMD 110 based on the value of each point (each coordinate value in the global coordinate system). More specifically, the HMD sensor 120 can detect a temporal change in the position and inclination of the HMD 110 by using each value detected over time.

The global coordinate system is parallel to the coordinate system in real space. Therefore, each inclination of the HMD 110 detected by the HMD sensor 120 corresponds to each inclination of the HMD 110 around three axes in the global coordinate system. The HMD sensor 120 sets the uvw field of view coordinate system to the HMD 110 based on the tilt of the HMD 110 in the global coordinate system. The uvw field-of-view coordinate system set in the HMD 110 corresponds to the viewpoint coordinate system when the user 190 wearing the HMD 110 sees an object in the virtual space.

The uvw field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually representing the uvw field coordinate system set in the HMD 110 according to a certain embodiment. The HMD sensor 120 detects the position and tilt of the HMD 110 in the global coordinate system when the HMD 110 is activated. Processor 202 sets the uvw field coordinate system to HMD110 based on the detected values.

As shown in FIG. 3, the HMD 110 sets a three-dimensional uvw visual field coordinate system centered (origin) on the head of the user wearing the HMD 110. More specifically, the HMD 110 defines the global coordinate system in the horizontal, vertical, and front-back directions (x-axis, y-axis, z-axis) by the inclination of the HMD 110 around each axis in the global coordinate system. The three directions newly obtained by tilting each around the axis are set as the pitch direction (u-axis), the yaw direction (v-axis), and the roll direction (w-axis) of the uvw field coordinate system in the HMD 110.

In some embodiments, when the user 190 wearing the HMD 110 is upright and visually recognizing the front, the processor 202 sets the uvw field coordinate system parallel to the global coordinate system to the HMD 110. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and front-back direction (z-axis) in the global coordinate system are the pitch direction (u-axis) and yaw direction (v-axis) of the uvw field coordinate system in the HMD 110. And the roll direction (w-axis).

After the uvw visual field coordinate system is set to the HMD 110, the HMD sensor 120 can detect the inclination (change amount of the inclination) of the HMD 110 in the set uvw visual field coordinate system based on the movement of the HMD 110. In this case, the HMD sensor 120 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 110 in the uvw visual field coordinate system as the inclination of the HMD 110, respectively. The pitch angle (θu) represents the tilt angle of the HMD 110 around the pitch direction in the uvw visual field coordinate system. The yaw angle (θv) represents the tilt angle of the HMD 110 around the yaw direction in the uvw visual field coordinate system. The roll angle (θw) represents the tilt angle of the HMD 110 around the roll direction in the uvw visual field coordinate system.

The HMD sensor 120 sets the uvw field coordinate system in the HMD 110 after the HMD 110 has moved to the HMD 110 based on the detected tilt angle of the HMD 110. The relationship between the HMD 110 and the uvw field coordinate system of the HMD 110 is always constant regardless of the position and inclination of the HMD 110. When the position and inclination of the HMD 110 change, the position and inclination of the uvw visual field coordinate system of the HMD 110 in the global coordinate system change in conjunction with the change of the position and inclination.

In some embodiments, the HMD sensor 120 is based on the infrared light intensity acquired based on the output from the infrared sensor and the relative positional relationship between the points (eg, the distance between the points). The position of the light in the real space may be specified as a relative position with respect to the HMD sensor 120. Further, the processor 202 may determine the origin of the uvw visual field coordinate system of the HMD 110 in the real space (global coordinate system) based on the specified relative position.

The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually representing one aspect of expressing the virtual space 400 according to a certain embodiment. The virtual space 400 has an all-sky spherical structure that covers the entire center 406 in the 360-degree direction. In FIG. 4, the celestial sphere in the upper half of the virtual space 400 is illustrated so as not to complicate the explanation. Each mesh is defined in the virtual space 400. The position of each mesh is predetermined as a coordinate value in the XYZ coordinate system defined in the virtual space 400. The computer 200 associates each partial image constituting the content (still image, moving image, etc.) expandable in the virtual space 400 with each corresponding mesh in the virtual space 400, and creates a virtual space image visible to the user. The virtual space 400 to be expanded is provided to the user.

In some embodiments, the virtual space 400 defines an xyz coordinate system with the center 406 as the origin. The xyz coordinate system is, for example, parallel to the global coordinate system. Since the xyz coordinate system is a kind of viewpoint coordinate system, the horizontal direction, the vertical direction (vertical direction), and the front-back direction in the xyz coordinate system are defined as the x-axis, the y-axis, and the z-axis, respectively. Therefore, the x-axis (horizontal direction) of the xyz coordinate system is parallel to the x-axis of the global coordinate system, and the y-axis (vertical direction) of the xyz coordinate system is parallel to the y-axis of the global coordinate system. The z-axis (front-back direction) is parallel to the z-axis of the global coordinate system.

When the HMD 110 is activated, that is, in the initial state of the HMD 110, the virtual camera 404 is arranged at the center 406 of the virtual space 400. In some embodiments, the processor 202 displays an image captured by the virtual camera 404 on the display unit 112 of the HMD 110. The virtual camera 404 moves in the virtual space 400 in the same manner in conjunction with the movement of the HMD 110 in the real space. As a result, changes in the position and orientation of the HMD 110 in the real space can be similarly reproduced in the virtual space 400.

As in the case of the HMD 110, the virtual camera 404 is defined with an uvw field-of-view coordinate system. The uvw field-of-view coordinate system of the virtual camera 404 in the virtual space 400 is defined to be linked to the uvw field-of-view coordinate system of the HMD 110 in the real space (global coordinate system). Therefore, when the inclination of the HMD 110 changes, the inclination of the virtual camera 404 also changes accordingly. Further, the virtual camera 404 can also move in the virtual space 400 in conjunction with the movement of the user wearing the HMD 110 in the real space.

The processor 202 of the computer 200 defines the viewing area 410 in the virtual space 400 based on the arrangement position of the virtual camera 404 and the reference line of sight 408. The viewing area 410 corresponds to the area of the virtual space 400 that is visually recognized by the user wearing the HMD 110.

The line of sight of the user 190 detected by the gaze sensor 140 is a direction in the viewpoint coordinate system when the user 190 visually recognizes an object. The uvw field-of-view coordinate system of the HMD 110 is equal to the viewpoint coordinate system when the user 190 visually recognizes the display unit 112. Further, the uvw field-of-view coordinate system of the virtual camera 404 is linked to the uvw field-of-view coordinate system of the HMD 110. Therefore, the HMD system 100 according to a certain aspect can consider the line of sight of the user 190 detected by the gaze sensor 140 as the line of sight of the user in the uvw field of view coordinate system of the virtual camera 404.

The determination of the line of sight of the user will be described with reference to FIG. FIG. 5 is a top view of the head of the user 190 wearing the HMD 110 according to an embodiment.

In some embodiments, the gaze sensor 140 detects each line of sight of the user 190's right and left eyes. In some embodiments, the gaze sensor 140 detects gazes R1 and L1 when the user 190 is looking closer. In another embodiment, when the user 190 is looking far away, the gaze sensor 140 detects the lines of sight R2 and L2. In this case, the angle formed by the lines of sight R2 and L2 with respect to the roll direction w is smaller than the angle formed by the lines of sight R1 and L1 with respect to the roll direction w. The gaze sensor 140 transmits the detection result to the computer 200.

When the computer 200 receives the detection values of the line of sight R1 and L1 from the gaze sensor 140 as the detection result of the line of sight, the computer 200 identifies the gaze point N1 which is the intersection of the line of sight R1 and L1 based on the detected values. On the other hand, when the computer 200 receives the detected values of the lines of sight R2 and L2 from the gaze sensor 140, the computer 200 identifies the intersection of the lines of sight R2 and L2 as the gaze point. The computer 200 identifies the line of sight N0 of the user 190 based on the position of the identified gazing point N1. The computer 200 detects, for example, the extending direction of the straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 190 and the gazing point N1 as the line of sight N0. The line of sight N0 is the direction in which the user 190 actually directs the line of sight with both eyes. Further, the line of sight N0 corresponds to the direction in which the user 190 actually directs the line of sight with respect to the viewing area 410.

In another aspect, the HMD system 100 may include a microphone and a speaker in any part constituting the HMD system 100. The user can give a voice instruction to the virtual space 400 by speaking to the microphone.

In another aspect, the HMD system 100 may include a television broadcast receiving tuner. According to such a configuration, the HMD system 100 can display a television program in the virtual space 400.

In still another aspect, the HMD system 100 may include a communication circuit for connecting to the Internet or a telephone function for connecting to a telephone line.

The viewing area 410 will be described with reference to FIGS. 6 and 7. FIG. 6 is a diagram showing a yz cross section of the visual recognition area 410 viewed from the x direction in the virtual space 400. FIG. 7 is a diagram showing an xz cross section of the visual recognition area 410 viewed from the y direction in the virtual space 400.

As shown in FIG. 6, the visible region 410 in the yz cross section includes the region 602. The area 602 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the yz cross section of the virtual space 400. The processor 202 defines a range including the polar angle α centered on the reference line of sight 408 in the virtual space as a region 602.

As shown in FIG. 7, the visible region 410 in the xz cross section includes the region 702. The area 702 is defined by the arrangement position of the virtual camera 404, the reference line of sight 408, and the xz cross section of the virtual space 400. The processor 202 defines a range including the azimuth angle β centered on the reference line of sight 408 in the virtual space 400 as the region 702. The polar angles α and β are determined according to the arrangement position of the virtual camera 404 and the orientation of the virtual camera 404.

In some embodiments, the HMD system 100 provides the user 190 with a field of view in virtual space by displaying a field of view image on the display unit 112 based on a signal from the computer 200. The visual field image corresponds to a portion of the virtual space image 402 that is superimposed on the visual field area 410. When the user 190 moves the HMD 110 attached to the head, the virtual camera 404 also moves in conjunction with the movement. As a result, the position of the visible area 410 in the virtual space 400 changes. As a result, the field-of-view image displayed on the display unit 112 is updated to an image superimposed on the visual-view area 410 in the direction facing the user in the virtual space 400 among the virtual space images 402. The user can visually recognize the desired direction in the virtual space 400.

As described above, the direction (tilt) of the virtual camera 404 corresponds to the user's line of sight (reference line of sight 408) in the virtual space 400, and the position where the virtual camera 404 is arranged corresponds to the user's viewpoint in the virtual space 400. Therefore, by moving the virtual camera 404 (including the operation of changing the arrangement position and the operation of changing the orientation), the image displayed on the display unit 112 is updated, and the field of view (including the viewpoint and the line of sight) of the user 190 is moved. Will be done.

While wearing the HMD 110, the user 190 can visually recognize only the virtual space image 402 developed in the virtual space 400 without visually recognizing the real world. Therefore, the HMD system 100 can give the user a high sense of immersion in the virtual space 400.

In some embodiments, the processor 202 may move the virtual camera 404 in the virtual space 400 in conjunction with the movement of the user 190 wearing the HMD 110 in the real space. In this case, the processor 202 identifies an image area (that is, a viewing area 410 in the virtual space 400) projected onto the display unit 112 of the HMD 110 based on the position and orientation of the virtual camera 404 in the virtual space 400.

According to certain embodiments, the virtual camera 404 may include two virtual cameras, a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. Further, appropriate parallax may be set in the two virtual cameras so that the user 190 can recognize the three-dimensional virtual space 400.

The controller 160A, which is an example of the controller 160, will be described with reference to FIG. 8A. FIG. 8A is a diagram showing a schematic configuration of a controller 160A according to an embodiment.

In some embodiments, the controller 160A may include a right controller and a left controller. For simplicity of explanation, controller 160A in FIG. 8A shows the right controller. The right controller is operated by the right hand of the user 190. The left controller is operated by the left hand of the user 190. In some embodiments, the right controller and the left controller are symmetrically configured as separate devices. Therefore, the user 190 can freely move the right hand holding the right controller and the left hand holding the left controller. In another embodiment, the controller 160 may be an integrated controller that accepts operations of both hands. Hereinafter, the right controller 160A will be described.

The right controller 160A includes a grip 802, a frame 804, and a top surface 806. The grip 802 is configured to be gripped by the right hand of the user 190. For example, the grip 802 may be held by the palm of the user 190's right hand and three fingers (middle finger, ring finger, little finger).

The grip 802 includes buttons 808 and 810 and a motion sensor 130. The button 808 is arranged on the side surface of the grip 802 and accepts an operation by the middle finger of the right hand. The button 810 is arranged in front of the grip 802 and accepts an operation by the index finger of the right hand. In some embodiments, the buttons 808, 810 are configured as trigger-type buttons. The motion sensor 130 is built in the housing of the grip 802. If the operation of the user 190 can be detected from around the user 190 by a camera or other device, the grip 802 may not include the motion sensor 130.

The frame 804 includes a plurality of infrared LEDs 812 arranged along its circumferential direction. The infrared LED 812 emits infrared rays as the program progresses during the execution of the program using the controller 160A. The infrared rays emitted from the infrared LED 812 can be used to detect each position and orientation (tilt, orientation) of the right controller 160A and the left controller (not shown). In the example shown in FIG. 8, infrared LEDs 812 arranged in two rows are shown, but the number of arrays is not limited to that shown in FIG. An array of one column or three or more columns may be used.

The top surface 806 includes buttons 814 and 816 and an analog stick 818. Buttons 814 and 816 are configured as push buttons. Buttons 814 and 816 accept operations by the thumb of the user 190's right hand. In some embodiments, the analog stick 818 accepts an operation 360 degrees in any direction from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual space 400.

In some embodiments, the right controller 160A and the left controller include a battery for driving a member such as an infrared LED 812. The battery may be either a primary battery or a secondary battery, and the shape thereof may be arbitrary such as a button type and a dry battery type. In another embodiment, the right controller 160A and the left controller can be connected, for example, to the USB interface of computer 200. In this case, the right controller 800 and the left controller may be powered via the USB interface.

The controller 160B, which is an example of the controller 160, will be described with reference to FIG. 8B. FIG. 8B is a diagram showing a schematic configuration of a controller 160B according to an embodiment.

The controller 160B includes a plurality of buttons 820 (820a, 820b, 820c, 820d) and 822 (822a, 822b, 822c, 822d), and left and right analog sticks 824L and 824R. Each button 820 and 822 is configured as a push button. Buttons 820 and 822 accept operations by the thumb of the user 190's hand. A trigger-type button (not shown) capable of accepting an operation by the index finger or the middle finger of the user 190's hand may be further provided in the controller 160B. In some embodiments, the analog sticks 824L and 824R accept operations in any direction 360 degrees from the initial position (neutral position), respectively. The operation includes, for example, an operation for moving an object arranged in the virtual space 400. Separate operations for buttons 820 (820a, 820b, 820c, 820d) and 822 (822a, 822b, 822c, 822d) and analog sticks 824L and 824R (and, if included, trigger-type buttons (not shown)). A command is assigned. The operation commands include, for example, a command for giving a command to an object in the virtual space 400, a command for making various settings on a game menu screen, and a computer when the user 190 is experiencing the virtual space 400. Includes any other command that can be entered in 200. The operation command assigned to each button or analog stick may be dynamically changed according to, for example, the progress of the game or the change of the scene.

In some embodiments, the controller 160B may include a plurality of infrared LEDs (not shown) located on its outer surface. The infrared LED emits infrared rays as the program progresses during the execution of the program using the controller 160B. The infrared rays emitted from the infrared LEDs can be used to detect the position and orientation (tilt, orientation) of the controller 160B. Further, the controller 160B includes a battery for driving an internal electronic component. The battery may be either a primary battery or a secondary battery, and the shape thereof may be arbitrary such as a button type and a dry battery type. In another embodiment, the controller 160B may be connected to, for example, the USB interface of the computer 200. In this case, the controller 160B may be powered via the USB interface.
FIG. 9 is a block diagram showing a function of the computer 200 for providing a virtual experience to a user via the HMD system 100 according to an embodiment of the present disclosure. The computer 200 executes various processes mainly based on the inputs from the HMD sensor 120, the motion sensor 130, the gaze sensor 140, and the controller 160.

The computer 200 includes a processor 202, a memory 204, and a communication control unit 205. The processor 202 includes a virtual space identification unit 902, an HMD motion detection unit 904, a line-of-sight detection unit 906, a reference line-of-sight determination unit 908, a field of view area determination unit 910, a virtual viewpoint identification unit 912, and a field of view image generation unit 914. A virtual camera control unit 916, an event generation unit 918, a visibility degree specifying unit 920, a condition determination unit 922, an object control unit 923, and a field of view image output unit 924 may be included. The memory 204 may be configured to store various information. In one example, the memory 204 may include virtual space data 926, object data 928, application data 930, event data 932, visibility condition 934, behavior data 936, and other data 938. The memory 204 also includes various data necessary for the calculation for providing the output information corresponding to the input from the HMD sensor 120, the motion sensor 130, the gaze sensor 140, the controller 160, etc. to the display unit 112 associated with the HMD 110. It may be. The object data 928 may include data about various objects arranged in the virtual space. Event data 932 may include various data about events occurring in the virtual space. The event may occur as the scenario progresses in the virtual space. The visibility condition 934 may include information related to determining whether or not the user is visually recognizing the event (for example, the number of objects and the number of feature points, which will be described later, required to satisfy the visibility). The behavior data 936 may include data about the behavior performed by the objects in the virtual space to guide the user's attention. The display unit 112 may be built in the HMD 110, or may be a display of another device (for example, a smartphone) that can be attached to the HMD 110. In embodiments of the present disclosure, the HMD 110 may include various devices configured as image display devices associated with the user's head.

In FIG. 9, the component included in the processor 202 is only one example of expressing the function executed by the processor 202 as a concrete module. The functionality of multiple components may be achieved by a single component. Processor 202 may be configured to perform the functions of all components.

FIG. 10 is a flow chart of a general process for displaying an image of a virtual space in which a user is immersed on the display unit 112.

With reference to FIGS. 9 and 10, a general process of the HMD system 100 for providing an image of the virtual space will be described. The virtual space 400 can be provided by the interaction of the HMD sensor 120, the gaze sensor 140, the computer 200, and the like.

The process starts in step 1002. As an example, the game application included in the application data 930 may be executed by the computer 200. In step 1004, the processor 202 (or the virtual space identification unit 902) generates a spherical virtual space image 402 constituting the virtual space 400 in which the user is immersed by referring to the virtual space data 926 or the like. The position and tilt of the HMD 110 are detected by the HMD sensor 120. The information detected by the HMD sensor 120 is transmitted to the computer 200. In step 1006, the HMD motion detection unit 904 acquires the position information, tilt information, and the like of the HMD 110. In step 1008, the viewing direction is determined based on the acquired position information and inclination information.

When the gaze sensor 140 detects the movement of the eyeballs of the user's left and right eyes, the information is transmitted to the computer 200. In step 1010, the line-of-sight detection unit 906 specifies the directions in which the lines of sight of the right eye and the left eye are directed, and determines the line-of-sight direction N0. In step 1012, the reference line-of-sight determination unit 908 determines the field-of-view direction determined by the inclination of the HMD 110 or the user's line-of-sight direction N0 as the reference line-of-sight 408. The reference line of sight 408 may also be determined based on the position and tilt of the virtual camera 404 that follows the position and tilt of the HMD 110.

In step 1014, the field of view area determination unit 910 determines the field of view area 410 of the virtual camera 404 in the virtual space 400. As shown in FIG. 4, the field of view area 410 is a part of the virtual space image 402 that constitutes the user's field of view. The field of view 410 is determined based on the reference line of sight 408. The yz cross-sectional view of the field of view 410 viewed from the x direction and the xz cross section of the field of view 410 viewed from the y direction are shown in FIGS. 6 and 7, respectively, which have already been described.

In step 1016, the field of view image generation unit 914 generates a field of view image based on the field of view region 410. The field of view image includes two two-dimensional images, one for the right eye and one for the left eye. By superimposing these two-dimensional images on the display unit 112 (more specifically, the image for the right eye is output to the display unit for the right eye and the image for the left eye is output to the display unit for the left eye), the three-dimensional image is displayed. The virtual space 400 as an image is provided to the user. In step 1018, the field of view image output unit 924 outputs information about the field of view image to the display unit 112. The display unit 112 displays the field of view image based on the information of the received field of view image. The process ends in step 1020.

FIG. 11 is a flowchart of Method 1100 according to an embodiment of the present disclosure. Flow charts showing specific examples of the processes executed in steps 1110, 1112, 1116 and 1118 of FIG. 11 are shown in FIGS. 12, 15, 19, 19, 21 and 23, which will be described later. In one embodiment of the present disclosure, the computer program may cause the processor 202 (or computer 200) to perform each step shown in FIGS. 11, 12, 15, 19, 21, and 23. Another embodiment of the present disclosure may also be implemented as a computer comprising at least a processor and performing method 1100 under the control of the processor.

Hereinafter, embodiments of the present disclosure will be specifically described. Here, as a specific example to which the embodiment of the present disclosure can be applied, it is assumed that a user can immerse himself / herself in a virtual space in which an avatar, other objects, etc. associated with the user are arranged and enjoy the game. .. However, the embodiments of the present disclosure are not necessarily limited to such aspects. It will be apparent to those skilled in the art that embodiments of the present disclosure may take various aspects within the scope of the claims.

FIG. 13 is a diagram schematically showing how an event occurs during game play. Hereinafter, an embodiment of the present disclosure will be described with respect to a case where an event of "walking a dog with a girl" occurs as a specific example. However, the event is not limited to this type, and may be an event that occurs in various applications including RPG games, shooting games, and the like. For example, the event may be an event that includes various scenarios such as travel, tourism, lessons, and training.

User 1304 wears the HMD 110 on his head. In this example, user 1304 may operate controller 160B having the configuration shown in FIG. 8B. It will be understood that this is just one example of a controller. Various controllers can be applied to the embodiments of the present disclosure, including controllers of other embodiments that can be worn on the user's body, such as the controller 160A shown in FIG. 8A. In another example, the user 1304 may not be equipped with the controller, and a sensor not attached to the user 1304 may detect the movement of the user 1304. For example, the signal of the camera that captures the user 1304 may be input to the computer 200 as a signal representing the movement of the user 1304.

An avatar 1306 operated by the user 1304 is arranged in the virtual space 1300. During game play, the user 1304 can cause the avatar 1306 associated with the user 1304 to perform various actions by operating the controller 160B, moving the head on which the HMD 110 is attached, and the like.

As shown in FIG. 13, the virtual camera 1302 may be arranged at the position of the avatar 1306 (for example, the position of the eyes of the avatar 1306). The user 1304 can see the image of the virtual space 1300 obtained by the virtual camera 1302 (from the viewpoint of the avatar 1306). In this example, it is assumed that user 1304's avatar 1306 is in the vicinity of scene 1318, which includes objects such as girl 1310, dog 1312, lead 1314, kennel 1316, and any other background image.

Returning to FIG. 11, the process starts in step 1102. The processor 202 reads and executes the game program included in the application data 930 stored in the memory 204. The virtual space specifying unit 902 specifies the virtual space data for the executed game based on the virtual space data 926, the object data 928, and the like. Data related to the objects appearing in the avatar 1306 and the scene 1318 may be included in the object data 928, the application data 930 and the like. The virtual space data defines a virtual space that includes an avatar 1306 associated with user 1304. Data about the event may be included in the event data 932.

The process proceeds to step 1104, and the virtual camera control unit 916 moves the virtual camera 1302 in the virtual space 1300 based on the operation by the user 1304 (controls the field of view from the virtual viewpoint). The virtual camera 1302 may be moved by an operation on the controller 160B by the user 1304, a movement of the HMD 110 accompanying the movement of the head 190 of the user 1304, a movement of the virtual camera 1302 predetermined by the application, and the like.

The process proceeds to step 1106, and the event generation unit 918 generates an event (a walk event in this example) that occurs according to the progress of the game, etc., based on the event data 932, the object data 928, the application data 930, and the like.

In the example of FIG. 13, scene 1318 includes girl 1310, dog 1312, lead 1314 and kennel 1316. The direction of the line of sight of the avatar 1306, that is, the orientation of the virtual camera 1302 (or the orientation of the virtual viewpoint) is indicated by the arrow 1312.

The process proceeds to step 1108. The field of view image generation unit 914 generates a field of view image based on the virtual space data 926, the event data 932, the position and orientation of the virtual camera 1302, and the like. The field of view image is generated, for example, by the process already described in connection with FIG.

The process proceeds to step 1110, and the visibility degree specifying unit 920 specifies the visibility degree indicating how much the user 1304 is visually recognizing the event generated in step 1106.

The process proceeds to step 1112, and the condition determination unit 922 determines whether or not the visibility degree specified in step 1110 satisfies a predetermined condition based on the visibility degree condition 934 or the like.

FIG. 12 is a flowchart showing a specific example of the processing executed in steps 1110 and 1112.

In step 1202, the visibility degree specifying unit 920 calculates the number of objects included in the field of view among the objects appearing in the event generated in step 1106. As an example, whether or not an object is included in the field of view may be determined based on whether or not at least a predetermined portion of the object is included in the field of view image generated in step 1108.

FIG. 14 shows an example of a field of view image provided to the user with respect to the scene 1318 shown in FIG. In FIG. 14, the objects 1310, 1312, and 1316 are set to have rectangular areas 1402, 1404, and 1406 indicated by dotted lines, which are determined by the coordinates of the four corners. Setting such a rectangular area is only an example of a method for determining whether or not an object is included in the field of view. Those skilled in the art will appreciate that various techniques can be used. Although not shown, regions may also be defined for leads 1314.

If the field of view image 1410 is generated by step 1108, no objects in the scene 1318 related to the event are included in the field of view image 1410. That is, at this time, the line of sight of the avatar 1306 is not directed in the direction of the event in which it is occurring. In this case, in step 1202, the visibility degree specifying unit 920 determines that none of the objects (or the areas 1402, 1404 and 1406 set for each object) is included in the field of view, and the objects in the field of view. Identify that the number of is zero. In step 1202, the visibility degree specifying unit 920 may determine that the object is included in the field of view only when the entire area of the object is included in the field of view. Alternatively, the visibility degree specifying unit 920 may determine that the object is included in the field of view even when only a predetermined part of the area of the object is included in the field of view.

In another example, if the visibility image 1412 is generated by step 1108, the girl 1310, the dog 1312 and the lead 1314 are in sight and the kennel 1316 is not in sight. At this time, the visibility degree specifying unit 920 specifies that the number of objects in the field of view is three. In yet another example, when the field of view image 1414 is generated, the visibility degree specifying unit 920 determines that the girl 1310 is included in the field of view, and specifies that the number of objects in the field of view is 1. May be good. In yet another example, when the field of view image 1416 is generated, the visibility degree specifying unit 920 determines that the kennel 1316 and the girl 1310 are included in the field of view, and the number of objects in the field of view is 2. May be specified.

The above-mentioned processing with respect to FIG. 14 has been described as being performed using a two-dimensional field-of-view image as an example. However, the process may be executed in a three-dimensional virtual space. For example, the above processing may be performed before the visual field image is generated, with v in the uvw visual field coordinate system as the vertical axis and u as the horizontal axis. Further, for example, it may be determined whether or not each object is included in the field of view by determining the collision between the field of view and the object. In this case, the field of view is provided with a collision area corresponding to the shape of the field of view, and the object is also provided with a collision area having a shape that covers at least a part of the object. Whether or not each object is included in the field of view may be specified by the presence or absence of a collision between the collision area of the field of view and the collision area of each object.

Returning to FIG. 12, the process proceeds to step 1204. In step 1204, the condition determination unit 922 determines whether or not the number of objects in the field of view calculated in step 1202 is a predetermined number J or more. The predetermined number J may be stored in the visibility condition 934. For example, in the example of FIG. 14, a predetermined number J = 2 may be set.

As in the case of the field of view images 1412 and 1416, when the number of objects in the field of view is a predetermined number J or more (“Y” in step 1204), the process proceeds to step 1206. The visibility degree specifying unit 920 determines that the visibility degree of the event satisfies the condition (“Y” in step 1112 in FIG. 11). Therefore, the process of FIG. 11 ends.

On the other hand, when the number of objects in the field of view is less than a predetermined number J (“N” in step 1204) as in the case of the field of view images 1410 and 1414, the process proceeds to step 1208. The visibility degree specifying unit 920 determines that the visibility degree of the event does not satisfy the condition (“N” in step 1112 in FIG. 11). Therefore, the process of FIG. 11 proceeds to step 1114.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

Returning to FIG. 11, in step 1114, the object control unit 923 executes a process that suggests in advance that a guiding action for directing the user's attention to the event that the user wants to see is performed. The process may be a process of causing a predetermined object to perform a predetermined action (hereinafter, also referred to as “preliminary action”) in the virtual space. The data regarding the advance notice action may be stored in the action data 936. The advance notice action can take various forms in order to notify the user in advance that the position, orientation, etc. of the virtual viewpoint of the user 1304 will be changed regardless of the intention of the user 1304. For example, as a warning action, a voice including contents such as "Where are you looking?" And "I'm doing something over there, let's see." May be output and provided to the user. The voice may be associated with one or more character objects in the virtual space.

When the advance notice action is an output of voice, the voice may include various contents depending on the contents of the event. For example, in the case of a tourist information event that occurs in a travel-related application, voices such as "Look at your right hand" and "Look at the front left" may be output. In the case of a shopping event, a voice such as "Please look at the shelf on the right" may be output. In the case of an event such as a class or lesson, a voice such as "Please see the board on the right" may be output. By outputting such voice in advance, it is possible to inform the user 1304 that something is happening in a direction different from the direction in which the viewpoint of the user 1304 in the virtual space is facing. Further, the advance notice action may include the display of an image, a moving image, or the like in addition to or in place of the audio output. Further, the content of the audio may differ depending on the degree of dissociation between the direction of the viewpoint of the user 1304 in the virtual space and the direction of the event. For example, if the degree of dissociation is large, the voice may be provided with harsh words, harsh tones, and the like. If the degree of dissociation is small, the audio may be provided with gentle language, gentle tone, and so on. As an example, when the process of FIG. 12 is executed in steps 1110 and 1112, the degree of dissociation is determined according to how small the number of objects in the field of view is compared to the predetermined number J. May be good. The process of step 1114 is not essential in process 1100, but is an optional process. However, by the process of step 1114, the user 1304 can detect that his or her viewpoint can be changed in some way. Therefore, by executing the process of step 1114, it is possible to further reduce the VR sickness of the user 1304 as compared with the case where the process is not executed.

The process proceeds to step 1116, and the object control unit 923 causes the character object to perform a guiding action. Guidance behavior is an action to draw the user's attention to an event that the user wants to see. The guided action may be performed by one or more character objects in the virtual space that are not included in the event that is occurring. The guided action may also be performed by one or more character objects included in the event being generated. These character objects may be, for example, avatars, NPCs (non-player characters), etc. associated with other players.

FIG. 15 is a flowchart showing a specific example of the process executed in step 1116. In one example, the guided action moves the virtual camera 1302 (or virtual viewpoint) so that the first action in which one or more character objects cover the user's field of view and the degree of visibility of the event satisfy the conditions (virtual viewpoint). Includes a second action that triggers (controlling the field of vision from). The guided behavior is not limited to the above behavior and may include other behaviors.

In step 1502, the object control unit 923 causes the character object to perform the first action. For example, the object control unit 923 causes the character object to perform an operation of covering the user's field of view based on the data included in the action data 936. The operation may include, for example, an operation in which the character object covers the face, eyes, etc. of the avatar 1306 with the hand of the character object, and an operation in which the character object covers the face, eyes, etc. of the avatar 1306 using some means other than the hand. ..

16A to 16C schematically show the state of the processing performed in step 1502. Here, as an example, it is assumed that a character object that does not initially exist in the user's field of view covers the face of the avatar 1306 with both hands of the character object as the first action. Although the outside of the field of view image 1410 is not actually visible to the user, in FIGS. 16A to 16C, a part of the outside of the field of view image 1410 is also shown for ease of explaining the embodiment.

The examples of FIGS. 16A to 16C will be described with reference to FIGS. 13 and 14 which have already been described. In this example, the character object 1602 is a character object that does not appear in the scene 1318 shown in FIGS. 13 and 14. In the first stage of the first action, the character object 1602 may be placed outside the field of view (here, the field of view image 1410), away from the avatar 1306, as shown in FIG. 16A. As shown, the object control unit 923 may control the character object 1602 so that both hands 1604 are aligned and protrude forward in order to cover the user's face. Next, the object control unit 923 moves the character object 1602 to the left while approaching the avatar 1306 so that both hands 1604 are in the field of view. The object control unit 923 further keeps the character object 1602 approaching and moving to the left with respect to the avatar 1306. As a result, as shown in FIG. 16C, the central portion of both hands 1604 covers most of the field of view. That is, at this time, both hands 1604 are located at a place that covers the field of view of the virtual camera 1302. Therefore, the user 1304 has the experience of being blindfolded in the virtual space by the character object 1602. In the example of FIG. 16C, the field of view is not completely covered by both hands 1604, and the background portion 1610 is visible. In another example, the character object 1602 may be controlled to completely cover the field of view.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

Unlike the examples shown in FIGS. 16A to 16C, the first action may be performed by one hand of the character object 1602. Further, any character object such as another player's avatar may be made to perform the first action. The guided action, including the first action, may be performed by at least one of one or more character objects appearing in the event. In this case, the character object that performs the guiding action among the one or more character objects may have the same attributes as the other character objects of the one or more character objects. This makes it possible to prevent the user's immersive feeling in the game being executed or the like from being impaired. The guided action may be performed by a plurality of character objects. For example, two character objects may cover the face, eyes, etc. of the avatar 1306 with their hands.

Examples of common attributes described above may include at least one of gender, attire, group of affiliations and size. Examples of the affiliation group may include a family of girl 1310, a group or team to which the girl 1310 belongs, and the like. If the affiliation group is a family of girl 1310, the character object 1602 may be the mother of girl 1310. In this case, the voice output in step 1114 may include the content "My child is over there. Please accompany the dog for a walk." Examples of clothing may include school uniforms, corporate uniforms, common style clothing such as in music bands or entertainment groups.

The process proceeds to step 1504, and the object control unit 923 causes the character object to perform the second action. For example, the object control unit 923 may cause the character object to perform an operation of grasping and rotating the face of the avatar 1306 with both hands 1604 based on the data included in the action data 936. This second action is used as an opportunity to control the field of view from the virtual viewpoint so that the degree of visibility of the event satisfies a predetermined condition.

17A to 17C schematically show the state of the processing performed in step 1504. The outside of the field of view image 1410 is not actually visible to user 1304, but is shown here for clarity. FIG. 17A is substantially the same as FIG. 16C, showing a state in which the character object covers the user's face with both hands 1604 as a result of the first action. FIG. 17B shows how the character object rotates the face of the avatar 1306 to the left. In detail, the state when the virtual camera 1302 is rotated counterclockwise on the Y axis is shown. As shown, the object control unit 923 moves both hands 1604 to the left. At this time, the object control unit 923 may change the form of both hands 1604 in order to grab the face of the avatar 1306. This second action can give the user 1304 a suggestion that the direction of his line of sight can be moved or rotated to the left. As another example, FIG. 17C shows how the character object rotates the face of the avatar 1306 upward. In detail, a state when the virtual camera 1302 is rotated counterclockwise on the X-axis is shown. Although a small part of the background is visible in FIGS. 17A to 17C, the character object may be controlled so that both hands 1604 completely cover the field of view. According to the present embodiment, since the viewpoint of the avatar 1306 is moved in a state where a small part of the background can be seen, the direction of one's line of sight is moved or the direction of one's line of sight is moved without making the user 1304 excessively feel the change in the field of view. It is possible to give the user the feeling of being rotated. As a result, VR sickness can be reduced while maintaining the user's immersive feeling in the game. In addition, since the images of both hands 1604 in substantially the same state occupy most of the field of view image 1410 during the second action, the amount of data required to generate the field of view image 1410 can be reduced.

The second action may end when the rotation of the face of the avatar 1306 by both hands 1604 is completed as a result of the actions shown in FIGS. 17B and 17C. In another example, after the face rotation of the avatar 1306 is complete, the object control unit 923 may control the character object 1602 to move to another location in the virtual space. For example, the character object 1602 may move to the position shown in FIG. 16A after the rotation shown in FIG. 17B is completed. As a result, the character object 1602 can be made to take a natural action close to the actual experience, and the immersive feeling of the user 1304 in the game can be further improved.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

Returning to FIG. 11, the process proceeds to step 1118. While the guidance action (for example, the second action of step 1504) is being performed in step 1116, the virtual camera control unit 916 is based on the virtual space data 926, the object data 928, the event data 932, the visibility condition 934, and the like. Then, the virtual camera (or the virtual viewpoint of the user 1304) is moved so that the degree of visibility of the event satisfies a predetermined condition (the view from the virtual viewpoint is controlled).

FIG. 18 is a flowchart showing a specific example of the process executed in step 1118. In one example, in step 1802, the virtual camera control unit 916 identifies a reference point for the scene 1318 that should be placed in the center of the field of view image 1410.

FIG. 19 is a diagram similar to FIG. 14, and is a diagram for explaining a process of moving the virtual camera 1302 (virtual viewpoint of the user 1304) (controlling the field of view from the virtual viewpoint) in step 1118. In FIG. 19, the center 1902 of the view image 1410 (indicated by a black circle), the reference point 1904 of the scene 1318 (indicated by a cross), and the predetermined angular range 1906 (indicated by a dotted circle) based on the reference point 1904 are shown. Has been done. These pieces of information may be set for each event as data for satisfying the condition of the degree of visibility of the event, or may be stored in advance in the degree of visibility condition 934. The reference point 1904 may be set to a major character (eg, at or near the center point of the girl 1310) or at a position between the girl 1310 and the dog 1312, depending on the event. good. The angle range 1906 may be set as an allowable angle formed by the reference line of sight as compared with the direction when the reference line of sight of the virtual camera 1302 is facing the reference point 1904.

The process proceeds to step 1804, and in one example, the virtual camera control unit 916 moves the virtual camera 1302 so that the center point 1902 of the view image 1410 coincides with the reference point 1904 of the scene 1318 (controls the view from the virtual viewpoint). ). In this case, the virtual camera control unit 916 may calculate a vector from the coordinates of the center point 1902 to the coordinates of the reference point 1904, and move the virtual camera 1302 based on this. Alternatively, the virtual camera control unit 916 may move the virtual camera 1302 so that the center point 1902 of the view image 1410 is included in the angle range 1906.

FIG. 20 shows the view image 2002 when the virtual camera 1302 is moved so that the center point 1902 of the view image 1410 coincides with the reference point 1904. As the virtual camera 1302 moves, the vision image provided to the user 1304 changes from the vision image 1410 to the vision image 2002. Although omitted in FIGS. 19 and 20, most of the field of view is covered by both hands 1604 or the like during the transition from the field of view image 1410 to the field of view image 2002, as described in connection with FIGS. 17A-17C. Please note that it will be done.

In the examples shown in FIGS. 18, 19 and 20, processing is performed so that the center point 1902 of the visual field image coincides with the reference point of the scene 1318. However, instead of such processing, a predetermined area (for example, rectangular area 1402) surrounding the main character (for example, girl 1310) appearing in the scene 1318, at least a part of the area occupied by the main character, or the area concerned. The virtual camera 1302 may be moved so that the central portion is included in the view image.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

Returning to FIG. 11, the process proceeds to step 1120. In step 1120, the field of view image generation unit 914 generates the field of view image acquired by the virtual camera 1302 moved in step 1118. The field of view image output unit 924 outputs the generated field of view image to the display unit 112 associated with the HMD 110. By this process, the field of view image satisfying the condition of the visibility degree of the event is provided to the user 1304, so that the user 1304 can fully see the event occurring in the virtual space.

As described above, according to the present embodiment, when the user's virtual viewpoint is not oriented in the direction of the event, the field of view from the virtual viewpoint can be controlled by a method that does not make the user feel unnatural. Therefore, it is possible to show the event to the user by controlling the field of view from the virtual viewpoint while suppressing VR sickness without impairing the user's immersive feeling in the game or the like.

FIG. 21 is a flowchart showing a specific example of the processing executed in steps 1110 and 1112 of FIG. In this example, it is determined whether or not the visibility degree of the event in the field of view satisfies a predetermined condition by using the feature points set for the object appearing in the event.

In step 2102, the visibility degree specifying unit 920 includes the feature points included in the field of view image generated in step 1108 among the feature points preset for the object appearing in the event generated in step 1106 of FIG. To identify.

FIG. 22 is a diagram for explaining the relationship between the feature points set on the object and the field of view image. In one example, the feature points are feature points 2202, 2204, 2206 and 2208 set for the face, waist, hands and feet of the girl 1310, feature points 2210 set for the head of the dog 1312, kennel 1316. It may include the feature point 2212 set for. The data of these feature points may be stored in the object data 928, the event data 932, and the like. The number of feature points set for each object may vary, and the position where the feature points are set may be changed as necessary. In addition, feature points may be set for objects included in the background other than the above objects.

In the example of the scene 1318 shown in FIG. 22, in the case of the field of view image 2220, there are no feature points in the field of view. In the case of the field of view image 2222, the feature point in the field of view is the feature point 2210 of the dog's head. In the case of the field of view image 2224, the feature points in the field of view are the facial feature points 2202 of the girl 1310. In the case of the field of view image 2226, the feature points in the field of view are the feature points 2212 of the kennel and the feature points 2208 of the legs of the girl 1310. In the case of the field of view image 2228, the feature points in the field of view are all the feature points set in the girl 1310 and the dog 1312.

The process proceeds to step 2104, and the condition determination unit 922 determines whether or not the feature points specified in step 2102 include a predetermined feature point. Here, the predetermined feature points may be arbitrarily set for each event. For example, in the event of the example of FIG. 22, when the object that the user particularly wants to see is the girl 1310, the predetermined feature points are the face, waist, hands and feet feature points 2202, 2204, 2206 and the girl 1310. It may contain at least one of 2208. In this example, when the field of view image 2220 or 2222 is provided to the user 1304, the process proceeds to step 2106 because the feature points in the field of view do not include the predetermined feature points (“N” in step 2104). In step 2106, the visibility degree specifying unit 920 determines that the visibility degree of the event does not satisfy the predetermined condition. On the other hand, when the field of view image 2224, 2226 or 2228 is provided to the user 1304, the process proceeds to step 2108 because the feature points in the field of view include the predetermined feature points (“Y” in step 2104). In step 2108, the visibility degree specifying unit 920 determines that the visibility degree of the event satisfies the condition. According to the present embodiment, since the feature points set for each object are used for determining the visibility degree of the event, the visibility degree can be determined with various accuracy depending on how the feature points are set.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

FIG. 23 is a flowchart showing a specific example of the processing executed in steps 1110 and 1112 of FIG. In this example, a value based on the number of specified objects over a predetermined period is used to determine the degree of visibility.

In step 2302, the visibility degree specifying unit 920 calculates the number OB of the objects appearing in the visual field image generated in step 1108 among the objects appearing in the event generated in step 1106 of FIG. A timer may be set in step 2302 to start the measurement for a predetermined period T (eg, 5 seconds). In the example of the scene 1318 shown in FIG. 14, OB = 0 in the case of the view image 1410, OB = 3 in the case of the view image 1412, and OB = 2 in the case of the view image 1416.

The process proceeds to step 2304, and the condition determination unit 922 (or another component) determines whether or not the predetermined period T has elapsed based on the time measured by the timer. When the predetermined period T has not elapsed (“N” in step 2304), the process returns to before step 2302. When the predetermined period T has elapsed (“Y” in step 2304), the process proceeds to step 2306.

In step 2306, the visibility degree specifying unit 920 calculates a value V based on the number OB of objects appearing in the field of view within the predetermined period T. In one example, the value V may be the average value of the number of objects OB appearing in the field of view in each of the short time periods included in the predetermined period T.

FIG. 24 is a diagram showing an example of the number of objects OB in the field of view obtained within the predetermined period T and the value V based on the number OB. In this example, it is assumed that _{n OBs (OB 1 to} OB n) are obtained within a predetermined period T, and the average value thereof is calculated as the value V. Specifically, in Example 1 of FIG. 24, OB _{1 to OB n} are all 0, and therefore V = 0. This means that the direction of the virtual viewpoint of the user 1304 does not face the event as in the case of the field of view image 1410 of FIG. 14 over a predetermined period T. In Example 2 of FIG. 24, OB _{1 to OB n} are 1, 2, 0, ..., And 1, respectively, and V = 1. This means that the virtual viewpoint of the user 1304 is facing the direction of the event at least for a certain period of time during the predetermined period T, and not only the view image 1410 but also the view images 1414, 1416 and the like are provided to the user 1304. Means. In Example 3 of FIG. 24, OBs _{1 to OB n} are 2, 3, 3, ..., And 3, respectively, and V = 3. This means that the virtual viewpoint of the user 1304 is oriented in the direction of the event for most of the predetermined period T, and the field image 1412 or a field image in a region close thereto is provided to the user 1304.

The process proceeds to step 2308, and the condition determination unit 922 determines whether or not the value V is larger than the predetermined value K. In this example, the predetermined value K = 2 may be set. The predetermined value K can be set to an arbitrary value according to the content of the event. When the value V is equal to or greater than the predetermined value K (“Y” in step 2308), the process proceeds to step 2310. At step 2310, the value V is cleared (eg, initialized to zero) and processing returns before step 2302. If the value V is less than the predetermined value K (“N” in step 2308), the process proceeds to step 2312. In step 2312, the visibility degree specifying unit 920 determines that the visibility degree of the event does not satisfy the predetermined condition. Therefore, in the example of FIG. 11, the processes of steps 1114 to 1120 are executed.

According to the embodiment of FIG. 23, it is possible to determine whether the user is continuously watching the event for a certain period of time, not whether the user is watching the event temporarily. Even when the virtual viewpoint of user 1304 is basically facing the direction of the event, the user may occasionally see a place other than the event, an object other than the main character object in the event, and the like. In such a case, according to the present embodiment, it is possible to prevent the virtual viewpoint from being forcibly directed toward the event when the user is looking away for a short period of time during the predetermined period T. .. Therefore, the user 1304 is not forced to see only the event during the occurrence of the event, and can see various places in the virtual space more freely. Therefore, it is possible to improve the user's immersive feeling in the game or the like.

In the embodiment of FIGS. 23 and 24, the process may be performed using the feature points in the field of view as shown in the examples of FIGS. 21 and 22 instead of the number of objects in the field of view. For example, in step 2302 of FIG. 23, the number of feature points appearing in the field of view among the feature points set for the object appearing in the event may be calculated. Then, in step 2306, the value V may be calculated based on the number of feature points appearing in the field of view within the predetermined period T.

The above process using a field of view image has been described as being performed using a two-dimensional field of view image as an example. However, the process may be executed by the method described above in the three-dimensional virtual space.

As described above, according to the embodiments of the present disclosure, VR sickness is reduced without compromising the user's virtual experience when the user's virtual viewpoint does not point in the direction of the event when the event occurs. At the same time, the user's virtual viewpoint can be directed to the event.

It will be apparent to those skilled in the art that embodiments of the present disclosure can be implemented as a processor 202 (or computer 200), method 1100, computer program causing processor 202 (or computer 200) to execute method 1100, and the like. Let's do it.

In the above-described embodiment, the virtual space (VR space) in which the user is immersed by the HMD device has been described as an example, but a transmissive HMD device may be adopted as the HMD device. In this case, augmented reality (AR) space or mixed reality is output by outputting a view image obtained by synthesizing a part of the images constituting the virtual space in the real space visually recognized by the user via the transmissive HMD device. A virtual experience in (MR: Mixed Reality) space may be provided to the user. In this case, instead of the hand object, the action on the target object in the virtual space may be generated based on the movement of the user's hand. Specifically, the processor may specify the coordinate information of the position of the user's hand in the real space, and may define the position of the target object in the virtual space in relation to the coordinate information in the real space. As a result, the processor can grasp the positional relationship between the user's hand in the real space and the target object in the virtual space, and can execute a process corresponding to collision control or the like between the user's hand and the target object. As a result, it becomes possible to give an action to the target object based on the movement of the user's hand.

Although embodiments of the present disclosure have been described, it should be understood that these are merely examples and do not limit the scope of the present disclosure. It should be understood that the embodiments can be changed, added, improved, etc. as appropriate without departing from the spirit and scope of the present disclosure. The scope of the present disclosure should not be limited by any of the embodiments described above, but should be defined by the claims and their equivalents.

100 ... HMD system, 112 ... Display, 114 ... Sensor, 120 ... HMD sensor, 130 ... Motion sensor, 140 ... Gaze sensor, 150 ... Server, 160A, 160B ... Controller, 190 ... Head, 192 ... Network, 200 ... Computer, 202 ... Processor, 204 ... Memory, 205 ... Communication control unit, 206 ... Storage, 208 ... Input / output interface, 210 ... Communication interface, 212 ... Bus, 400 ... Virtual space, 402 ... Virtual space image, 404 ... Virtual camera , 406 ... center, 408 ... reference line of sight, 410 ... view area, 800 ... right controller, 802 ... grip, 804 ... frame, 806 ... top surface, 808, 810, 814, 816 ... button, 812 ... infrared LED, 818 ... Analog stick, 820, 822 ... button, 824R, 824L ... analog stick, 902 ... virtual space identification unit, 904 ... HMD motion detection unit, 906 ... line-of-sight detection unit, 908 ... reference line-of-sight determination unit, 910 ... view area determination unit, 912 ... Virtual viewpoint identification unit, 914 ... Visibility image generation unit, 916 ... Virtual camera control unit, 918 ... Event generation unit, 920 ... Visibility identification unit, 922 ... Condition determination unit, 923 ... Object control unit, 924 ... Visibility image Output unit, 926 ... virtual space data, 928 ... object data, 930 ... application data, 932 ... event data, 934 ... visibility condition, 936 ... behavior data, 938 ... other data, 1300 ... virtual space, 1302 ... virtual camera , 1304 ... user, 1306 ... avatar, 1318 ... scene, 1310 ... girl, 1312 ... dog, 1314 ... lead, 1316 ... kennel, 1402, 1404, 1406 ... rectangular area, 1410, 1412, 1414, 1416, 2220, 2222 , 2224, 2226, 2228 ... Visibility image, 1602 ... Character object, 1604 ... Both hands, 1902, 2004 ... Center point, 1904 ... Reference point, 1906 ... Angle range 2202, 2204, 2206, 2208, 2210, 2212 ... Feature points

Claims (9)

To a computer for providing a virtual experience to the user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A program for executing a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed. The step of executing the process is
Including the step of outputting a voice suggesting that the action is performed.
The program according to claim 1, wherein the one or more character objects are one or more character objects associated with the voice. The step of executing the process is
As the action, a first action of covering the view with the one or more character objects and a second action of triggering the view to be controlled so that the degree of visibility of the event is satisfied. The program according to claim 1 or 2, wherein the process of causing the one or more character objects to be performed is executed. The event is an event in which one or more objects appear.
When a predetermined number or more of the one or more objects are not included in the field of view, the computer determines that the degree of visibility of the event in the field of view does not satisfy the condition. The program according to any one of claims 1 to 3, which is to be further executed by the computer. The event is an event in which one or more objects appear.
When a predetermined feature point among the one or more feature points preset for the one or more objects is not included in the field of view, the degree of visibility of the event in the field of view determines the condition. The program according to any one of claims 1 to 3, for causing the computer to further perform a step of determining that the condition is not satisfied. The event is an event in which one or more objects appear.
In the determination step, when the value based on the number of the one or more objects appearing in the field of view within a predetermined period is less than the predetermined number, the degree of visibility of the event in the field of view is the condition. The program according to claim 4 or 5 , which comprises a step of determining that the above is not satisfied. The event is an event in which at least one of the one or more character objects appears.
The character object that performs the action among the one or more character objects and the character object that appears in the event among the one or more character objects have common attributes.
The common attribute comprises at least one of gender, attire, affiliation and size.
The program according to any one of claims 1 to 6. It ’s a computer,
Controlled by a processor included in the computer to provide a virtual experience to the user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A computer that performs a step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed. A method performed by a computer to provide a virtual experience to a user via an image display device associated with the user's head.
The step of identifying the virtual space data that defines the virtual space for providing the virtual experience, and
A step of controlling the field of view from a virtual viewpoint in the virtual space based on the movement of the image display device,
The step of causing an event in the virtual space and
When the degree of visibility of the event in the field of view from the virtual viewpoint does not satisfy the condition, an action for directing the user's attention to the event is given to one or a plurality of character objects existing in the virtual space. The steps to perform the processing to be performed and
A method comprising the step of controlling the field of view so that the degree of visibility of the event in the field of view satisfies the condition while the action is being performed.

Images