JP6479933B1 - Program, information processing apparatus, and method - Google Patents

Abstract

A virtual experience of a user is further improved. When the relationship between the left hand (42A) and the left hand (42B) in the virtual space (2) satisfies the first condition, the processor, depending on the first condition, the left hand (42A), the left hand (42B), And at least one of the fields of view from the virtual camera (1) is controlled in the virtual space (2). [Selection] Figure 15

Description

  The present invention relates to a program, an information processing apparatus, and a method.

  Non-Patent Documents 1 and 2 disclose examples of techniques in which a plurality of users share the same virtual space.

Naoki Tsukamoto, “Social VR imagined by Facebook is comical”, [online], April 18, 2016, Gizmodo Japan, [Search September 22, 2017], Internet <https: //www.gizmodo.jp/2016/04/vr_363.html> Anonymous, "VR Gotcha ni LTkai in Virtual # 1", [online], February 17, 2016, YouTube (registered trademark), [Search September 22, 2017], Internet <https: // www .youtube.com / watch? v = 1L9Z6TWh_60>

  The conventional technology has room for further improving the virtual experience of the user.

  An object of one aspect of the present disclosure is to further improve a user's virtual experience.

  According to an aspect of the present invention, a program executed by a computer having a processor is provided to provide a virtual experience to the first user via an image display device associated with the head of the first user. Is done. The program defines, in the processor, a virtual space for providing a virtual experience to the first user, and moves the first operation object in the virtual space according to the movement of a part of the body of the first user. In accordance with the step, the step of controlling the view from the virtual viewpoint in the virtual space according to the posture of the head of the first user and the position of the virtual viewpoint in the virtual space, and according to the movement of a part of the body of the second user When the step of moving the second operation object in the virtual space and the relationship between the first operation object and the second operation object in the virtual space satisfy the first condition, the first operation object according to the first condition, Controlling at least one of the second operation object and the field of view from the virtual viewpoint in the virtual space; and viewing from the virtual viewpoint A step of defining field image corresponding to, and a step of outputting a visual image on an image display device.

  According to one aspect of the present disclosure, a user's virtual experience can be further improved.

It is a figure showing the outline of a structure of the HMD system 100 according to a certain embodiment. It is a block diagram showing an example of the hardware constitutions of the computer 200 according to one situation. It is a figure which represents notionally the uvw visual field coordinate system set to the HMD apparatus 110 according to an embodiment. It is a figure which represents notionally the one aspect | mode which represents the virtual space 2 according to a certain embodiment. It is the figure showing the head of user 190 wearing HMD device 110 according to a certain embodiment from the top. 3 is a diagram illustrating a YZ cross section of a visual field region 23 viewed from the X direction in a virtual space 2. FIG. 3 is a diagram illustrating an XZ cross section of a visual field region 23 viewed from a Y direction in a virtual space 2. FIG. It is a figure showing schematic structure of the controller 160 according to a certain embodiment. FIG. 2 is a block diagram showing a computer 200 according to an embodiment as a module configuration. It is a flowchart showing the process which HMD system 100A performs. It is a figure showing typically virtual space 2 shared by a plurality of users. It is a figure showing an example of the visual field image M provided to the user 190A. It is a sequence diagram which shows the process which HMD system 100A, HMD system 100B, HMD system 100C, and the server 150 perform. It is a flowchart regarding the interlocking control process of avatar 40A and avatar 40B. It is a figure which shows an example of the mode of the virtual space 2 when the 1st condition is satisfied. It is a figure which shows an example of the visual field image M displayed when the 1st condition is satisfied. It is a figure which shows each example of 1st conditions. It is a figure which shows the other example of 1st conditions. It is a figure which shows the other example of 1st conditions. It is a figure which shows the other example of the mode of the virtual space 2 in case the 1st condition is satisfied. It is a figure which shows the other example of the visual field image M displayed when the 1st condition is satisfied.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Configuration of HMD system]
A configuration of an HMD (Head Mount Device) system 100 will be described with reference to FIG. FIG. 1 is a diagram representing an outline of a configuration of an HMD system 100 according to an embodiment. In one aspect, the HMD system 100 is provided as a home system or a business system.

  The HMD system 100 includes an HMD device 110 (user terminal), an HMD sensor 120, a controller 160, and a computer 200. The HMD device 110 includes a display 112 (display unit), a camera 116, a microphone 118, and a gaze sensor 140. The controller 160 can include a motion sensor 130.

  In one aspect, the computer 200 can be connected to the Internet and other networks 19, and can communicate with the server 150 and other computers connected to the network 19. In another aspect, the HMD device 110 may include a sensor 114 instead of the HMD sensor 120.

  The HMD device 110 may be worn on the user's head and provide a virtual space to the user during operation. More specifically, the HMD device 110 displays a right-eye image and a left-eye image on the display 112, respectively. When each eye of the user visually recognizes each image, the user can recognize the image as a three-dimensional image based on the parallax of both eyes.

  The display 112 is realized as a non-transmissive display device, for example. In one aspect, the display 112 is disposed on the main body of the HMD device 110 so as to be positioned in front of both eyes of the user. Therefore, when the user visually recognizes the three-dimensional image displayed on the display 112, the user can be immersed in the virtual space. In one embodiment, the virtual space includes, for example, a background, an object that can be operated by the user, an image of a menu that can be selected by the user, and the like. In an embodiment, the display 112 may be realized as a liquid crystal display or an organic EL (Electro Luminescence) display included in a so-called smartphone or other information display terminal. The display 112 may be configured integrally with the main body of the HMD device 110 or may be configured as a separate body.

  In one aspect, the display 112 may include a sub-display for displaying an image for the right eye and a sub-display for displaying an image for the left eye. In another aspect, the display 112 may be configured to display a right-eye image and a left-eye image together. In this case, the display 112 includes a high-speed shutter. The high-speed shutter operates so that an image for the right eye and an image for the left eye can be displayed alternately so that the image is recognized only by one of the eyes.

  The camera 116 acquires a face image of the user wearing the HMD device 110. The face image acquired by the camera 116 can be used to detect the user's facial expression through image analysis processing. The camera 116 may be, for example, an infrared camera built in the main body of the HMD device 110 in order to detect pupil movement, eyelid opening / closing, eyebrow movement, and the like. Alternatively, the camera 116 may be an external camera disposed outside the HMD device 110 as shown in FIG. 1 in order to detect movements of the user's mouth, cheeks, and jaws. The camera 116 may be configured by both the infrared camera and the external camera described above.

  The microphone 118 acquires the voice uttered by the user. The voice acquired by the microphone 118 can be used to detect a user's emotion by voice analysis processing. The voice can also be used to give a voice instruction to the virtual space 2. Further, the sound may be sent to the HMD system used by another user via the network 19 and the server 150, and output from a speaker or the like connected to the HMD system. Thereby, the conversation (chat) between the users who share a virtual space is implement | achieved.

  The HMD sensor 120 includes a plurality of light sources (not shown). Each light source is realized by, for example, an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 120 has a position tracking function for detecting the movement of the HMD device 110. The HMD sensor 120 detects the position and inclination of the HMD device 110 in the real space using this function.

  In another aspect, HMD sensor 120 may be realized by a camera. In this case, the HMD sensor 120 can detect the position and inclination of the HMD device 110 by executing image analysis processing using image information of the HMD device 110 output from the camera.

  In another aspect, the HMD device 110 may include a sensor 114 instead of the HMD sensor 120 as a position detector. The HMD device 110 can detect the position and inclination of the HMD device 110 itself using the sensor 114. For example, when the sensor 114 is an angular velocity sensor, a geomagnetic sensor, an acceleration sensor, a gyro sensor, or the like, the HMD device 110 uses any one of these sensors instead of the HMD sensor 120 to detect its position and inclination. Can be detected. As an example, when the sensor 114 is an angular velocity sensor, the angular velocity sensor detects angular velocities around the three axes of the HMD device 110 in real space over time. The HMD device 110 calculates the temporal change of the angle around the three axes of the HMD device 110 based on each angular velocity, and further calculates the inclination of the HMD device 110 based on the temporal change of the angle. The HMD device 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting the transmittance. Further, the view field image may include a configuration for presenting the real space in a part of the image configuring the virtual space. For example, an image captured by a camera mounted on the HMD device 110 may be displayed so as to be superimposed on a part of the field-of-view image, or a part of the transmission-type display device may be set to have a high transmittance. The real space may be visible from a part of the image.

  The gaze sensor 140 detects a direction (gaze direction) in which the gaze of the right eye and the left eye of the user 190 is directed. The detection of the direction is realized by, for example, a known eye tracking function. The gaze sensor 140 is realized by a sensor having the eye tracking function. In one aspect, the gaze sensor 140 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 140 may be, for example, a sensor that irradiates the right eye and the left eye of the user 190 with infrared light and detects the rotation angle of each eyeball by receiving reflected light from the cornea and iris with respect to the irradiated light. . The gaze sensor 140 can detect the line-of-sight direction of the user 190 based on each detected rotation angle.

  Server 150 may send a program to computer 200. In another aspect, the server 150 may communicate with other computers 200 for providing virtual reality to HMD devices used by other users. For example, when a plurality of users play a participatory game in an amusement facility, each computer 200 communicates a signal based on each user's operation with another computer 200, and a plurality of users are common in the same virtual space. Allows you to enjoy the game. The server 150 can be composed of one or more computer devices. The server 150 may have a hardware configuration (processor, memory, storage, etc.) similar to the hardware configuration of the computer 200 described later.

  The controller 160 receives input of commands from the user 190 to the computer 200. In one aspect, the controller 160 is configured to be gripped by the user 190. In another aspect, the controller 160 is configured to be attachable to the body of the user 190 or a part of clothing. In another aspect, the controller 160 may be configured to output at least one of vibration, sound, and light based on a signal sent from the computer 200. In another aspect, the controller 160 receives an operation given by the user 190 in order to control the position and movement of an object arranged in the virtual space.

  In one aspect, the motion sensor 130 is attached to the user's hand and detects the movement of the user's hand. For example, the motion sensor 130 detects the rotation speed, rotation speed, etc. of the hand. The detected signal is sent to the computer 200. The motion sensor 130 is provided in a glove-type controller 160, for example. In some embodiments, for safety in real space, it is desirable that the controller 160 be mounted on something that does not fly easily by being mounted on the hand of the user 190, such as a glove shape. In another aspect, a sensor that is not worn by the user 190 may detect the hand movement of the user 190. For example, a signal from a camera that captures the user 190 may be input to the computer 200 as a signal representing the operation of the user 190. The motion sensor 130 and the computer 200 are connected to each other by wire or wirelessly. In the case of wireless communication, the communication form is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication methods are used.

[Hardware configuration]
A computer 200 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram illustrating an example of a hardware configuration of computer 200 according to one aspect. The computer 200 includes a processor 10, a memory 11, a storage 12, an input / output interface 13, and a communication interface 14 as main components. Each component is connected to the bus 15.

  The processor 10 executes a series of instructions included in the program stored in the memory 11 or the storage 12 based on a signal given to the computer 200 or based on the establishment of a predetermined condition. In one aspect, the processor 10 is realized as a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an FPGA (Field-Programmable Gate Array), or other device.

  The memory 11 temporarily stores programs and data. The program is loaded from the storage 12, for example. Data stored in the memory 11 includes data input to the computer 200 and data generated by the processor 10. In one aspect, the memory 11 is realized as a RAM (Random Access Memory) or other volatile memory.

  The storage 12 holds programs and data permanently. The storage 12 is realized as, for example, a ROM (Read-Only Memory), a hard disk device, a flash memory, and other nonvolatile storage devices. The programs stored in the storage 12 include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, a program for realizing communication with another computer 200, and the like. The data stored in the storage 12 includes data and objects for defining the virtual space.

  In another aspect, the storage 12 may be realized as a removable storage device such as a memory card. In still another aspect, a configuration using a program and data stored in an external storage device may be used instead of the storage 12 built in the computer 200. According to such a configuration, for example, in a scene where a plurality of HMD systems 100 are used like an amusement facility, it is possible to update programs and data in a batch.

  In some embodiments, the input / output interface 13 communicates signals with the HMD device 110, the HMD sensor 120, or the motion sensor 130. In one aspect, the input / output interface 13 is realized using a USB (Universal Serial Bus) interface, a DVI (Digital Visual Interface), an HDMI (registered trademark) (High-Definition Multimedia Interface), or other terminals. The input / output interface 13 is not limited to the above. For example, the input / output interface 13 may include a wireless communication interface such as Bluetooth (registered trademark).

  In certain embodiments, the input / output interface 13 may further communicate with the controller 160. For example, the input / output interface 13 receives an input of a signal output from the motion sensor 130. In another aspect, the input / output interface 13 sends an instruction output from the processor 10 to the controller 160. The command instructs the controller 160 to vibrate, output sound, emit light, and the like. When the controller 160 receives the command, the controller 160 executes vibration, sound output, or light emission according to the command.

  The communication interface 14 is connected to the network 19 and communicates with other computers (for example, the server 150) connected to the network 19. In one aspect, the communication interface 14 is, for example, a LAN (Local Area Network) or other wired communication interface, or WiFi (Wireless Fidelity (registered trademark)), Bluetooth (registered trademark), NFC (Near Field Communication), or other wireless communication. Realized as an interface. The communication interface 14 is not limited to the above.

  In one aspect, the processor 10 accesses the storage 12, loads one or more programs stored in the storage 12 into the memory 11, and executes a series of instructions included in the program. The one or more programs may include an operating system of the computer 200, an application program for providing a virtual space, game software that can be executed in the virtual space using the controller 160, and the like. The processor 10 sends a signal for providing a virtual space to the HMD device 110 via the input / output interface 13. The HMD device 110 displays an image on the display 112 based on the signal.

  The server 150 is connected to each control device of the plurality of HMD systems 100 via the network 19. In the example shown in FIG. 2, the server 150 connects a plurality of HMD systems 100 including an HMD system 100A having an HMD device 110A, an HMD system 100B having an HMD device 110B, and an HMD system 100C having an HMD device 110C to each other. Connect to enable communication. Thereby, the virtual experience using a common virtual space is provided to the user who uses each HMD system. The HMD system 100A, the HMD system 100B, the HMD system 100C, and the other HMD systems 100 all have the same configuration. However, each HMD system 100 may be a different model, or may have different performance (processing performance, detection performance related to detection of user action, etc.).

  In the example illustrated in FIG. 2, the configuration in which the computer 200 is provided outside the HMD device 110 is illustrated. However, in another aspect, the computer 200 may be incorporated in the HMD device 110. As an example, a portable information communication terminal (for example, a smartphone) including the display 112 may function as the computer 200.

  Further, the computer 200 may be configured to be used in common for the plurality of HMD devices 110. According to such a configuration, for example, the same virtual space can be provided to a plurality of users, so that each user can enjoy the same application as other users in the same virtual space. In such a case, the plurality of HMD systems 100 in this embodiment may be directly connected to the computer 200 by the input / output interface 13. In addition, each function (for example, synchronization processing described later) of the server 150 in the present embodiment may be implemented in the computer 200.

  In an embodiment, in the HMD system 100, a global coordinate system is set in advance. The global coordinate system has three reference directions (axes) parallel to the vertical direction in the real space, the horizontal direction orthogonal to the vertical direction, and the front-rear direction orthogonal to both the vertical direction and the horizontal direction. In the present embodiment, the global coordinate system is one of the viewpoint coordinate systems. Therefore, the horizontal direction, the vertical direction (up-down direction), and the front-rear direction in the global coordinate system are defined as an x-axis, a y-axis, and a z-axis, respectively. More specifically, in the global coordinate system, the x axis is parallel to the horizontal direction of the real space. The y axis is parallel to the vertical direction of the real space. The z axis is parallel to the front-rear direction of the real space.

  In one aspect, HMD sensor 120 includes an infrared sensor. When the infrared sensor detects each infrared ray emitted from each light source of the HMD device 110, the presence of the HMD device 110 is detected. The HMD sensor 120 further determines the position and inclination of the HMD device 110 in the real space according to the movement of the user 190 wearing the HMD device 110 based on the value of each point (each coordinate value in the global coordinate system). To detect. More specifically, the HMD sensor 120 can detect temporal changes in the position and tilt of the HMD device 110 using each value detected over time.

  The global coordinate system is parallel to the real space coordinate system. Therefore, each inclination of the HMD device 110 detected by the HMD sensor 120 corresponds to each inclination around the three axes of the HMD device 110 in the global coordinate system. The HMD sensor 120 sets the uvw visual field coordinate system to the HMD device 110 based on the inclination of the HMD device 110 in the global coordinate system. The uvw visual field coordinate system set in the HMD device 110 corresponds to a viewpoint coordinate system when the user 190 wearing the HMD device 110 views an object in the virtual space.

[Uvw visual field coordinate system]
The uvw visual field coordinate system will be described with reference to FIG. FIG. 3 is a diagram conceptually showing a uvw visual field coordinate system set in HMD device 110 according to an embodiment. The HMD sensor 120 detects the position and inclination of the HMD device 110 in the global coordinate system when the HMD device 110 is activated. The processor 10 sets the uvw visual field coordinate system in the HMD device 110 based on the detected value.

  As shown in FIG. 3, the HMD device 110 sets a three-dimensional uvw visual field coordinate system with the head (origin) of the user wearing the HMD device 110 as the center (origin). More specifically, the HMD device 110 uses the horizontal direction, the vertical direction, and the front-rear direction (x axis, y axis, z axis) that define the global coordinate system around each axis of the HMD device 110 in the global coordinate system. The three new directions obtained by inclining around the respective axes by the inclination of the pitch are the pitch direction (u-axis), yaw direction (v-axis), and roll direction (w-axis) of the uvw visual field coordinate system in the HMD device 110. Set as.

  In one aspect, when the user 190 wearing the HMD device 110 stands upright and is viewing the front, the processor 10 sets the uvw visual field coordinate system parallel to the global coordinate system in the HMD device 110. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and front-rear direction (z-axis) in the global coordinate system are the pitch direction (u-axis) and yaw direction (v Axis) and the roll direction (w-axis).

  After the uvw visual field coordinate system is set in the HMD device 110, the HMD sensor 120 can detect the inclination of the HMD device 110 in the set uvw visual field coordinate system based on the movement of the HMD device 110. . In this case, the HMD sensor 120 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD device 110 in the uvw visual field coordinate system as the inclination of the HMD device 110. The pitch angle (θu) represents the tilt angle of the HMD device 110 around the pitch direction in the uvw visual field coordinate system. The yaw angle (θv) represents the tilt angle of the HMD device 110 around the yaw direction in the uvw visual field coordinate system. The roll angle (θw) represents the tilt angle of the HMD device 110 around the roll direction in the uvw visual field coordinate system.

Based on the detected tilt angle of the HMD device 110, the HMD sensor 120 sets the uvw visual field coordinate system in the HMD device 110 after the HMD device 110 has moved to the HMD device 110. The relationship between the HMD device 110 and the uvw visual field coordinate system of the HMD device 110 is always constant regardless of the position and inclination of the HMD device 110. Location of HMD device 110
When 8 and the tilt change, the position and the tilt of the uvw visual field coordinate system of the HMD device 110 in the global coordinate system change in conjunction with the change of the position and the tilt.

  In one aspect, the HMD sensor 120 is based on the infrared light intensity acquired based on the output from the infrared sensor and the relative positional relationship between a plurality of points (for example, the distance between the points). The position of the device 110 in the real space may be specified as a relative position with respect to the HMD sensor 120. Further, the processor 10 may determine the origin of the uvw visual field coordinate system of the HMD device 110 in the real space (global coordinate system) based on the specified relative position.

[Virtual space]
The virtual space will be further described with reference to FIG. FIG. 4 is a diagram conceptually showing one aspect of expressing virtual space 2 according to an embodiment. The virtual space 2 has a spherical structure that covers the entire 360 ° direction of the center 21. In FIG. 4, the upper half of the celestial sphere in the virtual space 2 is illustrated in order not to complicate the description. In the virtual space 2, each mesh is defined. The position of each mesh is defined in advance as coordinate values in the XYZ coordinate system defined in the virtual space 2. The computer 200 associates each partial image constituting content (still image, moving image, etc.) that can be developed in the virtual space 2 with each corresponding mesh in the virtual space 2, and the virtual space image 22 that can be visually recognized by the user. Is provided to the user.

  In one aspect, the virtual space 2 defines an XYZ coordinate system with the center 21 as the origin. The XYZ coordinate system is, for example, parallel to the global coordinate system. Since the XYZ coordinate system is a kind of viewpoint coordinate system, the horizontal direction, vertical direction (vertical direction), and front-rear direction in the XYZ coordinate system are defined as an X axis, a Y axis, and a Z axis, respectively. Therefore, the X axis (horizontal direction) of the XYZ coordinate system is parallel to the x axis of the global coordinate system, the Y axis (vertical direction) of the XYZ coordinate system is parallel to the y axis of the global coordinate system, and The Z axis (front-rear direction) is parallel to the z axis of the global coordinate system. Each position in the virtual space 2 is uniquely specified by a coordinate value in the XYZ coordinate system.

  When the HMD device 110 is activated, that is, in the initial state of the HMD device 110, the virtual camera 1 (virtual viewpoint) is arranged at the center 21 of the virtual space 2, for example. The virtual camera 1 similarly moves in the virtual space 2 in conjunction with the movement of the HMD device 110 in the real space. Thereby, changes in the position and inclination of the HMD device 110 in the real space are similarly reproduced in the virtual space 2.

  The virtual camera 1 is an example of a virtual viewpoint. The virtual viewpoint means a gaze source in the virtual space 2.

  As in the case of the HMD device 110, the uvw visual field coordinate system is defined for the virtual camera 1. The uvw visual field coordinate system of the virtual camera 1 in the virtual space 2 is defined so as to be linked to the uvw visual field coordinate system of the HMD device 110 in the real space (global coordinate system). Therefore, when the inclination of the HMD device 110 changes, the inclination of the virtual camera 1 also changes accordingly. The virtual camera 1 can also move in the virtual space 2 in conjunction with the movement of the user wearing the HMD device 110 in the real space.

  Since the orientation of the virtual camera 1 is determined according to the position and inclination of the virtual camera 1, the reference line of sight (reference line of sight 5) when the user visually recognizes the virtual space image 22 depends on the orientation of the virtual camera 1. Determined. Based on the reference line of sight 5, the processor 10 of the computer 200 defines a view area 23 that is a view from the virtual camera 1 in the virtual space 2. The view area 23 corresponds to the view of the user wearing the HMD device 110 in the virtual space 2.

  The gaze direction of the user 190 detected by the gaze sensor 140 is a direction in the viewpoint coordinate system when the user 190 visually recognizes the object. The uvw visual field coordinate system of the HMD device 110 is equal to the viewpoint coordinate system when the user 190 visually recognizes the display 112. The uvw visual field coordinate system of the virtual camera 1 is linked to the uvw visual field coordinate system of the HMD device 110. Therefore, the HMD system 100 according to a certain aspect can regard the line-of-sight direction of the user 190 detected by the gaze sensor 140 as the line-of-sight direction of the user in the uvw visual field coordinate system of the virtual camera 1.

[User's line of sight]
With reference to FIG. 5, determination of the user's line-of-sight direction will be described. FIG. 5 is a diagram showing the head of user 190 wearing HMD device 110 according to an embodiment from above.

  In one aspect, gaze sensor 140 detects each line of sight of user 190's right eye and left eye. In a certain aspect, when the user 190 is looking near, the gaze sensor 140 detects the lines of sight R1 and L1. In another aspect, when the user 190 is looking far away, the gaze sensor 140 detects the lines of sight R2 and L2. In this case, the angle formed by the lines of sight R2 and L2 with respect to the roll direction w is smaller than the angle formed by the lines of sight R1 and L1 with respect to the roll direction w. The gaze sensor 140 transmits the detection result to the computer 200.

  When the computer 200 receives the detection values of the lines of sight R1 and L1 from the gaze sensor 140 as the line-of-sight detection result, the computer 200 identifies the point of sight N1 that is the intersection of the lines of sight R1 and L1 based on the detection value. On the other hand, when the detected values of the lines of sight R2 and L2 are received from the gaze sensor 140, the computer 200 specifies the intersection of the lines of sight R2 and L2 as the point of sight. The computer 200 specifies the line-of-sight direction N0 of the user 190 based on the specified position of the gazing point N1. For example, the computer 200 detects the direction in which the straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user 190 and the gazing point N1 extends as the line-of-sight direction N0. The line-of-sight direction N0 is a direction in which the user 190 is actually pointing the line of sight with both eyes. The line-of-sight direction N0 corresponds to the direction in which the user 190 actually directs his / her line of sight with respect to the field-of-view area 23.

  In another aspect, HMD system 100 may include a television broadcast receiving tuner. According to such a configuration, the HMD system 100 can display a television program in the virtual space 2.

  In still another aspect, the HMD system 100 may include a communication circuit for connecting to the Internet or a call function for connecting to a telephone line.

[Visibility area]
With reference to FIGS. 6 and 7, the visual field region 23 will be described. FIG. 6 is a diagram illustrating a YZ cross section of the visual field region 23 viewed from the X direction in the virtual space 2. FIG. 7 is a diagram illustrating an XZ cross section of the visual field region 23 viewed from the Y direction in the virtual space 2.

  As shown in FIG. 6, the visual field region 23 in the YZ cross section includes a region 24. The region 24 is defined by the reference line of sight 5 of the virtual camera 1 and the YZ cross section of the virtual space 2. The processor 10 defines a range including the polar angle α around the reference line of sight 5 in the virtual space 2 as the region 24.

  As shown in FIG. 7, the visual field region 23 in the XZ cross section includes a region 25. The region 25 is defined by the reference line of sight 5 and the XZ cross section of the virtual space 2. The processor 10 defines a range including the azimuth angle β around the reference line of sight 5 in the virtual space 2 as a region 25.

  In one aspect, the HMD system 100 provides a virtual space to the user 190 by causing the display 112 to display a view field image based on a signal from the computer 200. The visual field image corresponds to a portion of the virtual space image 22 that is superimposed on the visual field region 23. When a virtual object described later is arranged between the virtual camera 1 and the virtual space image 22 in the view field area 23, the view object image includes the virtual object. That is, in the view field image, the virtual object on the near side of the virtual space image 22 is displayed superimposed on the virtual space image 22. When the user 190 moves the HMD device 110 worn on the head, the virtual camera 1 also moves in conjunction with the movement. As a result, the position of the visual field area 23 in the virtual space 2 changes. As a result, the view image displayed on the display 112 is updated to an image that is superimposed on the view region 23 in the virtual space 2 in the direction in which the user faces in the virtual space image 22. The user can visually recognize a desired direction in the virtual space 2.

  The user 190 can visually recognize only the virtual space image 22 developed in the virtual space 2 without visually recognizing the real world while wearing the HMD device 110. Therefore, the HMD system 100 can give the user a high sense of immersion in the virtual space 2.

  In one aspect, the processor 10 can move the virtual camera 1 in the virtual space 2 in conjunction with movement of the user 190 wearing the HMD device 110 in real space. In this case, the processor 10 specifies an image region (that is, a view field region 23 in the virtual space 2) projected on the display 112 of the HMD device 110 based on the position and inclination of the virtual camera 1 in the virtual space 2. That is, the visual field (view) of the user 190 in the virtual space 2 is defined by the virtual camera 1.

  According to an embodiment, the virtual camera 1 preferably includes two virtual cameras, that is, a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. Moreover, it is preferable that appropriate parallax is set in the two virtual cameras so that the user 190 can recognize the three-dimensional virtual space 2. In the present embodiment, the virtual camera 1 includes two virtual cameras, and the roll direction (w) generated by combining the roll directions of the two virtual cameras is the roll direction (w) of the HMD device 110. The technical idea concerning this indication is illustrated as what is constituted so that it may be adapted.

[controller]
An example of the controller 160 will be described with reference to FIG. FIG. 8 is a diagram showing a schematic configuration of controller 160 according to an embodiment.

  As shown in the state (A) of FIG. 8, in one aspect, the controller 160 may include a right controller 160R and a left controller (not shown). The right controller 160R is operated with the right hand of the user 190. The left controller is operated with the left hand of the user 190. In one aspect, the right controller 160R and the left controller are configured symmetrically as separate devices. Therefore, the user 190 can freely move the right hand holding the right controller 160R and the left hand holding the left controller. In another aspect, the controller 160 may be an integrated controller that receives operations of both hands. Hereinafter, the right controller 160R will be described.

  The right controller 160R includes a grip 30, a frame 31, and a top surface 32. The grip 30 is configured to be held by the right hand of the user 190. For example, the grip 30 can be held by the palm of the right hand of the user 190 and three fingers (middle finger, ring finger, little finger).

  The grip 30 includes buttons 33 and 34 and a motion sensor 130. The button 33 is disposed on the side surface of the grip 30 and receives an operation with the middle finger of the right hand. The button 34 is disposed in front of the grip 30 and accepts an operation with the index finger of the right hand. In one aspect, the buttons 33 and 34 are configured as trigger buttons. The motion sensor 130 is built in the housing of the grip 30. Note that when the operation of the user 190 can be detected from around the user 190 by a camera or other device, the grip 30 may not include the motion sensor 130.

  The frame 31 includes a plurality of infrared LEDs 35 arranged along the circumferential direction. The infrared LED 35 emits infrared light in accordance with the progress of the program during the execution of the program using the controller 160. The infrared rays emitted from the infrared LED 35 can be used to detect the positions and postures (tilt, orientation), etc., of the right controller 160R and the left controller. In the example shown in FIG. 8, infrared LEDs 35 arranged in two rows are shown, but the number of arrays is not limited to that shown in FIG. An array of one or more columns may be used.

  The top surface 32 includes buttons 36 and 37 and an analog stick 38. The buttons 36 and 37 are configured as push buttons. The buttons 36 and 37 receive an operation with the thumb of the right hand of the user 190. In one aspect, the analog stick 38 accepts an operation in an arbitrary direction of 360 degrees from the initial position (neutral position). The operation includes, for example, an operation for moving an object arranged in the virtual space 2.

  In one aspect, the right controller 160R and the left controller include a battery for driving the infrared LED 35 and other members. The battery includes, but is not limited to, a rechargeable type, a button type, a dry battery type, and the like. In another aspect, the right controller 160R and the left controller may be connected to a USB interface of the computer 200, for example. In this case, the right controller 160R and the left controller do not require batteries.

  As shown in the state (A) and the state (B) of FIG. 8, for example, the yaw, roll, and pitch directions are defined for the right hand 810 of the user 190. When the user 190 extends the thumb and index finger, the direction in which the thumb extends is the yaw direction, the direction in which the index finger extends is the roll direction, and the direction perpendicular to the plane defined by the yaw direction axis and the roll direction axis is the pitch direction. Is defined as

[Control device for HMD device]
The control device of the HMD device 110 will be described with reference to FIG. In one embodiment, the control device is realized by a computer 200 having a known configuration. FIG. 9 is a block diagram showing a computer 200 according to an embodiment as a module configuration.

  As shown in FIG. 9, the computer 200 includes a display control module 220, a virtual space control module 230, a memory module 240, and a communication control module 250. The display control module 220 includes a virtual camera control module 221, a visual field region determination module 222, a visual field image generation module 223, and a reference visual line identification module 224 as submodules. The virtual space control module 230 includes a virtual space definition module 231, a virtual object control module 232, a synchronization control module 233, and a determination module 234 as submodules.

  In an embodiment, the display control module 220 and the virtual space control module 230 are realized by the processor 10. In another embodiment, multiple processors 10 may operate as the display control module 220 and the virtual space control module 230. The memory module 240 is realized by the memory 11 or the storage 12. The communication control module 250 is realized by the communication interface 14.

  In one aspect, the display control module 220 controls image display on the display 112 of the HMD device 110. The virtual camera control module 221 arranges the virtual camera 1 in the virtual space 2 and controls the behavior, orientation, and the like of the virtual camera 1. The view area determination module 222 defines the view area 23 according to the orientation of the head of the user wearing the HMD device 110. The view image generation module 223 generates a view image to be displayed on the display 112 based on the determined view area 23. The reference line-of-sight identifying module 224 identifies the line of sight of the user 190 based on the signal from the gaze sensor 140.

  The virtual space control module 230 controls the virtual space 2 provided to the user 190. The virtual space definition module 231 defines the virtual space 2 in the HMD system 100 by generating virtual space data representing the virtual space 2.

  The virtual object control module 232 generates a virtual object arranged in the virtual space 2 based on content information 241 and object information 242 described later. The virtual object control module 232 also controls the movement (movement, state change, etc.) of the virtual object in the virtual space 2.

  The virtual object is an entire object arranged in the virtual space 2. The virtual objects may include, for example, forests, mountains and other landscapes, animals, etc. that are arranged according to the progress of the game story. In addition, the virtual object may include a character object such as an avatar that is a user's alternation in a virtual space and a game character (player character) operated by the user. Further, the virtual object may include an operation object that is an object that moves according to the movement of a part of the body (eg, a hand) of the user 190. The operation object may include, for example, a hand object corresponding to the hand of the user 190 wearing the HMD device 110, a finger object corresponding to the finger of the user 190, and the like. An object that is operated in association with the hand object can also function as an operation object that moves in accordance with the hand movement of the user 190. For example, a stick-like object such as a touch pen held by a hand object can function as an operation object. In the following description, if no misunderstanding occurs, the virtual object is simply referred to as “object”.

  The synchronization control module 233 transmits / receives data to be shared between users to / from other users' HMD systems 100 via the server 150. The data to be shared includes motion detection data for controlling the movement of a part of the avatar's body. The motion detection data is, for example, orientation data, eye tracking data, face tracking data, hand tracking data, and the like. The orientation data is information indicating the position and inclination of the HMD device 110 detected by the HMD sensor 120 or the like. The eye tracking data is information indicating the line-of-sight direction detected by the gaze sensor 140 or the like. The face tracking data is, for example, data generated by image analysis processing on image information acquired by the camera 116 of the HMD device 110A. The face tracking data is information indicating changes with time in the position and size of each part of the face of the user 190A. The hand tracking data is information indicating the movement of the hand of the user 190A detected by, for example, the motion sensor 130 or the like.

  In the present embodiment, the synchronization control module 233 communicates information including motion detection data (hereinafter referred to as “player information”) with other users' HMD systems 100 via the server 150 as information to be shared between users. Send and receive between. The player information is transmitted / received by using the function of the communication control module 250.

  The determination module 234 determines the relationship between objects in the virtual space 2.

  The virtual space control module 230 detects the collision when each of the objects arranged in the virtual space 2 collides with another object. For example, the virtual space control module 230 can detect a timing at which a certain object and another object touch each other, and performs a predetermined process when the detection is performed. The virtual space control module 230 can detect the timing at which the object is away from the touched state, and performs a predetermined process when the detection is made. The virtual space control module 230 can detect that the object is in contact with the object by executing a known hit determination based on, for example, a collision area set for each object.

  The memory module 240 holds data used for the computer 200 to provide the virtual space 2 to the user 190. In one aspect, the memory module 240 holds content information 241, object information 242, and user information 243.

  The content information 241 includes, for example, content that is reproduced in the virtual space 2, information for arranging objects used in the content, and the like. The content can include, for example, content representing a scene similar to a game or a real society. Specifically, the content information 241 can include virtual space image data (virtual space image 22) that defines the background of the virtual space 2 and definition information of objects arranged in the virtual space 2. The object definition information may include drawing information for drawing the object (for example, information indicating a design such as the shape and color of the object), information indicating an initial arrangement of the object, and the like. Further, the definition information of an object that operates autonomously based on a preset operation pattern may include information (program or the like) indicating the operation pattern. As an example of an operation based on a predetermined operation pattern, there is a simple repetitive operation such as an operation in which an object imitating grass swings in a certain pattern.

  The object information 242 includes information indicating the state of each object arranged in the virtual space 2 (a state that can change according to the progress of the game and the operation of the user 190). Specifically, the object information 242 may include position information indicating the position of each object (for example, the position of the center of gravity set for the object). Further, the object information 242 may further include motion information indicating the motion of the deformable object (that is, information for specifying the shape of the object). Examples of deformable objects include parts such as the above-described avatars that have parts such as a head, a torso, and a hand, and can move each part independently according to the movement of the user 190. Is mentioned.

  The user information 243 includes, for example, a program for causing the computer 200 to function as a control device of the HMD system 100, an application program that uses each content held in the content information 241, and the like.

  Data and programs stored in the memory module 240 are input by the user of the HMD device 110. Alternatively, the processor 10 downloads a program or data from a computer (for example, the server 150) operated by a provider providing the content, and stores the downloaded program or data in the memory module 240.

  The communication control module 250 can communicate with the server 150 and other information communication devices via the network 19.

  In an aspect, the display control module 220 and the virtual space control module 230 may be realized using, for example, Unity (registered trademark) provided by Unity Technologies. In another aspect, the display control module 220 and the virtual space control module 230 can also be realized as a combination of circuit elements that realize each process.

  Processing in the computer 200 is realized by hardware and software executed by the processor 10. Such software may be stored in advance in a memory module 240 such as a hard disk. The software may be stored in a CD-ROM or other non-volatile computer-readable data recording medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the Internet or other networks. Such software is read from a data recording medium by an optical disk drive or other data reader, or downloaded from the server 150 or other computer via the communication control module 250 and then temporarily stored in the memory module 240. The The software is read from the memory module 240 by the processor 10 and stored in the RAM in the form of an executable program. The processor 10 executes the program.

  The hardware configuring the computer 200 shown in FIG. 9 is general. Therefore, it can be said that the most essential part according to the present embodiment is a program stored in the computer 200. Since the hardware operation of computer 200 is well known, detailed description will not be repeated.

As data recording media, CD-ROM, FD (Flexible
Disk), not limited to hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile Disc)), IC (Integrated Circuit) card (including memory card), Non-volatile data recording that carries a fixed program such as optical memory, mask ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory, registered trademark), flash ROM, etc. It may be a medium.

  The program here may include not only a program directly executable by the processor 10, but also a program in a source program format, a compressed program, an encrypted program, and the like.

[Control structure]
With reference to FIG. 10, a control structure of computer 200 according to the present embodiment will be described. FIG. 10 is a flowchart showing processing executed by the HMD system 100A used by the user 190A to provide the virtual space 2 to the user 190A. Similar processing is executed in the other HMD systems 100B and 100C.

  In step S <b> 1, the processor 10 of the computer 200 specifies the virtual space image data (virtual space image 22) constituting the background of the virtual space 2 as the virtual space definition module 231, and defines the virtual space 2.

  In step S <b> 2, the processor 10 initializes the virtual camera 1 as the virtual camera control module 221. For example, the processor 10 places the virtual camera 1 at a predetermined center point in the virtual space 2 in the work area of the memory, and directs the line of sight of the virtual camera 1 in the direction in which the user 190 is facing.

  In step S <b> 3, the processor 10 generates view image data for displaying an initial view image as the view image generation module 223. The generated view image data is sent to the HMD device 110 by the communication control module 250 via the view image generation module 223.

  In step S <b> 4, the display 112 of the HMD device 110 displays a view field image based on the signal received from the computer 200. The user 190A wearing the HMD device 110A can recognize the virtual space 2 when viewing the visual field image.

  In step S <b> 5, the HMD sensor 120 detects the position and tilt of the HMD device 110 based on a plurality of infrared lights transmitted from the HMD device 110. The detection result is sent to the computer 200 as motion detection data.

  In step S6, the processor 10 uses the visual field direction determination module 222 to determine the visual field direction (that is, the position and tilt of the virtual camera 1) of the user 190A wearing the HMD device 110A based on the position and tilt of the HMD device 110A. Identify. The processor 10 executes the application program and places an object in the virtual space 2 based on instructions included in the application program.

  In step S7, the controller 160 detects the operation of the user 190A in the real space. For example, in one aspect, the controller 160 detects that a button has been pressed by the user 190A. In another aspect, the controller 160 detects the operation of both hands of the user 190A (for example, shaking both hands). A signal indicating the detected content is sent to the computer 200.

  In step S <b> 8, the processor 10 transmits / receives player information to / from another HMD system 100 (here, the HMD systems 100 </ b> B and 100 </ b> C) as the synchronization control module 233 via the server 150.

  In step S <b> 9, the processor 10 controls the operation of the avatar associated with each user as the virtual object control module 232 based on the player information of each user 190.

  In step S <b> 10, the processor 10 generates view image data for displaying the view image based on the processing result of step S <b> 9 as the view image generation module 223, and outputs the generated view image data to the HMD device 110.

  In step S11, the display 112 of the HMD device 110 updates the view image based on the received view image data, and displays the updated view image.

  The processes in steps S5 to S11 are repeatedly executed periodically.

  FIG. 11 is a diagram schematically illustrating the virtual space 2 shared by a plurality of users. In the example shown in FIG. 11, an avatar 40A associated with a user 190A (first user) wearing the HMD device 110A, an avatar 40B associated with a user 190B (second user) wearing the HMD device 110B, The avatar 40C associated with the user 190C wearing the HMD device 110C is arranged in the same virtual space 2. According to such virtual space 2 common to a plurality of users, it is possible to provide each user with a communication experience such as chatting with other users via the avatars 40A to 40C.

  In this example, each avatar 40A-40C is defined as a character object that imitates a human being. The avatars 40 </ b> A to 40 </ b> C have a head (face orientation), eyes (line of sight, blink, etc.), a face (facial expression), a right hand 41 </ b> A to 41 </ b> C, and a left hand 42 </ b> A to 42 </ b> C, Is included. Hereinafter, the avatar's hand may be referred to as a “virtual hand”.

  The head is a portion that moves in conjunction with the movement of the HMD device 110 detected by the HMD sensor 120 or the like. The eye is a portion that moves in conjunction with the movement of the user's eyes and changes in the line of sight detected by the camera 116, the gaze sensor 140 and the like. The face is a part in which a facial expression determined based on face tracking data described later is reflected. The hand is a part that moves in conjunction with the movement of the user's hand detected by the motion sensor 130 or the like. In addition, the avatars 40A to 40C include a body part and an arm part that are displayed accompanying the head, the right hand 41A to 41C, and the left hand 42A to 42C. In addition, since operation | movement control becomes complicated about the leg part from the waist, avatar 40A-40C does not include a leg part. It is not necessary to move all the operable parts of the avatar 40 described above. For example, the movement of the head and hand of the user 190 is reflected in the movement of the head and hand of the avatar 40, but the movement of other parts of the user 190 may not be reflected in the avatar 40.

  The field of view of the avatar 40A matches the field of view of the virtual camera 1 in the HMD system 100A. Thereby, the view field image M in the first person viewpoint of the avatar 40A is provided to the user 190A. That is, a virtual experience as if the user 190A exists in the virtual space 2 as the avatar 40A is provided to the user 190A.

  FIG. 12 is a diagram illustrating an example of a field-of-view image M provided to the user 190A via the HMD device 110A. Similarly, a view field image at the first person viewpoint of the avatars 40B and 40C is provided to the users 190B and 190C.

  FIG. 13 is a sequence diagram illustrating processing executed by the HMD system 100A, the HMD system 100B, the HMD system 100C, and the server 150 in order to realize user interaction in the virtual space 2.

  In step S <b> 21 </ b> A, the processor 10 in the HMD system 100 </ b> A acquires player information for determining the operation of the avatar 40 </ b> A as the synchronization control module 233. The player information includes information (such as a user ID) that identifies the avatar 40A (or the user 190A associated with the avatar 40A), information that identifies the virtual space 2 in which the avatar 40A exists (such as a room ID), and the like. Also good. The processor 10 transmits the player information acquired as described above to the server 150 via the network 19 as the synchronization control module 233.

  In step S <b> 21 </ b> B, the processor 10 in the HMD system 100 </ b> B acquires player information for determining the operation of the avatar 40 </ b> B and transmits it to the server 150, similarly to the process in step S <b> 21 </ b> A. Similarly, in step S <b> 21 </ b> C, the processor 10 in the HMD system 100 </ b> C acquires player information for determining the operation of the avatar 40 </ b> C and transmits it to the server 150.

  In step S22, the server 150 temporarily stores player information received from each of the HMD system 100A, the HMD system 100B, and the HMD system 100C. The server 150 integrates player information of all users (users 190A to 190C in this example) associated with the common virtual space 2 based on the user ID, room ID, and the like included in each player information. Then, the server 150 transmits the integrated player information to all users associated with the virtual space 2 at a predetermined timing. Thereby, a synchronous process is performed. By such synchronization processing, the HMD system 100A, the HMD system 100B, and the HMD system 100C can share each other's player information at substantially the same timing.

  Subsequently, based on the player information transmitted from the server 150 to each of the HMD systems 100A to 100C, each of the HMD systems 100A to 100C executes the processes of steps S23A to S23C. The process of step S23A corresponds to the process of step S9 in FIG.

  In step S23A, the processor 10 in the HMD system 100A controls the operations of the avatars 40A to 40C of the users 190A to 190C in the virtual space 2 as the virtual object control module 232. Specifically, the processor 10 controls the operations of the avatars 40A to 40C based on the motion detection data of the users 190A to 190C included in the player information transmitted from the HMD system 100B. The process of steps S23B and S23C is the same as the process of step S23A.

  Note that the synchronization processing described above may be performed between the HMD systems 100 without using the server 150. In other words, the synchronization process may be performed in a so-called mesh type.

[Avatar interlocking control processing]
FIG. 14 is a flowchart relating to the interlock control process of the avatar 40A and the avatar 40B. In the present embodiment, a description will be given assuming that processing related to interlocking control of the avatar 40A and the avatar 40B is executed by the HMD system 100A. However, the process may be executed by the other HMD systems 100B and 100C, or a part or all of the process may be executed by the server 150.

  In step S <b> 31, the processor 10 of the HMD system 100 </ b> A (hereinafter simply “processor 10”) defines the virtual space 2 as the virtual space definition module 231. This process corresponds to the process in step S1 of FIG. Specifically, the processor 10 defines the virtual space 2 by generating virtual space data that defines the virtual space 2. The virtual space data includes the content information 241 and the object information 242 described above.

  In step S32, the processor 10 determines the position and inclination of the virtual camera 1 in the virtual space 2 according to the movement of the HMD device 110A. More specifically, the visual field region 23 that is the visual field from the virtual camera 1 in the virtual space 2 is controlled according to the posture of the head of the user 190 </ b> A and the position of the virtual camera 1 in the virtual space 2. This process corresponds to a part of the process of step S6 in FIG.

  In step S33, the processor 10 provides the view image M (see FIG. 12) to the user 190A. Specifically, the processor 10 determines the visual field image M corresponding to the visual field region 23 based on the movement of the HMD device 110A (that is, the position and inclination of the virtual camera 1) and the virtual space data defining the virtual space 2. Define The processor 10 further causes the display 112 to display the view image M by outputting the view image M to the display 112 of the HMD device 110A. This process corresponds to the process of step S10 in FIG.

  The processes in steps S32 and S33 described above (that is, the update of the view image M in accordance with the movement of the HMD device 110A) are continuously and repeatedly executed while steps S34 to S40 described later are executed.

  In step S34, the processor 10 moves the left hand 42A of the avatar 40A in the virtual space 2 based on the movement of the left hand of the user 190A. Furthermore, the processor 10 can move the right hand 41A of the avatar 40A in the virtual space 2 based on the movement of the right hand of the user 190A.

  On the other hand, the processor 10 in the HMD system 100B acquires the player information of the user 190B based on the left hand movement of the user 190B. This information includes at least information regarding the left hand movement of the user 190B. The processor 10 in the HMD system 100B transmits the acquired player information to the server 150.

  In step S35, the processor 10 acquires the player information of the user 190B from the server 150. In step 36, the processor 10 moves the left hand 42B of the avatar 40B in the virtual space 2 based on the movement of the left hand of the user 190B. Specifically, the processor 10 acquires information related to the movement of the left hand 42B from the player information of the user 190B, and moves the left hand 42B of the avatar 40B based on the information. When the information regarding the movement of the right hand 41B is included in the player information of the user 190B, the processor 10 can also move the right hand 41B of the avatar 40B based on the information.

  In step S37, the processor 10 determines, as the determination module 234, whether the left hand 42A of the avatar 40A and the left hand 42B of the avatar 40B in the virtual space 2 satisfy the first condition. If NO in step S37, the process in FIG. 14 ends. Therefore, the processor 10 does not control the avatar 40 </ b> A and the avatar 40 </ b> B in the virtual space 2.

  If YES in step S37, the processor 10 controls at least one of the left hand 42A, the left hand 42B, and the virtual camera 1 in the virtual space 2 according to the first condition. In other words, the processor 10 controls the avatar 40A and the avatar 40B in conjunction with each other.

(Linkage of Avatar 40A and Avatar 40B)
FIG. 15 is a diagram illustrating an example of the state of the virtual space 2 when the first condition is satisfied. FIG. 16 is a diagram illustrating an example of the field-of-view image M displayed when the first condition is satisfied. In the state (A) of FIG. 15, the avatar 40 </ b> A is located behind the avatar 40 </ b> B in the virtual space 2. Both the user 190A and the user 190B are stationary in the real space, and thus both the avatar 40A and the avatar 40B are stationary in the virtual space 2. The left hand 42A of the avatar 40A is held by the left hand 42B of the avatar 40B.

  In the present embodiment, the user 190 makes a contact with the controller 160 while the collision area set for the right hand 41 (or the left hand 42) of the avatar 40 is in contact with the collision area set for an arbitrary object. By performing a predetermined input, the arbitrary object can be selected (gripped) by the right hand 41 (or the left hand 42) of the avatar 40. Examples of the predetermined input include pressing of the trigger type buttons 33 and 34, but are not limited thereto. Therefore, when the user 190B makes a predetermined input to the left controller while the collision area set for the left hand 42B of the avatar 40B is in contact with the collision area set for the left hand 42A of the avatar 40A, the avatar The left hand 42A of the avatar 40A can be held by the left hand 42B of 40B.

  The processor 10 causes the display 112 to display the field-of-view image M shown in the state (A) of FIG. This view image M corresponds to the virtual space 2 shown in the state (A) of FIG. The user visually recognizes the view image M shown in the state (A) of FIG. 16, so that the avatar 40B is turned to the back and is stationary in front of the avatar 40A, and the left hand 42A of the avatar 40A is left by the left hand 42B of the avatar 40B. Recognize that

  The processor 10 determines that the relationship between the left hand 42A and the left hand 42B satisfies the first condition in the virtual space 2 when the left hand 42A and the left hand 42B are in the relationship shown in the state (A) of FIG. Based on the determination result, the processor 10 controls the avatar 40A and the avatar 40B in the virtual space 2 as shown in the state (B) of FIG. For example, the avatar 40 </ b> B and the left hand 42 </ b> B are moved in the direction 51 in the virtual space 2. At the same time, the processor 10 follows the movement of the avatar 40B and the left hand 42B while maintaining the positional relationship between the left hand 42A and the virtual camera 1, and moves the avatar 40A, the left hand 42A, and the virtual camera 1 in the virtual space 2. In the direction 51. At this time, since the HMD device 110A and the HMD device 110B are not moving, the user 190A and the user 190B remain stationary in the real space.

  Further, the processor 10 causes the display 112 to display the field-of-view image M shown in the state (B) of FIG. This view image M corresponds to the virtual space 2 shown in the state (B) of FIG. The user 190A follows the movement of the avatar 40B and the left hand 42B forward (direction 51) by visually recognizing the view image M shown in the state (B) of FIG. 16, and the avatar 40A automatically moves together with the left hand 42A. Recognize that you are moving.

  The user 190A can experience an action in the virtual space 2 that the user 190A is guided by the avatar 40B and moves together with the left hand 42 of the avatar 40B being held. As described above, according to one aspect of the present disclosure, a natural virtual experience as if the user 190A acted together with another person in the real space can be provided to the user 190A through the virtual space 2. Accordingly, the user 190A can accurately recognize an operation (behavior) to be performed in the virtual space 2 based on past experience in the real space. Therefore, the user 190A can spend the virtual space 2 without a sense of incongruity.

(Example of the first condition)
FIG. 17 is a diagram illustrating examples of the first condition. In the state (A) of FIG. 17, the first condition is that the left hand 42A of the avatar 40A is selected by the left hand 42B of the avatar 40B. More specifically, in the state (A) of FIG. 17, the left hand 42A of the avatar 40A is held by the left hand 42B of the avatar 40B. In the state (B) of FIG. 17, the first condition is that the left hand 42B of the avatar 40B is selected by the left hand 42A of the avatar 40A. More specifically, in the state (B) of FIG. 17, the left hand 42B of the avatar 40B is gripped by the left hand 42A of the avatar 40A.

  In the state (C) of FIG. 17, the first condition is that the left hand 42A of the avatar 40A is selected by the left hand 42B of the avatar 40B, and the left hand 42B of the avatar 40B is selected by the left hand 42A of the avatar 40A. It is that you are. More specifically, in the state (C) of FIG. 17, the left hand 42B of the avatar 40B is holding the left hand 42A of the avatar 40A, and the left hand 42A of the avatar 40A is holding back the left hand 42B of the avatar 40B. In this case, since the user 190A moves his / her left hand to grip the left hand 42B of the avatar 40B, the avatar 40A moves in the virtual space 2 as the user 190B moves as shown in the state (B) of FIG. It can be predicted well that it moves automatically inside. Therefore, even when the viewpoint of the user 190A moves due to the movement of the virtual camera 1 without the movement of the user 190A itself (the movement of the HMD device 110), the occurrence of VR sickness in the user 190A is reduced. I can do it.

  In the state (D) of FIG. 17, the first condition is that the left hand 42A of the avatar 40A is selected by the left hand 42B of the avatar 40B, and the left hand 42A and the left hand 42B move in the same direction 52 in the virtual space 2. It is that. In this case, with the left hand 42A held by the left hand 42B moving in the same direction 52 together with the left hand 42B as a trigger, the left hand 42A automatically interlocks with the movement of the avatar 40B. For example, when the avatar 40B moves in the direction 52 while the left hand 42B holds the left hand 42A, not only the left hand 42B but also the left hand 42A held by the left hand 42B moves in the direction 52. Further, for example, when the left hand 42B grasps the left hand 42A and the left hand 42B moves in the direction 52, not only the left hand 42B but also the left hand 42A grasped by the left hand 42B moves in the direction 52. The movement of the avatar 40B is assumed to move in conjunction with the movement of the head of the user 190B. However, the movement is not limited to this, and the movement of the avatar 40B may be performed by another operation or operation of the user 190B. The movement may be performed automatically regardless of the operation or operation of the user 190B. Similarly, the movement of the left hand 42B is assumed to move in conjunction with the movement of the left hand 42B of the user 190B. However, the movement is not limited to this, and the movement may be performed by another operation or operation of the user 190B. However, it may move automatically regardless of the operation or operation of the user 190B. As a result, the user 190 </ b> A grasps his / her left hand by the left hand 42 of the avatar 40 </ b> B and pulls his / her left hand further, leading the user 190 </ b> A to move together in the virtual space 2. You can experience it. Therefore, a natural experience of acting together with another person in the real space can be reflected in the virtual space 2.

  FIG. 18 is a diagram illustrating another example of the first condition. In the state (A) of FIG. 18, the first condition is that the left hand 42A of the avatar 40A is selected by the left hand 42B of the avatar 40B, and the left hand 42A and the left hand 42B are included in the view area 23. In this case, in a situation where the user 190A can visually recognize that the left hand 42A is selected by the left hand 42B, the avatar 40A automatically moves in the virtual space 2 following the movement of the avatar 40B. Since the user visually recognizes that the left hand 42A has been selected by the left hand 42B, the user can sufficiently predict that the avatar 40A automatically moves in the virtual space 2 following the movement of the avatar 40B. Therefore, even when the viewpoint of the user 190A moves due to the movement of the virtual camera 1 without the movement of the user 190A itself (the movement of the HMD device 110), the occurrence of VR sickness in the user 190A is reduced. it can.

  In the state (B) of FIG. 18, unlike the state (A) of FIG. 18, the left hand 42A of the avatar 40A is selected by the left hand 42B of the avatar 40B, but the left hand 42A and the left hand 42B are in the temporary view area 23. Not included. Since the range of the visual field area 23 in the virtual space 2 can be easily changed by the movement of the head by the user 190A, such a situation is likely to occur relatively easily. The processor 10 does not automatically move the avatar 40A in the virtual space 2 following the movement of the avatar 40B when the condition shown in the state (B) of FIG. Accordingly, since the viewpoint of the user 190A does not automatically move in a situation where the user 190A cannot predict the automatic movement of the avatar 40A, it is possible to prevent the VR 190 from being caused by the user 190A.

  FIG. 19 is a diagram illustrating another example of the first condition. In the state (A) of FIG. 19, the first condition is that the left hand 42A of the avatar 40A and the left hand 42B of the avatar 40B approach each other and the left hand 42A and the left hand 42B are in the first positional relationship. Specifically, in the state (A) of FIG. 19, the left hand 42A moves in the direction 53A in the virtual space 2, and the left hand 42B moves in the direction 53B opposite to the direction 53A in the virtual space 2. As a result of these movements, when the left hand 42A and the left hand 42B are in the first positional relationship, the first condition is satisfied. The first positional relationship is, for example, a positional relationship in which the collision area set for the left hand 42A and the collision area set for the left hand 42B are in contact, and the distance between the left hand 42A and the left hand 42B is the first distance. There are certain things, but it is not limited to this. When the first condition shown in the state (A) of FIG. 19 is satisfied, the processor 10 causes the left hand 42 </ b> A in the virtual space 2 as a result of the actions of the user 190 </ b> A and the user 190 </ b> B trying to interact with each other in the virtual space 2. At least one of the left hand 42B and the virtual camera 1 is automatically controlled. Thereby, the virtual experience of the user 190A in the virtual space 2 can be further improved.

  In the state (B) of FIG. 19, the first condition is that the left hand 42A of the avatar 40A and the left hand 42B of the avatar 40B are in the first positional relationship, and the direction 54A (first direction) defined by the left hand 42A, That is, the direction 54B (second direction) defined by the left hand 42B is opposed. The first positional relationship is the same as that described with reference to the state (A) in FIG. The direction 54A and the direction 54B are, for example, both roll directions as shown in the state (B) of FIG. Both the direction 54A and the direction 54B may be the yaw direction, or both may be the pitch direction. When the first condition shown in the state (A) of FIG. 19 is satisfied, the processor 10 determines that the left hand 42A, the left hand 42B, and the virtual as a result of actions that the user 190A and the user 190B try to interact with each other in the virtual space 2 Control at least one of the cameras 1. Thereby, the virtual experience of the user 190A in the virtual space 2 can be further improved.

  In the state (C) of FIG. 19, the first condition is that the collision area 55A (first area) set for the left hand 42A of the avatar 40A and the collision area 55B (second area) defined for the left hand 42B of the avatar 40B. But there is a collision. In this case, the processor 10 can easily determine whether or not the left hand 42A and the left hand 42B satisfy the first condition. The first condition may be that the left hand 42A of the avatar 40A and the left hand 42B of the avatar 40B are in direct contact.

(Change of position in virtual space 2)
FIG. 20 is a diagram illustrating another example of the state of the virtual space 2 when the first condition is satisfied. FIG. 21 is a diagram illustrating another example of the field-of-view image M displayed when the first condition is satisfied. In the example of FIG. 20, the user 190 </ b> A and the user 190 </ b> B are playing a battle game in the common virtual space 2. In the state (A) of FIG. 20, the avatar 40B has the weapon 61B in the right hand 41B, and the avatar 40A does not have the weapon 61A in the right hand 41A. In the virtual space 2, the avatar 40B is located in front of the enemy character 62, and the avatar 40A is located behind the avatar 40B. The avatar 40 </ b> B has a right to fight the enemy character 62 in the virtual space 2, and the avatar 40 </ b> A does not have a right to fight the enemy character 62 in the virtual space 2. The right to fight is, for example, that the avatar 40 is located in a range where the enemy character 62 can be attacked, and the avatar 40 is located in a range where the enemy character 62 can be attacked (targeted). However, the present invention is not limited to this. The avatar 40B can fight the enemy character 62 based on the operation by the user 190B. The avatar 40 </ b> A cannot participate in the battle between the avatar 40 </ b> B and the enemy character 62.

  The processor 10 causes the display 112 to display the field-of-view image M shown in the state (A) of FIG. This view image M corresponds to the virtual space 2 shown in the state (A) of FIG. The user recognizes that the avatar 40B is positioned in front of the avatar 40A with the back facing and is fighting the enemy character 62 by visually recognizing the view image M shown in the state (A) of FIG. . The user 190 </ b> A further recognizes that the avatar 40 </ b> A cannot fight the enemy character 61. The user 190A observes the battle between the avatar 40B and the enemy character 62 by visually recognizing the view image M shown in the state (A) of FIG.

  When the view image M of FIG. 21 is displayed, the user 190A moves the left hand 42A of the avatar 40A in the direction 63 facing the left hand 42B of the avatar 40B by moving the left hand, and further contacts the left hand 42A with the left hand 42B. Let As a result, the first condition is established, so that the processor 10 stands in the virtual space 2 between the avatar 40A associated with the left hand 42A and the avatar 40B associated with the left hand 42B, as shown in the state (B) of FIG. Replace. In this example, the processor 10 exchanges the right to fight the enemy character 62 in the virtual space 2 between the avatar 40A and the avatar 40B. As a result, the avatar 40B loses the right to fight the enemy character 62, and the avatar 40A newly obtains the right to fight the enemy character 62. The processor 10 further changes the target attacked by the enemy character 62 from the avatar 40B to the avatar 40A.

  Further, the processor 10 switches the position of the avatar 40A and the position of the avatar 40B in the virtual space 2 so that the avatar 40A can fight the enemy character 62. As a result, the avatar 40 </ b> A is positioned in front of the avatar 40 </ b> B and faces the enemy character 62. Following the movement of the avatar 40A, the virtual camera 1 also moves in front of the avatar 40B. On the other hand, the avatar 40B comes to be located behind the avatar 40A. The processor 10 further causes the right hand 41A of the avatar 40A to hold the weapon 61A and removes the weapon 61B from the right hand 41B of the avatar 40B.

  After these controls, the processor 10 causes the display 112 to display a view field image M shown in the state (B) of FIG. This view image M corresponds to the virtual space 2 shown in the state (B) of FIG. The user recognizes that the avatar 40A can battle the enemy character 62 in place of the avatar 40B by visually recognizing the view image M shown in FIG. Therefore, the user 190A can attack the enemy character 62 with the weapon 61A by operating the right hand 41A of the avatar 40A.

  As described above, as long as the user 190A operates the left hand 42A so that the relationship between the left hand 42A of the avatar 40A and the left hand 42B of the avatar 40B satisfies the first condition, the user 190A replaces the avatar 40A with the enemy character 62. The avatar 40 </ b> A can be made to fight the enemy character 62 without performing an operation (or action) for making it fight. As described above, according to one aspect of the present disclosure, the user 190A and the user 190B are intuitive because the avatar capable of fighting with the enemy character 62 can be appropriately switched by simply bringing the left hand 42A and the left hand 42B into contact with each other. The avatar 40 that can be battled by movement can be replaced, and a virtual experience with a higher immersive feeling can be provided to the user 190A and the user 190B.

  As mentioned above, although embodiment of this indication was described, the technical scope of this invention should not be limitedly interpreted by description of this embodiment. This embodiment is an example, and it is understood by those skilled in the art that various modifications can be made within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the equivalents thereof.

  The selection of the left hand 42B by the left hand 42A and the selection of the left hand 42A by the left hand 42B can also be an indirect selection via an intermediate object. Specifically, when the left hand 42A holds (selects) the intermediate object, the processor 10 can handle the intermediate object as a part of the left hand 42A. Similarly, when the left hand 42B holds (selects) the intermediate object, the processor 10 can handle the intermediate object as a part of the left hand 42B. That is, the intermediate object held by the operation object can also be handled as a part of the operation object. From these facts, the left hand 42B holds the intermediate object and the left hand 42A selects (holds, contacts, etc.) the intermediate object, and the left hand 42A is selected by the left hand 42A. It is synonymous with that. Similarly, the fact that the left hand 42A holds the intermediate object and the left hand 42B selects (holds, touches, etc.) the intermediate object is that the left hand 42B is selected by the left hand 42B. It is synonymous.

  It is assumed that, for example, a relay competition is performed in the virtual space 2. During the relay competition, the avatar 40A as the first runner passes the baton (intermediate object) held in the left hand 42A to the left hand 42B of the avatar 40B as the second runner. As a result, the baton is gripped by the left hand 42B, so the relationship between the left hand 42A and the left hand 42B satisfies the first condition. Therefore, when such a baton delivery is performed, the processor 10 can control the operations of the avatar 40A and the avatar 40B in the virtual space 2 in a manner corresponding to the baton delivery.

  In step S37, the processor 10 can also determine whether or not the relationship between the left hand 42A of the avatar 40A and the right hand 41B of the avatar 40B satisfies the first condition. In step 37, the processor 10 can also determine whether the relationship between the right hand 41A of the avatar 40A and the left hand 42B of the avatar 40B satisfies the first condition. Without being limited thereto, the processor 10 determines whether or not the relationship between an arbitrary first operation object included in the avatar 40A and an arbitrary second operation object included in the avatar 40B satisfies the first condition. Can do.

  In step S38, the processor 10 can also control at least one of the left hand 42A and the left hand 42B in the virtual space 2 while the position of the virtual camera 1 is fixed. For example, in step S38, the processor 10 moves the left hand 42A and the left hand 42B so as to approach the avatar 40A or the avatar 40B. In this case, triggered by the establishment of the first condition, the avatar 40B automatically pulls the left hand 42A of the avatar 40A, or the avatar 40A automatically pulls the left hand 42B of the avatar 40B.

  In the example of FIG. 15, the processor 10 can also move the avatar 40 </ b> A and the avatar 40 </ b> B only in the direction included in the view area 23. Thereby, since it can prevent that the avatar 40A moves to the direction which the user 190A cannot visually recognize, it can prevent a user being confused.

  In the example of FIG. 15, the processor 10 can prompt the user 190A to grip the left hand 42B with the left hand 42A. For example, the processor 10 changes the color of the left hand 42A when the left hand 42A is gripped by the left hand 42B. Thereby, the user 190A can recognize that the left hand 42A is required to hold back the left hand 42B by visually recognizing that the color of the left hand 42A has been changed. For example, when the user 190A is using the controller 160 to operate the left hand 42A, the processor 10 vibrates the controller 160 when the left hand 42A is gripped by the left hand 42B. Accordingly, the user 190A can recognize that the left hand 42A is required to hold back the left hand 42B by feeling vibration with the left hand.

  As illustrated in FIG. 16, when the avatar 40B faces the avatar 40A, it is preferable that the processor 10 determines the relationship between the left hand 42A and the left hand 42B based on the stricter first condition. For example, when the left hand 42A and the left hand 42B are in contact with each other, the processor 10 determines that the relationship between the left hand 42A and the left hand 42B satisfies the first condition. Thereby, it is possible to more effectively prevent the erroneous determination that the first condition is satisfied.

  In the example of FIG. 20, when the first condition is satisfied, the processor 10 changes the avatar 40 </ b> B to the avatar 40 </ b> A as a target for changing game parameters such as hit points when receiving an attack from the enemy character 62. You can also. Such a target change is also included as a kind of change of position. In addition, a player change in a sports game in which a plurality of users play together in the virtual space 2 is also included as a kind of interchange of the positions of the avatar 40A and the avatar 40B in the virtual space 2.

  In the present embodiment, the virtual space (VR space) in which the user 190 is immersed by the HMD device 110 has been described as an example. However, as the HMD device 110, a transmissive HMD device may be employed. In this case, the augmented reality (AR) is output by outputting the view field image so that a part of the image constituting the virtual space is superimposed as the view field image on the real space visually recognized by the user 190 via the transmissive HMD device. : A virtual reality experience in an augmented reality (MR) space or a mixed reality (MR) space may be provided to the user 190. In this case, instead of the hand object H, an action on the target object (for example, the display object D) in the virtual space 2 may be generated based on the hand movement of the user 190. Specifically, the processor 10 may specify the coordinate information of the position of the hand of the user 190 in the real space and define the position of the target object in the virtual space 2 in relation to the coordinate information in the real space. . Thereby, the processor 10 grasps the positional relationship between the hand of the user 190 in the real space and the target object in the virtual space 2, and performs processing corresponding to the above-described collision control between the hand of the user 190 and the target object. It becomes executable. As a result, it is possible to act on the target object based on the hand movement of the user 190.

[Additional Notes]
The contents according to one aspect of the present invention are listed as follows.

  (Item 1) The program was explained. According to one aspect of the present disclosure, a program (10) provides a virtual experience to a first user via an image display device (display 112) associated with the head of the first user (190A). ) (Computer 200 or computer included in server 150). The program defines a virtual space 2 for providing the virtual experience to the first user in the processor 10 (S14), and a program in the virtual space 2 according to the movement of a part of the body of the first user. Depending on the step of moving one operation object (the left hand 42A) (S34), the posture of the head of the first user and the position of the virtual viewpoint (virtual camera 1) in the virtual space 2, the virtual viewpoint 2 in the virtual space 2 A step (step S33) of controlling the field of view (field of view region 23) and a step of moving the second operation object (left hand 42B) in the virtual space 2 in accordance with the movement of a part of the body of the second user (user 190B) And when the relationship between the first operation object and the second operation object in the virtual space 2 satisfies the first condition, the first operation object according to the first condition , Controlling at least one of the second operation object and the view from the virtual viewpoint in the virtual space 2, defining a view image M corresponding to the view from the virtual viewpoint (S33), and the view image M Is output to the image display device (S33).

  (Item 2) In (Item 1), the first condition is that at least one of selection of the second operation object by the first operation object and selection of the first operation object by the second operation object is performed. is there.

  (Item 3) In (Item 1), the first condition is that the first operation object and the second operation object are further moved in the same direction 53.

  (Item 4) In (Item 1), the first condition is that the first operation object and the second operation object are in the first positional relationship and are defined in the first direction (roll direction) defined by the first operation object. The second direction (roll direction) defined by the second operation object is opposed to the second operation object.

  (Item 5) In (Item 1) to (Item 4), the first condition is that the first operation object and the second operation object are close to each other and the first operation object and the second operation object are in the first positional relationship. It became that.

  (Item 6) In (Item 2) to (Item 5), the first condition is that the first operation object and the second operation object are further included in the field of view from the virtual viewpoint.

  (Item 7) In (Item 1) to (Item 6), in the controlling step, the first operation object follows the movement of the second operation object while maintaining the positional relationship between the first operation object and the virtual viewpoint. The virtual viewpoint is moved in the virtual space 2.

  (Item 8) In any one of (Item 1) to (Item 6), in the controlling step, a first avatar (avatar 40A) associated with the first operation object and a second avatar (associated with the second operation object) The position in the virtual space 2 with the avatar 40B) is changed.

  (Item 9) The information processing apparatus has been described. According to an aspect of the present disclosure, the information processing device (computer 200) performs a virtual experience through the image display device (display 112) associated with the head of the first user (user 190A) and the image display device. In order to provide to one user, a storage unit (memory 11) that stores a program executed by the information processing apparatus and a control unit (processor 10) that controls the operation of the information processing apparatus are provided. The control unit defines the virtual space 2 for providing the virtual experience to the first user, and moves the first operation object (the left hand 42A) in the virtual space according to the movement of a part of the body of the first user. The view from the virtual viewpoint in the virtual space 2 is controlled according to the posture of the head of the first user and the position of the virtual viewpoint (virtual camera 1) in the virtual space 2, and the body of the second user (user 190B) When the second operation object (left hand 42B) is moved in the virtual space 2 in accordance with a partial movement of the first movement object and the relationship between the first operation object and the second operation object in the virtual space 2 satisfies the first condition, According to the first condition, at least one of the first operation object, the second operation object, and the field of view from the virtual viewpoint (view field region 23) is controlled in the virtual space 2, and the virtual Define the view field image M corresponding to the view from the point, and outputs the view field image M on the image display device.

  (Item 10) A method of executing a program has been described. According to one aspect of the present disclosure, the program causes the processor 10 to provide a virtual experience to the first user via an image display device (display 112) associated with the head of the first user (user 190A). It is executed by the provided computer (the computer provided in the computer 200 or the server 150). The method defines, in the processor 10, a virtual space 2 for providing a virtual experience to the first user (S13), and in the virtual space 2 according to the movement of a part of the body of the first user. The field of view from the virtual viewpoint in the virtual space according to the step (S34) of moving one operation object (the left hand 42A) and the position of the head of the first user and the position of the virtual viewpoint (virtual camera 1) in the virtual space 2 A step (S32) of controlling the (viewing area 23) and a step of moving the second operation object (left hand 42B) in the virtual space 2 according to the movement of a part of the body of the second user (user 190B) (S36) ) And the first operation object and the second operation object in the virtual space 2 satisfy the first condition, the first operation object, Controlling at least one of the view from the operation object and the virtual viewpoint in the virtual space 2 (S38), defining a view image M corresponding to the view from the virtual viewpoint (S33), and the view image M Is output to the image display device (S33).

1 virtual camera, 2 virtual space, 5 reference line of sight, 10 processor, 11 memory, 12 storage, 13 input / output interface, 14 communication interface, 15 bus, 19 network, 21 center, 22 virtual space image, 23 viewing area, 24, 25 area, 31 frame, 32 top surface, 33, 34, 36, 37 button, 35 infrared LED, 38 analog stick, 100, 100A, 100B, 100C HMD system, 110, 110A, 110B, 110C HMD device, 112 display, 114 sensor, 116 camera, 118 microphone, 120 HMD sensor, 130 motion sensor, 140 gaze sensor, 150 server, 160 controller, 160R right controller, 190, 190A, 190B, 190C user, 00 computer, 220 display control module, 221 virtual camera control module, 222 visual field determination module, 223 visual field image generation module, 224 reference visual line identification module, 230 virtual space control module, 231 virtual space definition module, 232 virtual object control module, 233 synchronization control module, 234 determination module, 240 memory module, 241 content information, 242 object information, 243 user information, 250 communication control module, 810 right hand, 40A, 40B, 40C avatar, 41A, 41B, 41C right hand, 42A, 42B 42C Left hand,
51, 52, 53A, 53B, 54A, 54B Direction 55A, 55B Collision area, 61A, 61B Weapon, 62 Enemy character, M view image

Claims (6)

A program executed by a computer with a processor to provide a virtual experience to the first user via an image display device associated with the head of the first user,
The program is stored in the processor.
Defining a virtual space for providing the virtual experience to the first user;
Moving the first operating object in the virtual space in response to movement of a part of the body of the first user;
Controlling the field of view from the virtual viewpoint in the virtual space according to the posture of the head of the first user and the position of the virtual viewpoint in the virtual space;
Moving the second operation object in the virtual space in response to movement of a part of the body of the second user;
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space in the virtual space;
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the visual field image to the image display device ,
The first condition is that at least one of selection of the second operation object by the first operation object and selection of the first operation object by the second operation object is performed.
The first condition is a program in which the first operation object and the second operation object are further moved in the same direction. A program executed by a computer with a processor to provide a virtual experience to the first user via an image display device associated with the head of the first user,
The program is stored in the processor.
Defining a virtual space for providing the virtual experience to the first user;
Moving the first operating object in the virtual space in response to movement of a part of the body of the first user;
Controlling the field of view from the virtual viewpoint in the virtual space according to the posture of the head of the first user and the position of the virtual viewpoint in the virtual space;
Moving the second operation object in the virtual space in response to movement of a part of the body of the second user;
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space in the virtual space;
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the visual field image to the image display device,
The program which changes the position in the said virtual space of the 1st avatar linked | related with the said 1st operation object, and the 2nd avatar linked | related with the said 2nd operation object in the said step to control. An information processing apparatus,
The information processing apparatus includes:
An image display device associated with the head of the first user;
A storage unit for storing a program executed by the information processing device to provide a virtual experience to the first user via the image display device;
A control unit for controlling the operation of the information processing apparatus,
The controller is
Defining a virtual space for providing the virtual experience to the first user;
In response to the movement of a part of the body of the first user, the first operation object is moved in the virtual space,
According to the posture of the first user's head and the position of the virtual viewpoint in the virtual space, the field of view from the virtual viewpoint in the virtual space is controlled,
In response to the movement of a part of the body of the second user, the second operation object is moved in the virtual space,
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space,
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the view image to the image display device;
The first condition is that at least one of selection of the second operation object by the first operation object and selection of the first operation object by the second operation object is performed.
The information processing apparatus, wherein the first condition is that the first operation object and the second operation object are moving in the same direction. An information processing apparatus,
The information processing apparatus includes:
An image display device associated with the head of the first user;
A storage unit for storing a program executed by the information processing device to provide a virtual experience to the first user via the image display device;
A control unit for controlling the operation of the information processing apparatus,
The controller is
Defining a virtual space for providing the virtual experience to the first user;
In response to the movement of a part of the body of the first user, the first operation object is moved in the virtual space,
According to the posture of the first user's head and the position of the virtual viewpoint in the virtual space, the field of view from the virtual viewpoint in the virtual space is controlled,
In response to the movement of a part of the body of the second user, the second operation object is moved in the virtual space,
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space,
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the view image to the image display device;
When the control unit controls at least one of the information, the position of the first avatar associated with the first operation object and the second avatar associated with the second operation object in the virtual space is switched. Processing equipment. A method in which a computer with a processor executes a program to provide a virtual experience to the first user via an image display device associated with the head of the first user,
The method includes:
Defining a virtual space for providing the virtual experience to the first user;
Moving the first operating object in the virtual space in response to movement of a part of the body of the first user;
Controlling the field of view from the virtual viewpoint in the virtual space according to the posture of the head of the first user and the position of the virtual viewpoint in the virtual space;
Moving the second operation object in the virtual space in response to movement of a part of the body of the second user;
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space in the virtual space;
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the field-of-view image to the image display device,
The first condition is that at least one of selection of the second operation object by the first operation object and selection of the first operation object by the second operation object is performed.
The method is further characterized in that the first condition is that the first operation object and the second operation object are moving in the same direction. A method in which a computer with a processor executes a program to provide a virtual experience to the first user via an image display device associated with the head of the first user,
The method includes:
Defining a virtual space for providing the virtual experience to the first user;
Moving the first operating object in the virtual space in response to movement of a part of the body of the first user;
Controlling the field of view from the virtual viewpoint in the virtual space according to the posture of the head of the first user and the position of the virtual viewpoint in the virtual space;
Moving the second operation object in the virtual space in response to movement of a part of the body of the second user;
When the relationship between the first operation object and the second operation object in the virtual space satisfies the first condition, the view from the first operation object, the second operation object, and the virtual viewpoint according to the first condition Controlling at least one of the virtual space in the virtual space;
Defining a view image corresponding to the view from the virtual viewpoint;
Outputting the field-of-view image to the image display device,
In the controlling step, a method of exchanging positions in the virtual space between a first avatar associated with the first operation object and a second avatar associated with the second operation object.

Images