JP6203346B1 - Method, program, and recording medium for providing virtual space - Google Patents

Abstract

An object of the present invention is to improve a sense of immersion in a virtual space. A control circuit unit (200) changes an attribute of a virtual object placed in a virtual space (2) based on a first operation (step S21) by a user (step S23), and then performs a second operation. The movement destination of the virtual camera (1) is specified based on the attribute of the virtual object at the time when the operation is input (step S26) (step S28). [Selection] Figure 11

Description

  The present disclosure relates to a method, a program, and a recording medium that provide a virtual space.

Patent Document 1 discloses a head-mounted display (Head-Mounted Dis).
It is described that the viewpoint at the time of providing video content to a user using play (HMD) is switched from a third person viewpoint to a first person viewpoint that is a viewpoint from a shooting target appearing in the video content.

JP 2014-127987 A (released on July 7, 2014)

  Unlike the case of enjoying video content as disclosed in Patent Document 1 with an HMD, the viewpoint position is changed in providing virtual space content that changes the field of view in conjunction with the movement of the head of the user wearing the HMD. It is necessary to be careful not to impair the sense of immersion in the virtual space. For example, it is assumed that a certain degree of restriction is imposed on the movement of the virtual space so as not to lose the pleasure of developing a strategy for capturing the game. However, too much emphasis is placed on movement restrictions, and if the operability at the time of movement deteriorates, the user may be stressed during the game and the sense of immersion in the virtual space may be impaired. It can be.

  This indication is made in order to solve the above-mentioned subject. The purpose is to improve the sense of immersion in the virtual space.

  In order to solve the problem, a method of providing a virtual space according to the present disclosure is a method of providing a virtual space to a user wearing a head-mounted display (hereinafter, HMD) on a head, A step of specifying a reference line of sight, a step of specifying a virtual camera that is arranged in the virtual space and is set to be viewed by the user based on the reference line of sight, and an object that specifies a destination of the virtual camera are arranged in the virtual space Receiving a first operation input by the user, changing an attribute of the object according to the first operation, receiving a second operation input by the user after the first operation, Based on the information about the position in the virtual space included in the attribute of the object at the time when A identifying a Dosaki, and moving the virtual camera to the destination.

  According to the present disclosure, it is possible to improve the feeling of immersion in the virtual space.

It is a figure which shows the structure of a HMD system. It is a figure which shows the hardware constitutions of a control circuit part. It is a figure which illustrates the visual field coordinate system set to HMD. It is a figure explaining the outline | summary of the virtual space provided to a user. It is a figure which shows the cross section of a visual field area | region. It is a figure which illustrates the method of determining a user's gaze direction. It is a figure showing the structure of a right controller. It is a block diagram which shows the functional structure of a control circuit part. It is a sequence diagram which shows the flow of the process in which an HMD system provides a virtual space to a user. It is a figure explaining an example of the moving method of the virtual camera in virtual space. It is a sequence diagram explaining the flow of processing of HMD system 100 which realizes an example of a movement method. It is a figure explaining the other example of the moving method of the virtual camera in virtual space. It is a sequence diagram explaining the flow of a process of the HMD system 100 which implement | achieves the other example of the movement method. It is a figure explaining the further another example of the moving method of the virtual camera in virtual space. It is a sequence diagram explaining the flow of a process of the HMD system which implement | achieves the further another example of the movement method.

[Details of the embodiment of the present invention]
A specific example of a method and program for providing a virtual space according to an embodiment of the present invention will be described below with reference to the drawings. The present invention is not limited to these exemplifications, but is defined by the scope of claims for patent, and it is intended that all modifications within the meaning and scope equivalent to the scope of claims for patent are included in the present invention. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and repeated description is not repeated.

(Configuration of HMD system 100)
FIG. 1 is a diagram showing a configuration of the HMD system 100. As shown in this figure, the HMD system 100 includes an HMD 110, an HMD sensor 120, a controller sensor 140, a control circuit unit 200, and a controller 300.

  The HMD 110 is worn on the user's head. The HMD 110 includes a display 112 which is a non-transmissive display device, a sensor 114, and a gaze sensor 130. The HMD 110 displays a right-eye image and a left-eye image on the display 112, thereby allowing the user to visually recognize a three-dimensional image that is stereoscopically viewed by the user based on the parallax between both eyes of the user. This provides a virtual space to the user. Since the display 112 is disposed in front of the user's eyes, the user can immerse in the virtual space through an image displayed on the display 112. Thereby, the user can experience virtual reality (Virtual Reality: VR). The virtual space may include a background, various objects that can be operated by the user, menu images, and the like.

  The display 112 may include a right-eye sub-display that displays a right-eye image and a left-eye sub-display that displays a left-eye image. Alternatively, the display 112 may be a single display device that displays the right-eye image and the left-eye image on a common screen. As such a display device, for example, a display device that alternately displays a right-eye image and a left-eye image independently by switching a shutter so that the display image can be recognized only by one eye. .

  In the present embodiment, a transmissive display may be applied to the HMD 110. That is, the HMD 110 may be a transmissive HMD. In this case, a virtual object to be described later can be virtually arranged in the real space by displaying the three-dimensional image on the transmissive display. Thereby, the user can experience mixed reality (Mixes Reality: MR) in which virtual objects are arranged in the real space. In the present embodiment, an experience such as virtual reality or mixed reality in which a user can interact with a virtual object may be referred to as a virtual experience. Hereinafter, a method for providing virtual reality will be described in detail as an example.

(Hard structure of the control circuit unit 200)
FIG. 2 is a diagram illustrating a hardware configuration of the control circuit unit 200. The control circuit unit 200 is a computer for causing the HMD 110 to provide a virtual space. As shown in FIG. 2, the control circuit unit 200 includes a processor, a memory, a storage, an input / output interface, and a communication interface. These are connected to each other in the control circuit unit 200 through a bus as a data transmission path.

  The processor includes a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), and the like, and controls operations of the control circuit unit 200 and the HMD system 100 as a whole.

  The memory functions as main memory. The memory stores a program processed by the processor and control data (such as calculation parameters). The memory may include a ROM (Read Only Memory) or a RAM (Random Access Memory).

  The storage functions as auxiliary storage. The storage stores a program for controlling the operation of the entire HMD system 100, various simulation programs, a user authentication program, and various data (images, objects, etc.) for defining a virtual space. Furthermore, a database including a table for managing various data may be constructed in the storage. The storage can be configured to include a flash memory or an HDD (Hard Disc Drive).

  The input / output interface includes various wired connection terminals such as a USB (Universal Serial Bus) terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, and various wireless connection terminals. These processing circuits are included. The input / output interface connects the HMD 110, various sensors including the HMD sensor 120 and the controller sensor 140, and the controller 300 to each other.

  The communication interface includes various wired connection terminals for communicating with an external device via the network NW, and various processing circuits for wireless connection. The communication interface is configured to conform to various communication standards and protocols for communicating via a LAN (Local Area Network) or the Internet.

  The control circuit unit 200 provides a virtual space to the user by loading a predetermined application program stored in the storage into a memory and executing it. When the program is executed, the memory and the storage store various programs for operating various virtual objects arranged in the virtual space and displaying and controlling various menu images.

  The control circuit unit 200 may or may not be mounted on the HMD 110. That is, the control circuit unit 200 may be another hardware independent of the HMD 110 (for example, a personal computer or a server device that can communicate with the HMD 110 through a network). The control circuit unit 200 may be a device in which one or a plurality of functions are implemented by cooperation of a plurality of hardware. Alternatively, only a part of all the functions of the control circuit unit 200 may be mounted on the HMD 110, and the remaining functions may be mounted on different hardware.

  A global coordinate system (reference coordinate system, xyz coordinate system) is set in advance for each element such as the HMD 110 constituting the HMD system 100. This global coordinate system has three reference directions (axes) parallel to the vertical direction, the horizontal direction orthogonal to the vertical direction, and the front-rear direction orthogonal to both the vertical direction and the horizontal direction in the real space. In the present embodiment, since the global coordinate system is a kind of viewpoint coordinate system, the horizontal direction, vertical direction (vertical direction), and front-rear direction in the global coordinate system are set as an x-axis, a y-axis, and a z-axis, respectively. Specifically, the x-axis of the global coordinate system is parallel to the horizontal direction of the real space, the y-axis is parallel to the vertical direction of the real space, and the z-axis is parallel to the front-rear direction of the real space.

  The HMD sensor 120 has a position tracking function for detecting the movement of the HMD 110. With this function, the HMD sensor 120 detects the position and inclination of the HMD 110 in the real space. In order to realize this detection, the HMD 110 includes a plurality of light sources (not shown). Each light source is, for example, an LED that emits infrared rays. The HMD sensor 120 includes, for example, an infrared sensor. The HMD sensor 120 detects the detection point of the HMD 110 by detecting the infrared ray irradiated from the light source of the HMD 110 with the infrared sensor. Further, based on the detection value of the detection point of the HMD 110, the position and inclination of the HMD 110 in the real space according to the user's movement are detected. The HMD sensor 120 can determine the time change of the position and inclination of the HMD 110 based on the change with time of the detected value.

  The HMD sensor 120 may include an optical camera. In this case, the HMD sensor 120 detects the position and inclination of the HMD 110 based on the image information of the HMD 110 obtained by the optical camera.

  Instead of the HMD sensor 120, the HMD 110 may detect its position and inclination using the sensor 114. In this case, the sensor 114 may be an angular velocity sensor, a geomagnetic sensor, an acceleration sensor, or a gyro sensor, for example. The HMD 110 uses at least one of these. When the sensor 114 is an angular velocity sensor, the sensor 114 detects the angular velocity around the three axes in the real space of the HMD 110 over time according to the movement of the HMD 110. The HMD 110 can determine the temporal change of the angle around the three axes of the HMD 110 based on the detected value of the angular velocity, and can detect the inclination of the HMD 110 based on the temporal change of the angle.

  When the HMD 110 detects the position and inclination of the HMD 110 based on the detection value of the sensor 114, the HMD sensor 120 is not necessary for the HMD system 100. Conversely, when the HMD sensor 120 arranged at a position away from the HMD 110 detects the position and inclination of the HMD 110, the sensor 114 is not necessary for the HMD 110.

  As described above, the global coordinate system is parallel to the real space coordinate system. Therefore, each inclination of the HMD 110 detected by the HMD sensor 120 corresponds to each inclination around the three axes of the HMD 110 in the global coordinate system. The HMD sensor 120 sets the uvw visual field coordinate system to the HMD 110 based on the detected value of the inclination of the HMD sensor 120 in the global coordinate system. The uvw visual field coordinate system set in the HMD 110 corresponds to a viewpoint coordinate system when a user wearing the HMD 110 views an object.

(Uuv visual field coordinate system)
FIG. 3 is a diagram illustrating a uwv visual field coordinate system set in the HMD 110. The HMD sensor 120 detects the position and inclination of the HMD 110 in the global coordinate system when the HMD 110 is activated. Then, a three-dimensional uvw visual field coordinate system based on the detected tilt value is set in the HMD 110. As shown in FIG. 3, the HMD sensor 120 sets a three-dimensional uvw visual field coordinate system around the head of the user wearing the HMD 110 as the center (origin) in the HMD 110. Specifically, the horizontal direction, the vertical direction, and the front-rear direction (x axis, y axis, z axis) that define the global coordinate system are respectively set around each axis by an inclination around each axis of the HMD 110 in the global coordinate system. Three new directions obtained by tilting are set as the pitch direction (u-axis), yaw direction (v-axis), and roll direction (w-axis) of the uvw visual field coordinate system in the HMD 110.

  As shown in FIG. 3, when the user wearing the HMD 110 stands upright and visually recognizes the front, the HMD sensor 120 sets the uvw visual field coordinate system parallel to the global coordinate system to the HMD 110. In this case, the horizontal direction (x-axis), the vertical direction (y-axis), and the front-back direction (z-axis) of the global coordinate system are the same as the pitch direction (u-axis) and yaw direction (v of the uvw visual field coordinate system in the HMD 110. Axis) and the roll direction (w-axis).

  After setting the uvw visual field coordinate system in the HMD 110, the HMD sensor 120 can detect the inclination (the amount of change in inclination) of the HMD 110 in the currently set uvw visual field coordinate system in accordance with the movement of the HMD 110. In this case, the HMD sensor 120 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 110 in the currently set uvw visual field coordinate system as the inclination of the HMD 110. The pitch angle (θu) is an inclination angle of the HMD 110 around the pitch direction in the uvw visual field coordinate system. The yaw angle (θv) is an inclination angle of the HMD 110 around the yaw direction in the uvw visual field coordinate system. The roll angle (θw) is an inclination angle of the HMD 110 around the roll direction in the uvw visual field coordinate system.

  Based on the detected value of the inclination of the HMD 110, the HMD sensor 120 newly sets the uvw visual field coordinate system in the HMD 110 after moving to the HMD 110. The relationship between the HMD 110 and the uvw visual field coordinate system of the HMD 110 is always constant regardless of the position and inclination of the HMD 110. When the position and inclination of the HMD 110 change, the position and inclination of the uvw visual field coordinate system of the HMD 110 in the global coordinate system similarly change in conjunction with the change.

  The HMD sensor 120 determines the position of the HMD 110 in the real space relative to the HMD sensor 120 based on the infrared light intensity acquired by the infrared sensor and the relative positional relationship (distance between the detection points) between the plurality of detection points. You may specify as a position. The origin of the uvw visual field coordinate system of the HMD 110 in the real space (global coordinate system) may be determined based on the specified relative position. The HMD sensor 120 detects the inclination of the HMD 110 in the real space based on the relative positional relationship between the plurality of detection points, and further, based on the detected value, the uvw visual field coordinates of the HMD 110 in the real space (global coordinate system). The orientation of the system may be determined.

(Outline of virtual space 2)
FIG. 4 is a diagram illustrating an outline of the virtual space 2 provided to the user. As shown in this figure, the virtual space 2 has a spherical structure that covers the entire 360 ° direction of the center 21. FIG. 4 illustrates only the upper half celestial sphere in the entire virtual space 2. A plurality of substantially square or substantially rectangular meshes are associated with the virtual space 2. The position of each mesh in the virtual space 2 is defined in advance as coordinates in a space coordinate system (XYZ coordinate system) defined in the virtual space 2. The control circuit unit 200 associates each partial image constituting content (still image, moving image, etc.) that can be developed in the virtual space 2 with each corresponding mesh in the virtual space 2, thereby enabling the virtual space image that can be visually recognized by the user. The virtual space 2 in which 22 is expanded is provided to the user.

  The virtual space 2 defines an XYZ space coordinate system with the center 21 as the origin. The XYZ coordinate system is parallel to the global coordinate system, for example. Since the XYZ coordinate system is a kind of viewpoint coordinate system, the horizontal direction, vertical direction (vertical direction), and front-rear direction in the XYZ coordinate system are defined as an X axis, a Y axis, and a Z axis, respectively. That is, the X axis (horizontal direction) of the XYZ coordinate system is parallel to the x axis of the global coordinate system, the Y axis (vertical direction) of the XYZ coordinate system is parallel to the y axis of the global coordinate system, and The Z axis (front-rear direction) is parallel to the z axis of the global coordinate system.

  When the HMD 110 is activated (initial state), the virtual camera 1 is arranged at the center 21 of the virtual space 2. The virtual camera 1 moves similarly in the virtual space 2 in conjunction with the movement of the HMD 110 in the real space. Thereby, changes in the position and orientation of the HMD 110 in the real space are similarly reproduced in the virtual space 2.

  As with the HMD 110, the uvw visual field coordinate system is defined for the virtual camera 1. The uvw visual field coordinate system of the virtual camera 1 in the virtual space 2 is defined to change to the uvw visual field coordinate system of the HMD 110 in the real space (global coordinate system). Therefore, when the inclination of the HMD 110 changes, the inclination of the virtual camera 1 also changes accordingly. The virtual camera 1 can also move in the virtual space 2 in conjunction with the movement of the user wearing the HMD 110 in the real space.

  The orientation of the virtual camera 1 in the virtual space 2 is determined according to the position and inclination of the virtual camera 1 in the virtual space 2. As a result, a line of sight (reference line of sight 5) as a reference when the user visually recognizes the virtual space image 22 developed in the virtual space 2 is determined. The control circuit unit 200 determines the visual field region 23 in the virtual space 2 based on the reference visual line 5. The visual field area 23 is an area corresponding to the visual field of the user wearing the HMD 110 in the virtual space 2.

  FIG. 5 is a view showing a cross section of the visual field region 23. FIG. 5A shows a YZ cross section of the visual field region 23 viewed from the X direction in the virtual space 2. FIG. 5B shows an XZ cross-section of the visual field region 23 viewed from the Y direction in the virtual space 2. The visual field region 23 is defined by the first region 24 (see FIG. 5A) that is a range defined by the reference line of sight 5 and the YZ section of the virtual space 2, and the reference line of sight 5 and the XZ section of the virtual space 2. And a second region 25 (see FIG. 5B) which is a defined range. The control circuit unit 200 sets a range including the polar angle α around the reference line of sight 5 in the virtual space 2 as the first region 24. In addition, a range including the azimuth angle β around the reference line of sight 5 in the virtual space 2 is set as the second region 25.

  The HMD system 100 provides the user with the virtual space 2 by causing the display image 112 of the HMD 110 to display a view image 26 that is a portion of the virtual space image 22 that is superimposed on the view region 23. If the user moves the HMD 110, the virtual camera 1 also moves in conjunction with it, and as a result, the position of the visual field area 23 in the virtual space 2 changes. As a result, the view image 26 displayed on the display 112 is updated to an image that is superimposed on a portion of the virtual space image 22 where the user faces (= view region 23). Therefore, the user can visually recognize a desired location in the virtual space 2.

  While wearing the HMD 110, the user visually recognizes only the virtual space image 22 developed in the virtual space 2 without seeing the real world. Therefore, the HMD system 100 can give a high immersive feeling to the virtual space 2 to the user.

  The control circuit unit 200 may move the virtual camera 1 in the virtual space 2 in conjunction with the movement of the user wearing the HMD 110 in the real space. In this case, the control circuit unit 200 identifies the visual field region 23 that is visually recognized by the user by being projected on the display 112 of the HMD 110 in the virtual space 2 based on the position and orientation of the virtual camera 1 in the virtual space 2.

  The virtual camera 1 preferably includes a right-eye virtual camera that provides a right-eye image and a left-eye virtual camera that provides a left-eye image. Furthermore, it is preferable that appropriate parallax is set for the two virtual cameras so that the user can recognize the three-dimensional virtual space 2. In the present embodiment, only the virtual camera 1 in which the roll direction (w) generated by combining the roll directions of the two virtual cameras is adapted to the roll direction (w) of the HMD 110 is represented. Will be shown and described.

(Gaze direction detection)
The gaze sensor 130 has an eye tracking function that detects the direction (gaze direction) in which the line of sight of the user's right eye and left eye is directed. As the gaze sensor 130, a known sensor having an eye tracking function can be employed. The gaze sensor 130 preferably includes a right eye sensor and a left eye sensor. The gaze sensor 130 may be, for example, a sensor that detects the rotation angle of each eyeball by irradiating the user's right eye and left eye with infrared light and receiving reflected light from the cornea and iris with respect to the irradiated light. The gaze sensor 130 can detect the direction of the user's line of sight based on each detected rotation angle.

  The user's gaze direction detected by the gaze sensor 130 is a direction in the viewpoint coordinate system when the user visually recognizes the object. As described above, the uvw visual field coordinate system of the HMD 110 is equal to the viewpoint coordinate system when the user visually recognizes the display 112. Further, the uvw visual field coordinate system of the virtual camera 1 is linked to the uvw visual field coordinate system of the HMD 110. Therefore, in the HMD system 100, the user's line-of-sight direction detected by the gaze sensor 130 can be regarded as the user's line-of-sight direction in the uvw visual field coordinate system of the virtual camera 1.

  FIG. 6 is a diagram illustrating a method of determining the user's line-of-sight direction. As shown in this figure, the gaze sensor 130 detects the line of sight of the right eye and the left eye of the user U. When the user U is looking nearby, the gaze sensor 130 detects the line of sight R1 and L1 of the user U. When the user looks far away, the gaze sensor 130 identifies the line of sight R2 and L2 that are smaller in angle with the roll direction (w) of the HMD 110 than the line of sight R1 and L1 of the user. The gaze sensor 130 transmits the detection value to the control circuit unit 200.

  When receiving the line of sight R1 and L1 as the line-of-sight detection value, the control circuit unit 200 specifies the gazing point N1 that is the intersection of the two. On the other hand, also when the lines of sight R2 and L2 are received, the gazing point N1 (not shown) that is the intersection of the two is specified. The control circuit unit 200 detects the gaze direction N0 of the user U based on the identified gazing point N1. For example, the control circuit unit 200 detects, as the line-of-sight direction N0, the direction in which the straight line passing through the midpoint of the straight line connecting the right eye R and the left eye L of the user U and the gazing point N1 extends. The line-of-sight direction N0 is a direction in which the user U actually points the line of sight with both eyes. The line-of-sight direction N0 is also a direction in which the user U actually directs his / her line of sight with respect to the visual field region 23.

  The HMD system 100 may include a microphone and a speaker in any element constituting the HMD system 100. Thereby, the user can give a voice instruction to the virtual space 2. Further, in order to receive a broadcast of a television program on a virtual television in a virtual space, the HMD system 100 may include a television receiver as any element. Further, it may include a communication function or the like for displaying an electronic mail or the like acquired by the user.

(Controller 300)
The controller 300 is an example of an apparatus used to control the movement of a virtual object by detecting the movement of a part of the user's body. As shown in FIG. 1, the controller 300 includes a right controller 320 that the user uses with the right hand and a left controller 330 that the user uses with the left hand. The right controller 320 and the left controller 330 are configured as separate devices. The user moves the right hand with the right controller 320 and the left hand with the left controller 330 to move a virtual object (hereinafter referred to as a movable object, or more specifically, depending on the situation). , Virtual hand, virtual right hand, virtual left hand, etc.) can be moved freely. The method for detecting the movement of a part of the body other than the user's head is not limited to an example using a controller including a sensor attached to the part of the body. An optical method or the like can be applied. For example, by identifying the initial position of a part of the user's body using an external camera and continuously identifying the position of the part of the user's body, the movement of the part of the body other than the user's head Can be detected. In the following description, detection of movement of a part of the body other than the user's head using the controller 300 will be described in detail.

  As shown in FIG. 1, the right controller 320 and the left controller 330 each include an operation button 302, an infrared LED (Light Emitting Diode) 304, a sensor 306, and a transceiver 308. The right controller 320 and the left controller 330 may include only one of the infrared LED 304 and the sensor 306. FIG. 7 is a diagram illustrating the configuration of the right controller 320. In the following description, since the right controller 320 and the left controller 330 have a common configuration, only the configuration of the right controller 320 will be described.

  The controller sensor 140 has a position tracking function for detecting the movement of the right controller 320. The controller sensor 140 detects the position and inclination of the right controller 320 in the real space. The controller sensor 140 detects the infrared light emitted from the infrared LED 304 of the right controller 320, respectively. The controller sensor 140 includes an infrared camera that captures an image in the infrared wavelength region, and detects the position and inclination of the right controller 320 based on image data captured by the infrared camera.

  Instead of the controller sensor 140, the right controller 320 may detect its position and tilt using the sensor 306. In this case, the three-axis angular velocity sensor (sensor 306) of the right controller 320 detects the rotation of the right controller 320 around each of three orthogonal axes. The right controller 320 detects how much the right controller 320 has rotated in which direction based on each detection value, and calculates the inclination of the right controller 320 by accumulating the sequentially detected rotation direction and rotation amount. To do. The right controller 320 may use the detection value from the triaxial magnetic sensor and / or the triaxial acceleration sensor in addition to the detection value of the triaxial angular velocity sensor.

  The operation buttons 302 are a group of buttons configured to receive operation inputs from the user for the controller 300. In the present embodiment, the operation button 302 includes a push button, a trigger button, and an analog stick.

  The push-type button is a button configured to be operated by an operation of pressing downward with a thumb. The right controller 320 includes thumb buttons 302a and 302b on the top surface 322 as push buttons. Both thumb buttons 302a and 302b are operated (pressed) with the thumb of the right hand. The thumb buttons 302a and 302b are pressed by the thumb of the right hand, or the thumb is placed on the top surface 322, whereby the thumb of the virtual right hand is changed from the extended state to the bent state.

  The trigger type button is a button configured to be operated by an operation of pulling a trigger with an index finger or a middle finger. The right controller 320 includes an index finger button 302e on the front surface of the grip 324 as a trigger type button. By operating the index finger button 302e with the index finger of the right hand, the index finger of the virtual right hand is changed from the extended state to the bent state. Further, a middle finger button 302f is provided on a side surface portion of the grip 324. By operating the middle finger button 302f with the middle finger of the right hand, the middle finger, ring finger, and little finger of the virtual right hand are changed to the bent state.

  The right controller 320 detects the pressed state of the thumb buttons 302 a and 302 b, the index finger button 302 e, and the middle finger button 302 f, and outputs these detected values to the control circuit unit 200.

  In the present embodiment, the detection value of the pressed state of each button of the right controller 320 can take any value from 0 to 1. For example, when the user has not pressed the thumb button 302a at all, “0” is detected as the pressing state of the thumb button 302a. On the other hand, when the user presses the thumb button 302a completely (deepest), “1” is detected as the pressing state of the thumb button 302a. Thereby, you may adjust the bending condition of each finger | toe in a virtual hand. For example, by defining the state of “0” as a state where the finger is extended and defining the state of “1” as a state where the finger is bent, the user controls the finger of the virtual hand by an intuitive operation. can do.

  The analog stick is a stick-type button that can be operated by being tilted 360 degrees from a predetermined neutral position. An analog stick 302 i is provided on the top surface 322 of the right controller 320. The analog stick 302i is operated with the thumb of the right hand.

  The right controller 320 includes a frame 326 that extends from both side surfaces of the grip 324 in a direction opposite to the top surface 322 to form a semicircular ring. A plurality of infrared LEDs 304 are embedded on the outer surface of the frame 326.

  The infrared LED 304 emits infrared light during the play of a computer game. Infrared light emitted from the infrared LED 304 is used to detect the position and tilt of the right controller 320.

  The right controller 320 further incorporates a sensor 306 instead of or in addition to the infrared LED 304. The sensor 306 may be, for example, a magnetic sensor, an angular velocity sensor, an acceleration sensor, or a combination thereof. The position and inclination of the right controller 320 can be detected by the sensor 306.

  The transceiver 308 is configured to transmit and receive data between the right controller 320 and the control circuit unit 200. The transceiver 308 transmits data based on the operation input given to the right controller 320 by the user via the operation button 302 to the control circuit unit 200. The transceiver 308 also receives a command for instructing the right controller 320 to emit light from the infrared LED 304 from the control circuit unit 200. Further, the transceiver 308 transmits data corresponding to various values detected by the sensor 306 to the control circuit unit 200.

  The right controller 320 may include a vibrator for transmitting tactile feedback by vibration to the user's hand. In this configuration, the transceiver 308 can receive a command for causing the vibrator to perform tactile feedback from the control circuit unit 200 in addition to the transmission / reception of each data described above.

(Functional configuration of control circuit unit 200)
FIG. 8 is a block diagram showing a functional configuration of the control circuit unit 200. The control circuit unit 200 controls the virtual space 2 provided to the user by using various data received from the HMD sensor 120, the controller sensor 140, the gaze sensor 130, and the controller 300, and to the display 112 of the HMD 110. Control image display. As illustrated in FIG. 8, the control circuit unit 200 includes a detection unit 210, a display control unit 220, a virtual space control unit 230, a storage unit 240, and a communication unit 250. The control circuit unit 200 functions as a detection unit 210, a display control unit 220, a virtual space control unit 230, a storage unit 240, and a communication unit 250 by the cooperation of the hardware illustrated in FIG. The functions of the detection unit 210, the display control unit 220, and the virtual space control unit 230 can be realized mainly by the cooperation of the processor and the memory. The function of the storage unit 240 can be realized mainly by the cooperation of the memory and the storage. The function of the communication unit 250 can be realized mainly by the cooperation of the processor and the communication interface.

  The detection unit 210 receives detection values from various sensors (such as the HMD sensor 120) connected to the control circuit unit 200. Moreover, the predetermined process using the received detected value is performed as needed. The detection unit 210 includes an HMD detection unit 211, a line-of-sight detection unit 212, and a controller detection unit 213. The HMD detection unit 211 receives detection values from the HMD 110 and the HMD sensor 120, respectively. The line-of-sight detection unit 212 receives the detection value from the gaze sensor 130. The controller detection unit 213 receives detection values from the controller sensor 140, the right controller 320, and the left controller 330.

  The display control unit 220 controls image display on the display 112 of the HMD 110. The display control unit 220 includes a virtual camera control unit 221, a visual field region determination unit 222, and a visual field image generation unit 223. The virtual camera control unit 221 arranges the virtual camera 1 in the virtual space 2 and controls the behavior of the virtual camera 1 in the virtual space 2. The view area determination unit 222 determines the view area 23. The view image generation unit 223 generates a view image 26 displayed on the display 112 based on the determined view area 23.

  The virtual space control unit 230 controls the virtual space 2 provided to the user. The virtual space control unit 230 includes a virtual space defining unit 231, a virtual hand control unit 232, a user interface (UI) object control unit 233, and a movement destination specifying unit 234.

  The virtual space defining unit 231 defines the virtual space 2 in the HMD system 100 by generating virtual space data representing the virtual space 2 provided to the user. The virtual hand control unit 232 arranges each virtual hand (virtual right hand and virtual left hand) of the user in accordance with the operation of the right controller 320 and the left controller 330 by the user in the virtual space 2 and each virtual hand in the virtual space 2 Control hand behavior.

  The UI object control unit 233 arranges the UI object in the virtual space 2 and controls the behavior of the UI object in the virtual space 2. The UI object is an object for designating the movement destination of the virtual camera 1 among the virtual objects arranged in the virtual space 2. The UI object includes an attribute related to the position in the virtual space 2. The attribute includes coordinate information in the virtual space. The coordinate information is information indicating the coordinates of the center of the UI object, the coordinates of the outer edge of the UI object, the coordinates of the intersection of a plurality of UI objects, and the like. The UI object control unit 233 arranges one or more UI objects based on the first operation input by the user, and changes one or more attributes by changing the coordinate information of the arranged UI objects.

  The movement destination specifying unit 234 specifies the movement destination of the virtual camera 1 based on the attribute of the UI object. The movement destination specifying unit 234 specifies the coordinates of a predetermined position of the UI object in the virtual space 2 based on the attribute of the UI object at the time when the second operation is input by the user after the first operation is input. To do. Then, the coordinates are specified as the movement destination of the virtual camera 1. The movement destination specifying unit 234 outputs the coordinates of the specified movement destination in the virtual space 2 to the virtual camera control unit 221.

  The storage unit 240 stores various data used by the control circuit unit 200 to provide the virtual space 2 to the user. The storage unit 240 includes a model storage unit 241, a content storage unit 242, and a change rule storage unit 243. The template storage unit 241 stores various template data representing the template of the virtual space 2. The content storage unit 242 stores various types of content that can be reproduced in the virtual space 2. The change rule storage unit 243 stores a change rule for changing the attribute of the UI object. The UI object control unit 233 changes the attribute of the UI object according to the change rule according to the first operation.

  The template data has spatial structure data that defines the spatial structure of the virtual space 2. The spatial structure data is data that defines the spatial structure of a 360-degree celestial sphere centered on the center 21, for example. The template data further includes data defining the XYZ coordinate system of the virtual space 2. The template data further includes coordinate data for specifying the position of each mesh constituting the celestial sphere in the XYZ coordinate system. The template data further includes a flag indicating whether or not a virtual object can be arranged in the virtual space 2.

  The content is content that can be reproduced in the virtual space 2. In the present embodiment, this content is game content. The content includes at least data defining a background image of the game and virtual objects (characters, items, etc.) appearing in the game. In each content, an initial direction facing an image to be shown to the user in the initial state (at the time of activation) of the HMD 110 is defined in advance.

  The communication unit 250 transmits / receives data to / from an external device 400 (for example, a game server) via the network NW.

(Process of providing virtual space 2)
FIG. 9 is a sequence diagram showing a flow of processing in which the HMD system 100 provides the virtual space 2 to the user. The virtual space 2 is basically provided to the user through the cooperation of the HMD 110 and the control circuit unit 200. First, in step S1, the virtual space defining unit 231 defines the virtual space 2 by generating virtual space data representing the virtual space 2 provided to the user. The generation procedure is as follows. First, the virtual space defining unit 231 defines the prototype of the virtual space 2 by acquiring the template data of the virtual space 2 from the template storage unit 241. The virtual space defining unit 231 further acquires content to be played back in the virtual space 2 from the content storage unit 242. In the present embodiment, this content is game content.

  The virtual space defining unit 231 generates virtual space data defining the virtual space 2 by adapting the acquired content to the acquired template data. The virtual space defining unit 231 appropriately associates each partial image constituting the background image included in the content with the management data of each mesh constituting the celestial sphere of the virtual space 2 in the virtual space data. The virtual space defining unit 231 preferably associates each partial image with each mesh so that the initial direction defined in the content matches the Z direction in the XYZ coordinate system of the virtual space 2.

  The virtual space defining unit 231 further adds management data of each virtual object included in the content to the virtual space data as necessary. At that time, coordinates representing the position where the corresponding virtual object is arranged in the virtual space 2 are set in each management data. Thus, each virtual object is arranged at the position of the coordinate in the virtual space 2.

  After that, when the HMD 110 is activated by the user, the HMD sensor 120 detects the position and inclination of the HMD 110 in the initial state in step S2, and outputs the detected value to the control circuit unit 200 in step S3. The HMD detection unit 211 receives this detection value. Thereafter, in step S <b> 4, the virtual camera control unit 221 initializes the virtual camera 1 in the virtual space 2.

  The initialization procedure is as follows. First, the virtual camera control unit 221 places the virtual camera 1 at an initial position in the virtual space 2 (eg, the center 21 in FIG. 4). Next, the orientation of the virtual camera 1 in the virtual space 2 is set. At that time, the virtual camera control unit 221 specifies the uvw visual field coordinate system of the HMD 110 in the initial state based on the detection value from the HMD sensor 120 and sets the uvw visual field coordinate system that matches the uvw visual field coordinate system of the HMD 110 to the virtual camera 1. The orientation of the virtual camera 1 may be set by setting to. When setting the uvw visual field coordinate system for the virtual camera 1, the virtual camera control unit 221 adapts the roll direction (w axis) of the virtual camera 1 to the Z direction (Z axis) of the XYZ coordinate system. Specifically, the virtual camera control unit 221 matches the direction obtained by projecting the roll direction of the virtual camera 1 on the XZ plane with the Z direction of the XYZ coordinate system, and the roll direction of the virtual camera 1 with respect to the XZ plane. Is matched with the inclination of the roll direction of the HMD 110 with respect to the horizontal plane. By such adaptation processing, the roll direction of the virtual camera 1 in the initial state is adapted to the initial direction of the content, so that the horizontal direction that the user first faces after the start of content reproduction matches the initial direction of the content. Can be made.

  When the initialization process of the virtual camera 1 is completed, the visual field region determination unit 222 determines the visual field region 23 in the virtual space 2 based on the uvw visual field coordinate system of the virtual camera 1. Specifically, the roll direction (w axis) of the uvw visual field coordinate system of the virtual camera 1 is specified as the reference visual line 5 of the user, and the visual field region 23 is determined based on the reference visual line 5. In step S <b> 5, the visual field image generation unit 223 processes the virtual space data, and corresponds to a portion projected to the visual field region 23 in the virtual space 2 out of the entire virtual space image 22 developed in the virtual space 2. A visual field image 26 to be generated is generated (rendered). In step S6, the visual field image generation unit 223 outputs the generated visual field image 26 to the HMD 110 as an initial visual field image. In step S <b> 7, the HMD 110 displays the received initial view image on the display 112. Thereby, the user visually recognizes the initial view image.

  Thereafter, in step S8, the HMD sensor 120 detects the current position and inclination of the HMD 110, and outputs these detected values to the control circuit unit 200 in step S9. The HMD detection unit 211 receives each detection value. The virtual camera control unit 221 specifies the current uvw visual field coordinate system in the HMD 110 based on the position and tilt detection values of the HMD 110. Furthermore, in step S <b> 10, the virtual camera control unit 221 specifies the roll direction (w axis) of the uvw visual field coordinate system in the XYZ coordinate system as the visual field direction of the HMD 110.

  In the present embodiment, in step S <b> 11, the virtual camera control unit 221 specifies the identified visual field direction of the HMD 110 as the user's reference visual line 5 in the virtual space 2. In step S <b> 12, the virtual camera control unit 221 controls the virtual camera 1 based on the identified reference line of sight 5. If the position (starting point) and direction of the reference line of sight 5 are the same as the initial state of the virtual camera 1, the virtual camera control unit 221 maintains the position and direction of the virtual camera 1 as they are. On the other hand, if the position (starting point) and / or direction of the reference line of sight 5 has changed from the initial state of the virtual camera 1, the position and / or inclination of the virtual camera 1 in the virtual space 2 is changed to the reference line of sight after the change. Change the position and / or inclination according to 5. Further, the uvw visual field coordinate system is reset for the virtual camera 1 after the control.

  In step S <b> 13, the visual field area determination unit 222 determines the visual field area 23 in the virtual space 2 based on the identified reference visual line 5. Thereafter, in step S <b> 14, the visual field image generation unit 223 processes (superimposes) the virtual space data on the visual field region 23 in the virtual space 2 among the entire virtual space image 22 developed in the virtual space 2 by processing the virtual space data. A view field image 26 that is a portion to be generated is generated (rendered). In step S15, the visual field image generation unit 223 outputs the generated visual field image 26 to the HMD 110 as a visual field image for update. In step S <b> 16, the HMD 110 updates the view image 26 by displaying the received view image 26 on the display 112. Thereby, if a user moves HMD110, the view image 26 will be updated in response to it.

<How to move the virtual camera-Overview>
The control circuit unit 200 provides the user interface with excellent operability for moving the virtual space 2 to the user by executing the moving method. The control circuit unit 200 places a virtual object that designates a destination of the virtual camera 1 in the virtual space 2. The virtual object is a UI object that realizes a user interface for the user to specify a destination.

  The control circuit unit 200 receives the first operation input by the user and changes the attribute related to the position of the UI object in the virtual space 2 based on the first operation. The control circuit unit 200 receives the second operation input by the user after the first operation, and specifies the destination of the virtual camera 1 based on the attribute of the UI object at the time when the second operation is input. . The control circuit unit 200 moves the virtual camera 1 to the identified destination.

(Example of moving method)
The UI object control unit 233 arranges the UI object so as to surround the virtual camera 1. When the detection unit 210 receives the first operation, the UI object control unit 233 changes the position of the outer edge of the UI object in the virtual space 2 based on the temporal information included in the first operation. The size of the UI object may be changed as time elapses from the reception time point of the first operation. In this method, the UI object is an object for designating a movement distance.

  After receiving the first operation, the detection unit 210 receives an input for specifying the moving direction of the virtual camera 1 from the user. The detection unit 210 identifies the inclination of the HMD 110 or the user's line-of-sight direction based on the received input content. The movement destination specifying unit 234 may specify the movement direction based on the specified inclination or line-of-sight direction.

  When the second operation is input by the user, the movement destination specifying unit 234 specifies the attribute of the UI object based on the movement direction and the outer edge of the UI object at the time when the second operation is input. Then, the destination of the virtual camera 1 is specified based on the specified attribute. The movement destination specifying unit 234 may specify the coordinates of the intersection of the movement direction and the outer edge in the virtual space 2 as the movement destination of the virtual camera 1.

  FIG. 10 is a diagram illustrating an example of the moving method, and FIG. 11 is a flowchart for explaining the flow of processing of the HMD system 100 that executes the moving method. The UI object is, for example, a circle object 40. The first operation and the second operation are performed via the controller 300, for example. The input for specifying the moving direction is made by, for example, pointing the HMD 110 in a desired direction.

  In step S <b> 21, the controller detection unit 213 detects whether or not the first operation is input based on the detection value received from the controller 300. If YES in step S21, the UI object control unit 233 places the circle object 40 in the virtual space 2 in step S22. The UI object control unit 233 may arrange the circle object 40 so that the position of the virtual camera 1 is at the center of the circle object 40.

  In step S23, the UI object control unit 233 changes the attribute of the circle object 40 until the second operation is input (NO in step S26). The UI object control unit 233 may gradually increase the radius of the circle object 40 based on the temporal information while maintaining the center of the circle object 40 at the position of the virtual camera 1. The circle object 40 may be presented to the HMD 110 so that the ripples spread around the virtual camera 1. The temporal information is specified by the controller detection unit 213 based on the detection value of the first operation. The time-dependent information includes an elapsed time from when the button is pressed, a time during which the button has been pressed, and the number of times the button has been pressed during a certain period.

  In step S <b> 24, the HMD detection unit 211 receives a detection value transmitted from the HMD sensor 120 as an input for specifying the moving direction of the virtual camera 1. The HMD detection unit 211 identifies the viewing direction of the HMD 110 based on the received detection value.

  In the case of YES in step S24, in step S25, the movement destination specifying unit 234 specifies the movement direction based on the specified visual field direction. The line of sight 41 indicating the moving direction may be presented to the HMD 110 in a manner like a laser beam.

  In step S <b> 26, the controller detection unit 213 detects whether the second operation is input based on the detection value received from the controller 300. In the case of YES in step S26, in step S27, the movement destination specifying unit 234 specifies the attribute of the circle object 40 when the second operation is input. For example, the intersection 50 between the outer edge of the circle object 40 and the line of sight 41 may be specified.

  In step S28, the movement destination specifying unit 234 acquires the coordinates of the specified intersection 50 in the virtual space 2, and specifies the movement destination of the virtual camera 1 based on the coordinates. The movement destination specifying unit 234 may specify the acquired coordinates as the movement destination as they are, or may specify the coordinates calculated based on a predetermined rule with reference to the coordinates as the movement destination.

  In step S <b> 29, the virtual camera control unit 221 moves the virtual camera 1 to the position indicated by the coordinates specified by the movement destination specifying unit 234 as the movement destination.

  Based on the position of the virtual camera 1 after the movement, the processes in steps S8 to S16 shown in FIG. 9 are executed, and the view image displayed on the HMD 110 is updated. The view image may be switched to warp to the position of the movement destination when the movement destination of the virtual camera 1 is determined by inputting the second operation.

  According to this method, the user starts changing the attribute of the UI object by inputting the first operation. Then, the line of sight 41 is directed in the direction of movement while observing the change in the attribute. While the attribute is changed, the user can freely change the direction in which the line of sight 41 is sent. The line of sight 41 is a line-of-sight direction or a visual field direction. The user determines the outer edge of the UI object by inputting the second operation when the outer edge of the UI object reaches the desired movement distance. Thereby, the movement destination of the virtual camera 1 is specified based on the outer edge of the UI object and the line of sight 41. As described above, the user can simultaneously specify the movement distance and the movement direction by one operation by the second operation. As a result, the operability related to the movement of the virtual space in the HMD system 100 is improved, and the feeling of immersion in the virtual space is improved.

(Other examples of moving methods)
In the present method, the first operation is an operation of inputting a motion of a movable object that moves in the virtual space 2 so as to be interlocked with a part of the user's body. The UI object control unit 233 characterizes the movement mode when the UI object moves in the virtual space 2 based on the motion of the movable object. The UI object control unit 233 changes the attribute related to the position of the UI object by moving the UI object in the virtual space 2 based on the movement mode. The movement destination specifying unit 234 specifies the position of the UI object in the virtual space 2 based on the movement mode at the time when the second operation is input. The movement destination specifying unit 234 specifies the movement destination of the virtual camera 1 based on the coordinates of the specified position.

  FIG. 12 is a diagram illustrating another example of the moving method, and FIG. 13 is a flowchart illustrating a processing flow of the HMD system 100 that realizes the moving method. The movable object is, for example, a virtual hand. The first operation for operating the virtual hand is performed, for example, when the user moves or tilts the controller 300 in a desired direction at a desired speed. The UI object is a moving object 42 that moves in the virtual space 2 based on, for example, the action of a virtual hand.

  In step S <b> 41, the controller detection unit 213 detects whether or not the first operation is input based on the detection value received from the controller sensor 140. In the case of YES in step S41, in step S42, the virtual hand control unit 232 operates the virtual hand based on the position and inclination of the controller 300 specified based on the detection value.

  In step S <b> 43, the UI object control unit 233 specifies the movement mode of the moving object 42 based on the movement of the virtual hand, and characterizes the specified movement mode in the moving object 42. The UI object control unit 233 may characterize the moving mode of the moving object 42 based at least on the initial speed and the initial speed vector of the operated virtual hand.

  In step S44, the UI object control unit 233 changes the attribute of the moving object 42 by moving the moving object 42 arranged in the virtual space 2 based on the movement mode. The UI object control unit 233 may continue to move the moving object 42 until the second operation is input (NO in step S45).

  In step S <b> 45, the controller detection unit 213 detects whether the second operation is input based on the detection value received from the controller 300. In the case of YES in step S45, in step S46, the movement destination specifying unit 234 specifies the position of the moving object 42 in the virtual space 2 when the second operation is input.

  In step S47, the movement destination specifying unit 234 acquires the coordinates of the position of the moving object 42 in the virtual space 2, and specifies the movement destination of the virtual camera 1 based on the coordinates. The movement destination specifying unit 234 may specify the acquired coordinates as the movement destination as they are, or may specify the coordinates calculated based on a predetermined rule with reference to the coordinates as the movement destination. Step S48 and subsequent steps are the same as step S29 and subsequent steps.

  According to this method, the user can specify the destination by one operation by the second operation. As a result, the operability related to the movement of the virtual space in the HMD system 100 is improved, and the feeling of immersion in the virtual space is improved.

(Another example of moving method)
In this method, the UI object is an avatar that moves in the virtual space 2 based on the direction information included in the first operation. The UI object control unit 233 changes the attribute by moving the avatar to an arbitrary position in the virtual space 2 based on the direction information. The movement destination specifying unit 234 specifies the position in the virtual space 2 of the position of the avatar when the second operation is input. The movement destination specifying unit 234 specifies the movement destination of the virtual camera 1 based on the coordinates of the specified position.

  FIG. 14 is a diagram showing still another example of the moving method, and FIG. 15 is a flowchart for explaining the processing flow of the HMD system 100 that realizes the moving method. The first operation for moving the avatar is performed by the user inputting direction information. The UI object is an avatar 43 that moves in the virtual space 2 based on the direction information.

  In step S <b> 61, the controller detection unit 213 detects whether or not the first operation is input based on the detection value received from the controller 300. If YES in step S61, the UI object control unit 233 places the avatar 43 in the virtual space 2 in step S62.

  In step S63, the UI object control unit 233 moves the avatar 43 based on the direction information included in the first operation. Furthermore, you may change the direction of the avatar 43 based on direction information.

  In step S <b> 64, the controller detection unit 213 detects whether the second operation is input based on the detection value received from the controller 300. In the case of YES in step S64, in step S65, the movement destination specifying unit 234 specifies the position of the avatar 43 in the virtual space 2 when the second operation is input. In addition, you may specify direction further as needed.

  In step S66, the movement destination specifying unit 234 acquires the coordinates of the position of the avatar 43 in the virtual space 2, and specifies the movement destination of the virtual camera 1 based on the coordinates. The movement destination specifying unit 234 may specify the acquired coordinates as the movement destination as they are, or may specify the coordinates calculated based on a predetermined rule with reference to the coordinates as the movement destination. The movement destination specifying unit 234 may further specify the direction of the avatar 43 as the direction after the movement of the virtual camera 1 as necessary.

  In step S <b> 67, the virtual camera control unit 221 moves the virtual camera 1 to the position indicated by the coordinates specified by the destination specification unit 234 as the destination. The virtual camera control unit 221 may set the orientation of the virtual camera 1 after movement based on the orientation of the specified avatar 43. Thereafter, the field-of-view image displayed on the HMD 110 is updated based on the position and orientation of the virtual camera 1 after the movement, as in the above-described respective movement methods.

  The virtual camera control unit 221 may arrange a second virtual camera (not shown) facing the front direction of the avatar 43 at the position of the avatar 43. The display control unit 220 may display a view image based on the second virtual camera on the HMD 110 in a partial area of the view image.

  According to this method, the user can specify the destination by one operation by the second operation. As a result, the operability related to the movement of the virtual space in the HMD system 100 is improved, and the feeling of immersion in the virtual space is improved.

(Modification)
In each of the moving methods described above, the control circuit unit 200 may specify the line-of-sight direction N0 as the reference line-of-sight 5 instead of the field-of-view direction. In this case, when the user changes the line of sight, the orientation of the virtual camera 1 changes in conjunction with the change of the line of sight, so the position of the visual field region 23 also changes in conjunction with the change of the line of sight. As a result, the contents of the view field image 26 change according to the line of sight change.

  In the example of the moving method described above, the line-of-sight detection unit 212 may accept an input for specifying the movement direction by specifying the user's line-of-sight direction N0 based on the detection value received from the gaze sensor 130. .

  The moving method is assumed to be applied to a game or the like that employs a user interface for selecting an object in the virtual space 2 watched by the user based on the reference line of sight 5. In the game, the line of sight 41 may be presented to the HMD 110 at a timing before the first operation is input.

  In the moving method, the UI object is not limited to the circle object 40, and may be defined in an arbitrary shape. The UI object may be defined by a plane like the circle object 40 or may be defined by a solid such as a sphere.

  In the movement method, the movement destination specifying unit 234 may further specify the direction after movement of the virtual camera 1 in step S28.

  In another example of the moving method described above, the attribute of the moving object 42 may include information regarding the orientation. In step S47, the movement destination specifying unit 234 may specify the direction of the moving object 42 at the time when the second operation is input as the direction after the movement of the virtual camera 1.

  In another example of the moving method described above, the moving object 42 is a virtual object of an object having an arbitrary shape, volume, mass, and texture, such as a ball or a dice. The change rule storage unit 243 stores various types of information related to the physical law regarding the environment of the moving object 42 and the virtual space 2. The various types of information include the shape, volume, mass, and surface texture of the moving object 42, gravity acceleration, friction coefficient, air resistance, and the like in the environment of the virtual space 2. The UI object control unit 233 may specify a movement mode to be characterized by the moving object 42 in consideration of various types of information. As an example, a physical law corresponds to a physical law in real space. The UI object control unit 233 may specify the moving mode by calculating the moving path and the moving speed of the moving object 42 in consideration of the above-described various information in addition to the initial speed and the initial speed vector of the movable object. . Thereby, the movement mode of the moving object 42 can be brought close to the movement mode in the real space.

  In each moving method described above, the UI object control unit 233 temporarily stores the moving path of the virtual camera 1 specified by the attribute in the memory while changing the attribute of the UI object according to the first operation. You may leave it. For example, the UI object control unit 233 temporarily stores the movement path 44 (FIG. 14) of the avatar 43 during the period until the second operation is input. The view image generation unit 223 may also generate a view image in the transitional transition period based on the trajectory 44, and sequentially output these view images to the HMD 110. As a result, it is possible to make the user visually recognize that the user has reached the position after the movement along the movement route. Note that it is preferable to perform a process of reducing the amount of information on each view image in the movement transition period. Specifically, it is preferable to reduce the resolution of the entire view image or a part thereof. Thereby, it becomes difficult for the user to cause video sickness.

  In addition, when a virtual experience is provided by applying an operation by contact with a virtual object to MR or the like, a part of the body other than the actual head of the user is physically and optically replaced with the operation target object. The contact between the body part and the virtual object may be determined based on the positional relationship between the body part and the virtual object. When providing a virtual experience using a transmissive HMD, the user's reference line of sight may be specified by detecting the movement of the HMD or the user's line of sight, similar to the non-transmissive HMD. . The determination region setting method based on the identified reference line of sight is as described in the above embodiment.

[Example of software implementation]
A control block (detection unit 210, display control unit 220, virtual space control unit 230, storage unit 240, and communication unit 250) of the control circuit unit 200 is a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. ) Or by software using a CPU (Central Processing Unit).

  In the latter case, the control block includes a CPU that executes instructions of a program that is software that realizes each function, a ROM (Read Only Memory) or a memory in which the program and various data are recorded so as to be readable by a computer (or CPU). A device (these are referred to as “recording media”), a RAM (Random Access Memory) for expanding the program, and the like are provided. And the objective of this invention is achieved when a computer (or CPU) reads the said program from the said recording medium and runs it. As the recording medium, a “non-temporary tangible medium” such as a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. The program may be supplied to the computer via an arbitrary transmission medium (such as a communication network or a broadcast wave) that can transmit the program. The present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining technical means disclosed in different embodiments are also included in the technical scope of the present invention. A new technical feature can also be formed by combining the technical means disclosed in each embodiment.

[Additional Notes]
The contents according to one aspect of the present invention are listed as follows.

  (Item 1) A method for providing a virtual space to a user wearing a head-mounted display (hereinafter referred to as HMD) on the head, wherein the user's gaze direction or the user's line-of-sight direction Specifying a reference line of sight in a virtual space; specifying a virtual camera that is arranged in the virtual space and that sets a field of view to be visually recognized by the user based on the reference line of sight; and specifies a destination of the virtual camera Placing a virtual object in the virtual space; receiving a first operation input by a user; and changing an attribute relating to a position of the virtual object in the virtual space based on the first operation; When a second operation input by the user after the first operation is received and the second operation is input Based on the attribute of the a and identifying a destination of the virtual camera, and moving the virtual camera to the destination method. Thereby, the immersion feeling in virtual space can be improved.

  (Item 2) The virtual object is arranged so as to surround the virtual camera, and the position of the outer edge of the virtual object in the virtual space is changed based on time-lapse information included in the first operation, Item 1: The input for specifying the moving direction of the virtual camera is received after input of the first operation, and the attribute is specified based on the moving direction and the outer edge at the time of input of the second operation. Method.

  (Item 3) The method according to item 2, wherein the moving direction is specified based on the inclination of the HMD or the line-of-sight direction of the user.

  (Item 4) The method according to item 2 or 3, wherein the attribute is specified based on a position of the intersection between the moving direction and the outer edge in the virtual space.

  (Item 5) The method according to any one of items 2 to 4, wherein the virtual object is a circle, and the radius of the circle is changed based on the time-dependent information.

  (Item 6) The first operation is to input an action of a movable object that moves in the virtual space so as to be interlocked with a part of the user's body, and the virtual object moves in the virtual space. The method according to item 1, characterized in that the movement mode is characterized as the virtual object based on the movement of the movable object, and the attribute is specified based on the movement mode at the time when the second operation is input. .

  (Item 7) The method according to item 6, wherein the movement mode is characterized based at least on an initial speed and an initial speed vector of the movable object.

  (Item 8) The virtual object is an avatar that moves in the virtual space based on direction information included in the first operation, and the avatar is moved to an arbitrary position in the virtual space based on the direction information. The method of item 1, wherein the attribute is changed by moving it.

  (Item 9) A step of arranging a second virtual camera facing the front direction of the avatar at the position of the avatar in the virtual space, and a partial area in the visual field image generated based on the visual field area, 9. The method of item 8, further comprising: arranging a view image based on the second virtual camera and displaying the image on the HMD.

  (Item 10) The method according to any one of items 1 to 9, further comprising: specifying a direction of the virtual object based on the first operation and setting a direction of the virtual camera based on the direction of the virtual object. Method.

  (Item 11) A program that causes a computer to execute each step of any one of items 1 to 12.

  (Item 12) A computer-readable recording medium on which the program of item 11 is recorded.

1 virtual camera, 2 virtual space, 5 reference line of sight, 21 center, 22 virtual space image, 23 view area, 24 first area, 25 second area, 26 view image, 40 circle object, 41 line of sight object, 42 moving object, 43 avatar, 44 locus, 50 intersection, 100 HMD system, 110 HMD, 112 display, 114, 306 sensor, 120 HMD sensor, 130 gaze sensor, 140 controller sensor, 200 control circuit unit, 210 detection unit, 211 HMD detection unit, 212 gaze detection unit, 213 controller detection unit, 220 display control unit, 221 virtual camera control unit, 222 visual field region determination unit, 223 visual field image generation unit, 230 virtual space control unit, 231 virtual space definition unit, 232 virtual hand control unit 233 UI Object control unit, 234 destination identification unit, 240 storage unit, 241 template storage unit, 242 content storage unit, 243 change rule storage unit, 250 communication unit, 300 controller, 302 operation buttons, 302a, 302b thumb button, 302e index finger button , 302f Middle finger button, 302i analog stick, 304 infrared LED, 308 transceiver, 320 right controller, 322 top surface, 324 grip, 326 frame, 330 left controller, 400 External equipment

Claims (11)

A method for providing a virtual space to a user wearing a head-mounted display (hereinafter, HMD) on the head,
Identifying a reference line of sight in the virtual space of the user based on the inclination of the HMD or the line of sight of the user;
Identifying a virtual camera that is arranged in the virtual space and that sets a field of view to be visually recognized by the user based on the reference line of sight;
Placing in the virtual space a virtual object that specifies the destination of the virtual camera;
Receiving a first operation input by a user , starting a change to an attribute indicating a movement destination of the virtual camera of the virtual object based on the first operation, and continuing the change ;
The second operation input by the user while the attribute is changed is received, the attribute is specified at the time when the second operation is input, and the destination of the virtual camera is moved based on the specified attribute Identifying steps,
Moving the virtual camera to the destination. The virtual object is arranged so as to surround the virtual camera,
The position of the outer edge of the virtual object in the virtual space is changed based on the temporal information included in the first operation,
After inputting the first operation, accepting an input for specifying a moving direction of the virtual camera;
The method according to claim 1, wherein the attribute is specified based on the moving direction and the outer edge at the time of input of the second operation. A method for providing a virtual space to a user wearing a head-mounted display (hereinafter, HMD) on the head,
Identifying a reference line of sight in the virtual space of the user based on the inclination of the HMD or the line of sight of the user;
Identifying a virtual camera that is arranged in the virtual space and that sets a field of view to be visually recognized by the user based on the reference line of sight;
Placing in the virtual space a virtual object that specifies the destination of the virtual camera;
Receiving a first operation input by a user, and changing an attribute relating to a position of the virtual object in the virtual space based on the first operation;
Receiving a second operation input by a user after the first operation, and identifying a destination of the virtual camera based on the attribute at the time when the second operation is input;
Moving the virtual camera to the destination,
The virtual object is arranged so as to surround the virtual camera,
The position of the outer edge of the virtual object in the virtual space is changed based on the temporal information included in the first operation,
After inputting the first operation, accepting an input for specifying a moving direction of the virtual camera;
The method of specifying the attribute based on the moving direction and the outer edge at the time of input of the second operation.   The method according to claim 2 or 3, wherein the moving direction is specified based on the inclination of the HMD or the line-of-sight direction of the user.   The method according to claim 2, wherein the attribute is specified based on a position in the virtual space of an intersection between the moving direction and the outer edge.   The method according to claim 2, wherein the virtual object is a circle, and the radius of the circle is changed based on the time-dependent information. The first operation is to input a motion of a movable object that moves in the virtual space so as to be interlocked with a part of the user's body,
Characterizing the movement mode when the virtual object moves in the virtual space, based on the movement of the movable object, to the virtual object,
The method according to claim 1, wherein the attribute is specified based on the movement mode at the time when the second operation is input.   The method of claim 7, wherein the movement aspect is characterized based at least on an initial speed and an initial speed vector of the movable object.   9. The method according to claim 1, further comprising: specifying an orientation of the virtual object based on the first operation, and setting an orientation of the virtual camera based on the orientation of the virtual object. Method.   The program which makes a computer perform each step of the method of any one of Claim 1 to 9.   The computer-readable recording medium which recorded the program of Claim 10.

Images