JP6416338B1 - Information processing method, information processing program, information processing system, and information processing apparatus - Google Patents

Abstract

An information processing method capable of providing a user with necessary information in a virtual space without a sense of incongruity is provided.
An information processing method in a system including a head-mounted device (HMD) includes: (a) a first object associated with a user of the HMD and a moving part, and the moving part determines a predetermined value for the first object. A step of generating virtual space data defining a virtual space including the second object that executes the operation of (b), and (b) showing a visual field image V2 displayed on the HMD based on the movement of the HMD and the virtual space data A step of generating visual field image data, and (c) a process of narrowing the visual field of the visual field image V2 and changing the display mode of the visual field image V2 when a predetermined operation is performed on the first object by the operation part. Performing display control of the first state.
[Selection] Figure 15

Description

  The present disclosure relates to an information processing method, an information processing program, an information processing system, and an information processing apparatus.

  Patent Document 1 discloses a method of changing the visual field in the virtual space in conjunction with the posture of the head mounted display.

JP 2017-004357 A

  In order to convey necessary information to the user wearing the head mounted device without a sense of incongruity, it is important how to convey the information. For example, a method is known in which a gauge is always displayed at the end of the field of view (view), but such a display method is far from reality and may be aroused.

  An object of the present disclosure is to provide an information processing method, an information processing program, an information processing system, and an information processing apparatus that can provide a user with necessary information in a virtual space without a sense of incongruity.

According to one aspect of the present disclosure, an information processing method in a system including a head mounted device,
The information processing method is as follows:
(A) a virtual space including a first object associated with a user of the head-mounted device and a second object that includes a motion part and performs a predetermined motion on the first object by the motion part. Generating virtual space data to be defined;
(B) generating visual field image data indicating a visual field image displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) When the predetermined motion is performed on the first object by the motion portion, a first state is applied in which processing for narrowing the visual field of the visual field image and changing the display mode of the visual field image is performed. Performing display control;
including.

  According to the present disclosure, necessary information in the virtual space can be provided to the user without a sense of incongruity.

It is the schematic which shows the virtual space delivery system which concerns on embodiment (henceforth this embodiment only) of this invention. It is the schematic which shows a user terminal. It is a figure which shows the head of the user with which HMD was mounted | worn. It is a figure which shows the hardware constitutions of a control apparatus. It is a figure which shows an example of a specific structure of an external controller. It is a flowchart which shows the process which displays a visual field image on HMD. It is xyz space figure which shows an example of virtual space. The state (a) is a yx plan view of the virtual space shown in FIG. The state (b) is a zx plan view of the virtual space shown in FIG. It is a figure which shows an example of the visual field image displayed on HMD. It is a figure which shows the hardware constitutions of the server shown in FIG. The state (a) is a diagram showing a real space where the user wearing the HMD exists, and the state (b) is a diagram showing a virtual space including a virtual camera and various virtual objects. It is a flowchart for demonstrating the information processing method which concerns on this embodiment. It is a figure which shows the visual field image in the state (b) of FIG. It is a figure which shows the virtual space of the state in which the virtual object contacted the collision object. It is a figure which shows the visual field image in the state of FIG. It is a figure which shows the visual field image after the state of FIG. 15 passes for a fixed time. It is a figure which shows a visual field image when a user's predetermined parameter becomes below a predetermined threshold value.

[Description of Embodiments Presented by the Present Disclosure]
An overview of an embodiment indicated by the present disclosure will be described.
(1) An information processing method in a system including a head mounted device,
The information processing method is as follows:
(A) a virtual space including a first object associated with a user of the head-mounted device and a second object that includes a motion part and performs a predetermined motion on the first object by the motion part. Generating virtual space data to be defined;
(B) generating visual field image data indicating a visual field image displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) When the predetermined motion is performed on the first object by the motion portion, display control in a first state that narrows the field of view of the visual field image and changes the display mode of the visual field image Steps to do,
including.

  According to this method, necessary information in the virtual space can be provided to the user without a sense of incongruity. In this method, for example, when a user is attacked by an enemy object in a virtual space, the user's view is narrowed and the display mode of the narrowed portion is changed to make it intuitive that the user has been attacked. Recognizable and uncomfortable.

  (2) The step (c) may include a step of hiding the operation portion in the visual field image.

  If the motion part is displayed in the field-of-view image when the motion part performs a predetermined motion on the first object, the inside of the motion part that is not supposed to be rendered appears in the field of view. There is a possibility of becoming a natural visual field image. Therefore, it is preferable to hide the operation part in step (c).

  (3) In the step (c), display control of the first state may be performed by displaying a first display relating to a predetermined parameter associated with the predetermined operation on the visual field image.

  (4) In the step (c), the visual effect at the corner of the visual field image is reduced to narrow the field of view, and the first display is displayed closer to the center of the visual field image than the corner. May be.

  By these methods, a new user interface for giving necessary information to the user can be provided.

  (5) (d) After performing the display control of the first state in the step (c), the method may further include a step of performing display control of the second state in which processing for further narrowing the field of view is performed.

  This method allows the user to recognize that the parameters (eg, hit points) associated with the user have decreased.

  (6) The display mode of the visual field image in the first state may be different from the display mode of the visual field image in the second state.

  By this method, the user can intuitively recognize, for example, a decrease in parameters without impairing the quality of the virtual space experience.

  (7) (e) When the predetermined parameter falls below a threshold value, the visual field of the visual field image is displayed in a display mode different from the display mode of the visual field image in the first state and the second state. You may further include the step which performs the display control of the 3rd state which performs the process to narrow.

  By this method, the user can recognize the decrease in the parameters step by step.

  (8) In the step (d), the visibility of the operating part may be higher than the visibility of parts other than the operating part.

  Even when the field of view is narrowed, the game performance can be improved by ensuring the visibility of the motion part.

  (9) In the step (d), the operation part may be displayed in color, and a part other than the operation part may be displayed in gray scale.

  According to this method, it is possible to ensure the visibility of the operating portion even when the parameter is reduced.

  (10) The information processing program is an information processing program for causing a computer to execute the information processing method described in any of the items (1) to (9).

  It is possible to provide an information processing program that can provide a user with necessary information in a virtual space without a sense of incongruity.

(11) An information processing system using a head mounted device,
The information processing system is an information processing system configured to execute the information processing method according to any one of items (1) to (9).

  It is possible to provide an information processing system that can provide a user with necessary information in a virtual space without a sense of incongruity.

(12) a processor;
An information processing apparatus comprising a memory for storing computer-readable instructions,
When the computer-readable instruction is executed by the processor, the information processing apparatus is an information processing apparatus that executes the information processing method according to any one of items (1) to (9).

  According to the above, it is possible to provide an information processing apparatus that can provide necessary information in a virtual space to a user without feeling uncomfortable. It should be noted that the information processing apparatus is either a user terminal or a server.

[Details of Embodiments Presented by the Present Disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. For the convenience of explanation, the description of the elements having the same reference numerals as those already described in the description of the present embodiment will not be repeated.

  First, an outline of the configuration of the virtual space distribution system 100 (information processing system) will be described with reference to FIG. FIG. 1 is a schematic diagram of a virtual space distribution system 100 (hereinafter simply referred to as distribution system 100). As shown in FIG. 1, the distribution system 100 includes a user terminal 1A (first user terminal) operated by a user A (first user) wearing a head mounted device (HMD) 110 (first HMD), and an HMD 110. A user terminal 1B (second user terminal) operated by a user B (second user) wearing the (second HMD) and a server 2 are provided. User terminals 1A and 1B are communicably connected to a server 2 via a communication network 3 such as the Internet. Hereinafter, for convenience of explanation, the user terminals 1A and 1B may be simply referred to as the user terminal 1 in some cases. Each user A and B may be simply referred to as a user U. In the present embodiment, the user terminals 1A and 1B are assumed to have the same configuration. The virtual space includes a VR (Virtual Reality) space, an AR (Arranged Reality) space, and an MR (Mixed Reality) space.

  Next, the configuration of the user terminal 1 will be described with reference to FIG. FIG. 2 is a schematic diagram showing the user terminal 1. As shown in FIG. 2, the user terminal 1 includes a head mounted device (HMD) 110 mounted on the head of the user U, a headphone 116, a position sensor 130, an external controller 320, and a control device 120. .

  The HMD 110 includes a display unit 112, an HMD sensor 114, and a gaze sensor 140. The display unit 112 includes a non-transmissive display device configured to completely cover the field of view (field of view) of the user U wearing the HMD 110. Thereby, the user U can immerse in the virtual space by viewing only the visual field image displayed on the display unit 112. The display unit 112 may include a left-eye display unit configured to provide an image to the user U's left eye and a right-eye display unit configured to provide an image to the user U's right eye. . The HMD 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting the transmittance.

  The HMD sensor 114 is mounted in the vicinity of the display unit 112 of the HMD 110. The HMD sensor 114 includes at least one of a geomagnetic sensor, an acceleration sensor, and a tilt sensor (such as an angular velocity sensor and a gyro sensor), and detects various movements (tilt and the like) of the HMD 110 mounted on the user U's head. be able to.

  The gaze sensor 140 has an eye tracking function that detects the line of sight of the user U. The gaze sensor 140 may include, for example, a right eye gaze sensor and a left eye gaze sensor. The right eye gaze sensor irradiates, for example, infrared light to the right eye of the user U, and detects reflected light reflected from the right eye (particularly the cornea and iris), thereby acquiring information related to the rotation angle of the right eye's eyeball. May be. On the other hand, the left eye gaze sensor irradiates the left eye of the user U with, for example, infrared light, and detects reflected light reflected from the left eye (particularly the cornea and iris), thereby providing information on the rotation angle of the left eye's eyeball. May be obtained.

  Headphones 116 (audio output units) are attached to the left and right ears of the user U, respectively. The headphones 116 are configured to receive audio data (electrical signals) from the control device 120 and output audio based on the received audio data.

  The position sensor 130 is configured by, for example, a position tracking camera, and is configured to detect the positions of the HMD 110 and the external controller 320. The position sensor 130 is communicably connected to the control device 120 by wireless or wired communication, and is configured to detect information on the position, inclination, or light emission intensity of a plurality of detection points (not shown) provided in the HMD 110. . The position sensor 130 is configured to detect information regarding the position, inclination, and / or emission intensity of a plurality of detection points (not shown) provided in the external controller 320. The detection point is, for example, a light emitting unit that emits infrared light or visible light. The position sensor 130 may include an infrared sensor and a plurality of optical cameras.

  The external controller 320 is used to control the movement of the hand object displayed in the virtual space by detecting the movement of the hand of the user U. The external controller 320 is an external controller 320R for right hand operated by the right hand of the user U (hereinafter simply referred to as controller 320R) and an external controller 320L for left hand operated by the left hand of the user U (hereinafter simply referred to as controller 320L). And). The controller 320R is a device that indicates the position of the right hand of the user U and the movement of the finger of the right hand. The controller 320L is a device that indicates the position of the left hand of the user U and the movement of the finger of the left hand. The left hand object and the right hand object existing in the virtual space move according to the movement of the controllers 320R and 320L.

  The control device 120 is a computer configured to control the HMD 110. The control device 120 specifies the position information of the HMD 110 based on the information acquired from the position sensor 130, and based on the specified position information, the position of the virtual camera in the virtual space (the user U in the virtual space). And the position of the user U wearing the HMD 110 in the real space can be accurately associated with each other. The control device 120 identifies an operation of the external controller 320 based on information acquired from the position sensor 130 and / or a sensor incorporated in the external controller 320, and based on the identified operation of the external controller 320, The movement of the hand object displayed in the virtual space can be accurately associated with the movement of the external controller 320 in the real space. In particular, the control device 120 identifies the operation of the controller 320L based on the information acquired from the position sensor 130 and / or the sensor built in the controller 320L, and determines the virtual operation based on the identified operation of the controller 320L. The motion of the left hand object displayed in the space can be accurately associated with the motion of the controller 320L in the real space (the motion of the user U's left hand). Similarly, the control device 120 identifies the operation of the controller 320R based on the information acquired from the position sensor and / or the sensor incorporated in the controller 320R, and based on the identified operation of the controller 320R, the virtual space It is possible to accurately associate the motion of the right-hand object displayed inside with the motion of the controller 320R in real space (the motion of the right hand of the user U).

  The control device 120 can identify the right eye line of sight and the left eye line of sight of the user U, respectively, and can identify the gaze point that is the intersection of the right eye line of sight and the left eye line of sight. The control device 120 can identify the line of sight of both eyes of the user U (the line of sight of the user U) based on the identified gazing point. The line of sight of the user U is the line of sight of both eyes of the user U, and coincides with the direction of a straight line passing through the middle point of the line segment connecting the right eye and the left eye of the user U and the gaze point. Based on the detection data (eye tracking data) transmitted from the gaze sensor 140 (the left eye gaze sensor and the right eye gaze sensor), the control device 120 and the user U's left eye black eye center position and the user U left eye black eye The center position of each can be specified.

  Next, a method for acquiring information related to the position and inclination of the HMD 110 will be described below with reference to FIG. FIG. 3 is a diagram illustrating the head of the user U wearing the HMD 110. Information regarding the position and inclination of the HMD 110 that is linked to the movement of the head of the user U wearing the HMD 110 can be detected by the position sensor 130 and / or the HMD sensor 114 mounted on the HMD 110. As shown in FIG. 2, three-dimensional coordinates (uvw coordinates) are defined centering on the head of the user U wearing the HMD 110. The vertical direction in which the user U stands up is defined as the v-axis, the direction orthogonal to the v-axis and passing through the center of the HMD 110 is defined as the w-axis, and the direction orthogonal to the v-axis and the w-axis is defined as the u-axis. The position sensor 130 and / or the HMD sensor 114 is an angle around each uvw axis (that is, a yaw angle indicating rotation about the v axis, a pitch angle indicating rotation about the u axis, and a center about the w axis). The inclination determined by the roll angle indicating rotation) is detected. The control device 120 determines angle information for controlling the visual axis of the virtual camera based on the detected angle change around each uvw axis.

  Next, the hardware configuration of the control device 120 will be described with reference to FIG. FIG. 4 is a diagram illustrating a hardware configuration of the control device 120. As illustrated in FIG. 4, the control device 120 includes a control unit 121, a storage unit 123, an I / O (input / output) interface 124, a communication interface 125, and a bus 126. The control unit 121, the storage unit 123, the I / O interface 124, and the communication interface 125 are connected to each other via a bus 126 so as to communicate with each other.

  The control device 120 may be configured as a personal computer, a smartphone, a fablet, a tablet, or a wearable device separately from the HMD 110, or may be built in the HMD 110. In addition, some functions of the control device 120 may be mounted on the HMD 110, and the remaining functions of the control device 120 may be mounted on another device separate from the HMD 110.

  The control unit 121 includes a memory and a processor. The memory includes, for example, a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) having a plurality of work areas in which various programs executed by the processor are stored, and the like. The processor is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and / or a GPU (Graphics Processing Unit), and a program specified from various programs incorporated in the ROM is expanded on the RAM. It is comprised so that various processes may be performed in cooperation with.

  In particular, the control unit 121 may control various operations of the control device 120 by a processor developing a control program on a RAM and executing the control program in cooperation with the RAM. The control unit 121 displays a field image on the display unit 112 of the HMD 110 based on the field image data. Thereby, the user U can be immersed in the virtual space.

  The storage unit (storage) 123 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a USB flash memory, and is configured to store programs and various data. The storage unit 123 may store a control program for causing a computer to execute at least a part of the information processing method according to the present embodiment, or a control program for realizing sharing of a virtual space by a plurality of users. The storage unit 123 may store data related to the user U authentication program, various images, and objects (for example, hand objects). In the storage unit 123, a database including a table for managing various data may be constructed.

  The I / O interface 124 is configured to connect the HMD 110, the position sensor 130, the external controller 320, and the headphones 116 to the control device 120 in a communicable manner. For example, a USB (Universal Serial Bus) is used. A terminal, a DVI (Digital Visual Interface) terminal, a HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, and the like. The control device 120 may be wirelessly connected to each of the HMD 110, the position sensor 130, the external controller 320, and the headphones 116.

  The communication interface 125 is configured to connect the control device 120 to a communication network 3 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication interface 125 includes various wired connection terminals for communicating with an external device such as the server 2 via the communication network 3 and various processing circuits for wireless connection, and communicates via the communication network 3. It is configured to conform to the communication standard.

  Next, an example of a specific configuration of the external controller 320 will be described with reference to FIG. The external controller 320 detects the movement of a part of the body of the user U (a part other than the head, which is the user U's hand in the present embodiment), thereby moving the hand object displayed in the virtual space. Used to control. The external controller 320 is an external controller 320R for right hand operated by the right hand of the user U (hereinafter simply referred to as controller 320R) and an external controller 320L for left hand operated by the left hand of the user U (hereinafter simply referred to as controller 320L). And). The controller 320R is a device that indicates the position of the right hand of the user U and the movement of the finger of the right hand. Further, the right hand object 400R (see FIG. 11) that exists in the virtual space moves according to the movement of the controller 320R. The controller 320L is a device that indicates the position of the left hand of the user U and the movement of the finger of the left hand. Further, the left-hand object 400L (see FIG. 11) that exists in the virtual space moves according to the movement of the controller 320L. Since the controller 320R and the controller 320L have substantially the same configuration, only the specific configuration of the controller 320R will be described below with reference to FIG. In the following description, for convenience, the controllers 320L and 320R may be simply referred to as the external controller 320.

  As shown in FIG. 5, the controller 320R includes an operation button 302, a plurality of detection points 304, a sensor (not shown), and a transceiver (not shown). Only one of the detection point 304 and the sensor may be provided. The operation button 302 includes a plurality of button groups configured to accept an operation input from the user U. The operation button 302 includes a push button, a trigger button, and an analog stick. The push-type button is a button operated by an operation of pressing with a thumb. For example, two push buttons 302 a and 302 b are provided on the top surface 322. The trigger type button is a button operated by an operation of pulling a trigger with an index finger or a middle finger. For example, a trigger button 302 e is provided on the front surface portion of the grip 324, and a trigger button 302 f is provided on the side surface portion of the grip 324. The trigger type buttons 302e and 302f are assumed to be operated by the index finger and the middle finger, respectively. The analog stick is a stick-type button that can be operated by being tilted 360 degrees from a predetermined neutral position in an arbitrary direction. For example, it is assumed that an analog stick 320i is provided on the top surface 322 and is operated using a thumb.

  The controller 320R includes a frame 326 that extends from both side surfaces of the grip 324 in a direction opposite to the top surface 322 to form a semicircular ring. A plurality of detection points 304 are embedded on the outer surface of the frame 326. The plurality of detection points 304 are, for example, a plurality of infrared LEDs arranged in a line along the circumferential direction of the frame 326. After the position sensor 130 detects information on the position, inclination, or light emission intensity of the plurality of detection points 304, the control device 120 detects the position or orientation (inclination / posture) of the controller 320 R based on the information detected by the position sensor 130. Get information about (orientation).

  The sensor of the controller 320R may be, for example, a magnetic sensor, an angular velocity sensor, an acceleration sensor, or a combination thereof. When the user U moves the controller 320R, the sensor outputs a signal (for example, a signal indicating information related to magnetism, angular velocity, or acceleration) according to the direction or movement of the controller 320R. The control device 120 acquires information related to the position and orientation of the controller 320R based on the signal output from the sensor.

  The transceiver of the controller 320R is configured to transmit and receive data between the controller 320R and the control device 120. For example, the transceiver may transmit an operation signal corresponding to the operation input of the user U to the control device 120. The transceiver may receive an instruction signal for instructing the controller 320 </ b> R to emit light at the detection point 304 from the control device 120. Further, the transceiver may send a signal to the controller 120 indicating the value detected by the sensor.

  Next, processing for displaying a field-of-view image on the HMD 110 will be described with reference to FIGS. 6 to 9. FIG. 6 is a flowchart showing a process for displaying the visual field image on the HMD 110. FIG. 7 is an xyz space diagram showing an example of the virtual space 200. The state (a) in FIG. 8 is a yx plan view of the virtual space 200 shown in FIG. The state (b) of FIG. 8 is a zx plan view of the virtual space 200 shown in FIG. FIG. 9 is a diagram illustrating an example of the visual field image V displayed on the HMD 110.

  As shown in FIG. 6, in step S1, the control unit 121 (see FIG. 4) generates virtual space data indicating the virtual space 200 including the virtual camera 300 and various objects. As shown in FIG. 7, the virtual space 200 is defined as an omnidirectional sphere centered on the center position 210 (in FIG. 7, only the upper half celestial sphere is shown). In the virtual space 200, an xyz coordinate system having the center position 210 as an origin is set. The virtual camera 300 defines a visual axis L for specifying the visual field image V (see FIG. 9) displayed on the HMD 110. The uvw coordinate system that defines the visual field of the virtual camera 300 is determined so as to be linked to the uvw coordinate system that is defined around the head of the user U in the real space. The control unit 121 may move the virtual camera 300 in the virtual space 200 in conjunction with the movement of the user U wearing the HMD 110 in the real space.

  Next, in step S2, the control unit 121 identifies the field of view CV (see FIG. 8) of the virtual camera 300. Specifically, the control unit 121 acquires information on the position and inclination of the HMD 110 based on data indicating the state of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114. Next, the control unit 121 identifies the position and orientation of the virtual camera 300 in the virtual space 200 based on information regarding the position and tilt of the HMD 110. Next, the control unit 121 determines the visual axis L of the virtual camera 300 from the position and orientation of the virtual camera 300 and specifies the visual field CV of the virtual camera 300 from the determined visual axis L. The visual field CV of the virtual camera 300 corresponds to a partial area of the virtual space 200 that can be viewed by the user U wearing the HMD 110 (in other words, corresponds to a partial area of the virtual space 200 displayed on the HMD 110). ). The visual field CV includes a first region CVa set as an angular range of the polar angle α around the visual axis L in the xy plane shown in the state (a) of FIG. 8, and xz shown in the state (b) of FIG. And a second region CVb set as an angle range of the azimuth angle β around the visual axis L in the plane. The control unit 121 identifies the line of sight of the user U based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140, and based on the information regarding the identified line of sight of the user U and the position and inclination of the HMD 110, The orientation of the camera 300 (the visual axis L of the virtual camera) may be determined.

  As described above, the control unit 121 can specify the visual field CV of the virtual camera 300 based on the data from the position sensor 130 and / or the HMD sensor 114. When the user U wearing the HMD 110 moves, the control unit 121 can update the visual field CV of the virtual camera 300 based on the data indicating the movement of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114. . That is, the control unit 121 can update the visual field CV according to the movement of the HMD 110. Similarly, when the line of sight of the user U changes, the control unit 121 may update the visual field CV of the virtual camera 300 based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140. That is, the control unit 121 may change the visual field CV according to the change in the line of sight of the user U.

  Next, in step S <b> 3, the control unit 121 generates visual field image data indicating the visual field image V displayed on the display unit 112 of the HMD 110. Specifically, the control unit 121 generates visual field image data based on virtual space data defining the virtual space 200 and the visual field CV of the virtual camera 300.

  Next, in step S4, the control unit 121 displays the field image V on the display unit 112 of the HMD 110 based on the field image data (see FIG. 8). As described above, the visual field CV of the virtual camera 300 is changed according to the movement of the user U wearing the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110 is changed. I can immerse myself in 200.

  The virtual camera 300 may include a left-eye virtual camera and a right-eye virtual camera. In this case, the control unit 121 generates left-eye view image data indicating the left-eye view image based on the virtual space data and the view of the left-eye virtual camera. Further, the control unit 121 generates right-eye view image data indicating a right-eye view image based on the virtual space data and the view of the right-eye virtual camera. Thereafter, the control unit 121 displays the left-eye field image on the left-eye display unit based on the left-eye field image data, and also displays the right-eye field image on the right-eye display unit based on the right-eye field image data. Is displayed. In this way, the user U can visually recognize the visual field image three-dimensionally due to the parallax between the left-eye visual field image and the right-eye visual field image.

  The processes in steps S1 to S4 shown in FIG. 6 may be executed for each frame. For example, when the frame rate of the moving image is 90 fps, the processes in steps S1 to S4 may be repeatedly executed at intervals of ΔT = 1/90 (seconds). As described above, since the processes in steps S1 to S4 are repeatedly performed at predetermined intervals, the visual field of the virtual camera 300 is updated according to the operation of the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110. Is updated.

  Next, the hardware configuration of the server 2 shown in FIG. 1 will be described with reference to FIG. FIG. 10 is a diagram illustrating a hardware configuration of the server 2. As shown in FIG. 10, the server 2 includes a control unit 23, a storage unit 22, a communication interface 21, and a bus 24. The control unit 23, the storage unit 22, and the communication interface 21 are communicably connected to each other via a bus 24. The control unit 23 includes a memory and a processor. The memory includes, for example, a ROM and a RAM, and the processor includes, for example, a CPU, an MPU, and / or a GPU.

  The storage unit (storage) 22 is, for example, a large-capacity HDD. The storage unit 22 may store a control program for causing a computer to execute at least a part of the information processing method according to the present embodiment, or a control program for realizing sharing of a virtual space by a plurality of users. Further, the storage unit 22 may store user management information for managing each user and data regarding various images and objects (for example, hand objects). The communication interface 21 is configured to connect the server 2 to the communication network 3.

  Hereinafter, the processing flow of the virtual space distribution system 100 according to the present embodiment will be described in detail with reference to FIGS. The state (a) of FIG. 11 is a diagram showing a real space where the user U wearing the HMD 110 and the external controllers 320R and 320L exists. The state (b) of FIG. 11 is a diagram showing a virtual space 200 including the virtual camera 300 and various virtual objects. FIG. 12 is a flowchart for explaining the information processing method according to the present embodiment. FIG. 13 is a diagram showing the visual field image V1 in the state (b) of FIG.

  In the present embodiment, the virtual space 200 is configured as a game space provided by the virtual space distribution system 100. As shown in the state (b) of FIG. 11, the virtual space 200 includes a virtual camera 300, a collision object 350 (an example of a first object), a left hand object 400L, a right hand object 400R, and an enemy object 500 (second). An example of an object). The control unit 121 generates virtual space data that defines the virtual space 200 including these objects. The control unit 121 may update the virtual space data for each frame. As described above, the virtual camera 300 is interlocked with the movement of the HMD 110 worn by the user U shown in the state (a) of FIG. That is, the visual field of the virtual camera 300 is updated according to the movement of the HMD 110. The virtual camera 300 has a viewpoint associated with the first-person viewpoint of the user U.

  The collision object 350 is an object associated with the user U wearing the HMD 110. The collision object 350 is used for collision determination (hit determination) between the user U and another object (for example, the enemy object 500) in the virtual space 200. For example, when the collision object 350 and the enemy object 500 are in contact with each other, it is determined that the user U and the enemy object 500 are in contact with each other. The collision object 350 may be defined by a spherical shape having a diameter R with the center position of the virtual camera 300 as the center, for example. In the following description, it is assumed that the collision object 350 is formed in a spherical shape having a diameter R with the center position of the virtual camera 300 as the center. Note that the collision object 350 is configured as a transparent body and is not displayed in the field-of-view image V (V1 to V4).

  The left hand object 400L is interlocked with the movement of the external controller 320L attached to the left hand of the user U. Similarly, the right hand object 400R is interlocked with the movement of the external controller 320R attached to the right hand of the user U. These hand objects 400L and 400R may be provided with predetermined equipment (for example, a weapon object 450 shown in FIG. 13). In the example shown in FIG. 13, the hand object 400 includes a gun as the weapon object 450, but the type of the weapon object 450 equipped on the hand object 400 can be changed by operating the hand object 400. For example, the user U can equip the hand object 400 with a sword or a bow and arrow instead of the gun by performing a predetermined action determined according to the weapon object 450 to be equipped while holding the controller 320. it can.

  The enemy object 500 is an object that has a predetermined influence on the user U, and includes, for example, a weapon object 600 (an example of an action part). The control unit 121 controls the enemy object 500 so that the enemy object 500 attacks the user U using the weapon object 600, that is, the weapon object 600 performs a predetermined action on the collision object 350. To do. The enemy object 500 and the weapon object 600 each have a collision area. In the present embodiment, it is assumed that the collision areas of the enemy object 500 and the weapon object 600 coincide with the areas constituting the enemy object 500 and the weapon object 600 (outer areas of the enemy object 500 and the weapon object 600), respectively. 11 and 13, the enemy object 500 includes a sword (sword) as the weapon object 600. However, the weapon object 600 differs depending on the type and attribute of the enemy object 500. Sometimes equipped with guns.

  A predetermined influence (collision effect) is given to the user U when the weapon object 600 included in the enemy object 500 contacts the collision object 350. Specifically, when the weapon object 600 collides with the collision object 350 due to the attack of the enemy object 500, for example, the user U receives a predetermined amount of damage, and a predetermined parameter associated with the user U, for example, the user U Hit points (hereinafter referred to as HP) decrease.

  Note that the user U can affect the enemy object 500 by operating the hand object 400. For example, the enemy object 500 can be attacked and damaged by using the weapon object 450 included in the hand object 400 by a predetermined action of the user U.

  Hereinafter, the flow of the game progress process provided by the virtual space distribution system 100 according to the present embodiment will be described in detail with reference to FIGS. Note that the game according to the present embodiment is a battle game in which the user U wearing the HMD 110 battles the enemy object 500 placed in the virtual space 200. As described above, the user U can attack the enemy object 500 using the hand object 400 or the weapon object 450 included in the hand object 400, but in this embodiment, the user U attacks from the enemy object 500. A description will be given centering on the processing when received.

  When this competitive game is started, in step S10 shown in FIG. 12, the control unit 121 (see FIG. 3) performs visual field image data based on the virtual space data defining the virtual space 200 and the visual field CV of the virtual camera 300. And the field image is displayed on the display unit 112 of the HMD 110 based on the field image data. Step S10 is the same as step S1 shown in FIG. For example, when the virtual camera 300 faces the enemy object 500, the enemy object 500 is displayed in the field-of-view image V1, as shown in FIG.

  Next, in step S <b> 12, the control unit 121 determines whether the attack from the enemy object 500 hits the user U, that is, the weapon object 600 included in the enemy object 500 is in the collision area CA defined around the virtual camera 300. Determine if there is a collision. As illustrated in FIG. 14, when it is determined that the weapon object 600 has contacted the collision area CA in the virtual space 200 (Yes in Step S12), the control unit 121 darkens the visual field image V1 in Step S14. . The processing in step S14 may be processing that changes the display mode of the visual field image, for example, processing that reduces the visual effect given to the user U. For example, instead of darkening the visual field image V1, the visual field image V1 is blurred. Processing may be performed.

  Next, in step S16, the control unit 121 causes the display unit 112 to display the visual field image V2 illustrated in FIG. In the field-of-view image V2 shown in FIG. 15, an attack display 700 for indicating that an attack from the enemy object 500 hits the user U is displayed at the center thereof, and an HP display 720 (HP display 720 regarding the HP of the user U) An example of the first display) is displayed. As a result, the user U can intuitively grasp the attack from the enemy object 500 and how much the HP has decreased due to the attack received, and the HP display 720 and the attack display 700 The field of view is narrowed.

  By the way, when the weapon object 600 collides with the collision area CA and passes through the collision area CA, if the weapon object 600 is left displayed in the field-of-view image V2, a weapon that is not supposed to be rendered. The inside of the object 600 appears in the field of view of the user U, which may result in an unnatural visual field image. Therefore, in step S14, the control unit 121 may hide the weapon object 600 on the field image V2 in the field image V2 while the weapon object 600 collides with the collision area CA and passes through the collision area CA. preferable. Here, as a method of “hiding the weapon object 600 on the visual field image V <b> 2”, the control unit 121 performs object control so that the weapon object 600 itself arranged in the virtual space 200 is hidden. The weapon object 600 may not be rendered when rendering the visual field image V2 on the display unit 112 of the HMD 110.

  Further, as shown in FIG. 15, in step S <b> 16, the control unit 121 sets a peripheral region (hereinafter referred to as a colored region R <b> 2) of the visual field image V <b> 2 of the remaining portion R <b> 1 of the view as an image of the virtual space 200. Is subjected to image processing such as coloring with a different color (for example, red). Thereby, the field of view when attacked from the enemy object 500 can be further narrowed.

  Next, in step S18, the control unit 121 determines whether or not a certain time has elapsed from the start of display of the visual field image V2 shown in FIG. When it is determined that a certain time (for example, 2 to 3 seconds) has elapsed (Yes in Step S18), in Step S20, the control unit 121 displays the visual field image V3 illustrated in FIG. To do. In step S20, the enemy object 500 is displayed again according to the orientation of the virtual camera 300, and the portion of the visual field image V3 excluding the weapon object 600 is displayed in grayscale, thereby narrowing the field of view (reducing visual effects). ) Instead of grayscale, the blurring process may be performed on the portion of the visual field image V3 excluding the weapon object 600. Thus, after displaying the visual field image V2 in the state (first state) in step S16, the control unit 121 continues the state from the first state in step S20 (second state for narrowing the field of view of the user U). State) visual field image V3 is displayed on the display unit 112. Thereby, the user U can easily recognize that his / her HP has decreased.

  If the display of the visual field image V2 including the attack display 700 and the HP display 720 is continued after receiving an attack from the enemy object 500, the user U cannot visually recognize the attack from the enemy object 500 again. , There is a risk that the game nature is impaired. Therefore, in this embodiment, after the visual field image V2 (first state visual field image) is displayed for a certain period of time, the visual field image V3 (second state visual field image) different from the display mode of the visual field image V2 is displayed on the display unit 112. Is displayed. That is, in the visual field image V3 in the second state, the attack display 700 and the HP display 720 are not displayed, and the visual field image V3 is displayed, for example, in gray scale, so that the game performance is not impaired.

  In the view image V3, it is preferable that only the weapon object 600 is left in a display mode that is not subjected to grayscale or blurring processing, that is, in color display, like the view image V1. That is, it is preferable that the visibility of the weapon object 600 in the visual field image V <b> 3 is higher than the visibility of the portion other than the weapon object 600 by displaying only the weapon object 600 in color. As a result, even when the user U's HP decreases and the field of view is narrowed, the visibility of the weapon object 600 that is an attack site is maintained, so that the game performance can be improved.

  Next, in step S22, the control unit 121 determines whether or not the user U's HP is equal to or lower than a predetermined threshold. When it is determined that the HP of the user U is equal to or lower than the predetermined threshold (Yes in Step S22), in Step S24, the control unit 121 displays the visual field image V4 (third state visual field image) illustrated in FIG. It is displayed on the display unit 112. In this step S24, in addition to displaying the visual field image V3 in gray scale, the colored region R2 which is the peripheral portion of the visual field image V4 has a color different from the central part of the visual field image V4, for example, a gray color in the central part. It is colored with a darker color (dark gray, black, etc.). Thereby, in the visual field image V4, the field of view of the user U is further narrowed. Thus, when the user U's HP falls below a predetermined threshold, the user's U field of view is displayed in a display mode different from the display mode of the visual field image V2 (first state) and the visual field image V3 (second state). By being narrowed, the user U can recognize the decrease in HP in stages. As the HP of the user U decreases, for example, the inner side of the colored region R2 gradually approaches the center side of the visual field image V4 (the range in which the enemy object 500 can be visually recognized is gradually reduced) The field of view may be narrowed in stages.

  Next, in step S26, the control unit 121 determines whether the HP of the user U has become zero. When it is determined that the HP of the user U has become zero (Yes in step S26), in step S28, the control unit 121 ends the battle game, and the battle is ended (the user U has lost). The user U is notified (for example, displayed on the display unit 112), and the series of processing ends. When the HP of the enemy object 500 becomes zero before the HP of the user U becomes zero, the control unit 121 ends the battle game, and the display unit 112 indicates that the user U has won. Can be displayed.

  As described above, when the weapon object 600 included in the enemy object 500 performs an attack on the collision area CA of the virtual camera 300, for example, the control unit 121 displays the HP display 720 or the attack to narrow the field of view. The display 700 is displayed in the field-of-view image V2, and the process of making the display mode of the field-of-view image V2 different for the remaining portion R1 (including the colored region R2) of the narrowed field of view, for example, reducing the visual effect on the user U (For example, darkening processing or coloring / blurring processing). In this way, when the user U is attacked from the enemy object 500, the visual effect is reduced by narrowing the field of view with the HP display 720 and the like, and changing the display mode of the remaining portion R1 of the narrowed field of view. The user U can intuitively recognize that the attack has been received. That is, according to this method, necessary information in the virtual space 200 can be provided to the user U without a sense of incongruity.

  In the above embodiment, the enemy object 500 arranged in the virtual space 200 attacks the user U using the weapon object 600, but is not limited to this example. For example, the enemy object 500 may attack the user U with a part of the body of the enemy object 500 such as a bare hand or a foot without using the weapon object 600. In this case, an attack part (the hand or foot of the enemy object 500) for attacking the user U is an operation part of the enemy object 500.

  Further, in the above embodiment, after darkening the visual field image V1 in step S14, the field of view is narrowed by displaying the HP display 720 and the colored region R2 in the visual field image V2 in step S16. It is not limited to examples. For example, when the weapon object 600 collides with the collision object 350, the visual effect of the portion (residual portion R1) that is not the HP display 720 or the coloring region R2 of the visual field image V2 is reduced instead of darkening the visual field image V1. After that, the HP display 720 and the colored region R2 may be displayed. In other words, the visual field may be narrowed after the visual effect is reduced by changing the display mode of the entire visual field image. After the visual field is narrowed, the visual field is changed by changing the display mode for the narrowed visual field part. You may make it reduce a target effect. Further, when the weapon object 600 collides with the collision object 350, a process for vibrating the field of view may be performed.

  Further, the colored region R2 displayed in step S16 or step S24 does not have to be provided in all corners of the visual field image, and is at least a part of the corners of the visual field image (for example, the left end portions of the visual field images V2 and V4). It may be provided only. In step S16, for example, only the HP display 720 may be displayed (that is, the attack display 700 and the colored region R2 may or may not be displayed).

  In the above-described embodiment, the collision object 350 is defined with the virtual camera 300 as a reference, taking a first-person viewpoint game as an example. For example, in the case of a third-person viewpoint game, the avatar object associated with the user U may be arranged at a position that can be viewed from the virtual camera 300 in the virtual space 200. In this case, the collision object may coincide with the outer diameter area of the avatar object, or may be defined as a certain range surrounding the avatar object. When the weapon object 600 collides with the collision object associated with the avatar object due to the attack of the enemy object 500, the control unit 121 proceeds with the processing after step S12 in FIG.

  As mentioned above, although embodiment of this indication was described, the technical scope of this invention should not be limitedly interpreted by description of this embodiment. This embodiment is an example, and it is understood by those skilled in the art that various modifications can be made within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the equivalents thereof.

  In the description of the present embodiment, it is assumed that the virtual space data indicating the virtual space 200 is updated on the user terminal 1 side, but the virtual space data may be updated on the server 2 side. Furthermore, although it is assumed that the visual field image data corresponding to the visual field image is updated on the user terminal 1 side, the visual field image data may be updated on the server 2 side. In this case, the user terminal 1 displays a field image on the HMD 110 based on the field image data transmitted from the server 2.

  Further, in order to implement various processes executed by the control unit 121 of the user terminal 1 by software, a control program for causing the computer (processor) to execute various processes may be incorporated in the storage unit 123 or the memory in advance. Good. Alternatively, the control program includes a magnetic disk (HDD, floppy disk), an optical disk (CD-ROM, DVD-ROM, Blu-ray disk, etc.), a magneto-optical disk (MO, etc.), flash memory (SD card, USB memory, SSD). Etc.) may be stored in a computer-readable storage medium. In this case, when the storage medium is connected to the control device 120, the control program stored in the storage medium is incorporated into the storage unit 123. The control unit 121 executes various processes by loading the control program incorporated in the storage unit 123 onto the RAM and executing the loaded program.

  The control program may be downloaded from a computer on the communication network 3 via the communication interface 125. Similarly in this case, the downloaded control program is incorporated into the storage unit 123.

  In this embodiment, the virtual space is used to provide a virtual experience such as VR (Virtual Reality), AR (Arranged Reality), and MR (Mixed Reality) to the user. When the virtual space provides VR, background data stored in the memory is used as the background of the virtual space. When the virtual space provides AR or MR, the real space is used as the background. In this case, the HMD 110 includes a transmissive display device (optical see-through or video see-through display device), so that the real space can be used as a background. If virtual space is applied to MR, the object may be influenced by real space. As described above, when the virtual space includes at least a part of a virtual scene such as a background or a virtual object, the user can be provided with a virtual experience capable of interacting with the virtual scene.

1, 1A, 1B: User terminal 2: Server 3: Communication network 21: Communication interface 22: Storage unit 23: Control unit 24: Bus 100: Virtual space distribution system 110: Head mounted device (HMD)
112: Display unit 114: HMD sensor 116: Headphone 118: Microphone 120: Control device 121: Control unit 123: Storage unit 124: I / O interface 125: Communication interface 126: Bus 130: Position sensor 140: Gaze sensor 200, 200A , 200B: Virtual space 210: Center position 300: Virtual camera 320: External controller 320L: Left hand external controller (controller)
320R: External controller for right hand (controller)
350: Collision object (example of first object)
400L, 400R: Hand object 450: Weapon object 500: Enemy object (example of second object)
600: Weapon object (example of moving part)
700: Attack display 720: HP display (example of first display)
R1: Residual portion R2: Colored regions V, V1 to V4: Field image

Claims (9)

An information processing method in a system including a head mounted device,
(A) a virtual space including a first object associated with a user of the head-mounted device and a second object that includes a motion part and performs a predetermined motion on the first object by the motion part. Generating virtual space data to be defined;
(B) generating visual field image data indicating a visual field image displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) A first process of narrowing the field of view of the field-of-view image and changing the display mode of the field-of-view image based on the fact that the predetermined motion is performed on the first object by the motion part. A step of controlling the display of the state;
( D ) After performing the display control of the first state in the step (c) , the visual effect of the visual field image is reduced so as to reduce the visual effect of the visual field image than before the predetermined operation is performed . Performing display control of the second state for performing processing for changing the display mode ;
Including
In the step ( d ), in the visual field image, the visibility of a part based on the motion part is higher than the visibility of a part other than the part based on the motion part. In step (c), while the predetermined motion is being performed on the first object by the motion portion, the motion portion is not displayed in the visual field image, and the execution of the predetermined motion is completed. Then, the information processing method according to claim 1 , wherein the operation part is redisplayed in the visual field image . The information according to claim 1 or 2 , wherein in the step ( c ), display control of the first state is performed by displaying a first display relating to a predetermined parameter associated with the predetermined operation on the visual field image. Processing method. In the step ( c ), the visual effect at the corner of the visual field image is reduced to narrow the field of view, and the first display is displayed on the center side of the visual field image from the corner. Item 4. The information processing method according to Item 3 . (E) When a predetermined parameter associated with the predetermined operation falls below a threshold value, the display mode is different from the display mode of the visual field image in the first state and the second state. The information processing method according to any one of claims 1 to 4 , further comprising a step of performing display control in a third state in which processing for narrowing the field of view is performed. In step (d), the portion based on the operation portion is displayed in color, other than the portion based on the operation portion is displayed in grayscale, processing according to any one of claims 1 5 Method. An information processing program for causing a computer to execute the information processing method according to any one of claims 1 to 6 . An information processing system using a head-mounted device, the information processing system configured to execute the information processing method according to any one of claims 1 to 6 . A processor;
An information processing apparatus comprising a memory for storing computer-readable instructions,
The information processing apparatus executes the information processing method according to any one of claims 1 to 6 , when the computer-readable instruction is executed by the processor.

Images