JP6190497B1 - Information processing method and program for causing computer to execute information processing method - Google Patents

Abstract

An information processing method for providing directivity to sound output from an object defined as a sound source in a virtual space. An information processing method includes generating virtual space data defining a virtual space 200 including a virtual camera 300 and a sound source object MC defined as a sound source of sound input to a microphone, and movement of an HMD The visual field CV of the virtual camera 300, a step of generating visual field image data based on the visual field CV of the virtual camera 300 and the virtual space data, and a visual field in the HMD based on the visual field image data. In the step of displaying an image, and in the field of view CV of the virtual camera 300 in the virtual space 200, an attenuation coefficient that defines the attenuation per unit distance of the sound propagating in the virtual space 200 is set to the attenuation coefficient α1, and the virtual Outside the field of view CV of the camera 300, setting the attenuation coefficient to an attenuation coefficient α2 different from the attenuation coefficient α1; Including the. [Selection] Figure 10

Description

  The present disclosure relates to an information processing method and a program for causing a computer to execute the information processing method.

  Patent Document 1 calculates a relative positional relationship between a moving object and a sound source when the moving object serving as a viewpoint in the game space moves or when the sound source moves during the execution of the game program, A process (sound image localization process) for processing a sound output from a sound source using a localization parameter based on the calculated relative positional relationship is disclosed.

JP 2007-050267 A

  However, Patent Document 1 does not disclose any technique for imparting directivity to sound output from an object defined as a sound source in a virtual space (VR (Virtual Reality) space).

  An object of the present invention is to provide an information processing method for providing directivity to sound output from an object defined as a sound source in a virtual space, and a program for causing a computer to execute the information processing method.

According to an aspect of the present disclosure, an information processing method in a system including a first user terminal having a first head mounted display and a voice input unit is provided.
The information processing method is as follows:
Generating virtual space data defining a virtual space including a virtual camera and a sound source object defined as a sound source of sound input to the sound input unit;
Determining a field of view of the virtual camera in response to movement of the first head mounted display;
Generating visual field image data based on the virtual camera visual field and virtual space data;
Displaying a field image on the first head mounted display based on the field image data;
In the first area of the virtual space, an attenuation coefficient that defines an attenuation amount per unit distance of sound propagating in the virtual space is set as the first attenuation coefficient, and the virtual space different from the first area In a second region, setting the attenuation coefficient to a second attenuation coefficient;
including.
The first attenuation coefficient and the second attenuation coefficient are different from each other.

  According to the present disclosure, it is possible to provide an information processing method that imparts directivity to sound output from an object defined as a sound source in a virtual space. Further, it is possible to provide a program for causing a computer to execute the information processing method.

It is the schematic which shows the structure of a game system. It is the schematic which shows this head mounted display (HMD) system. It is a figure which shows the head of the user wearing HMD. It is a figure which shows the hardware constitutions of a control apparatus. It is a flowchart which shows the process which displays a visual field image on HMD. It is xyz space figure which shows an example of virtual space. (A) is yx top view of the virtual space shown in FIG. FIG. 7B is a zx plan view of the virtual space illustrated in FIG. 6. It is a figure which shows an example of the visual field image displayed on HMD. It is a flowchart for demonstrating the information processing method which concerns on 1st Embodiment. When a virtual camera and a sound source object are integrally configured, an ally avatar object located within the visual field of the virtual camera (an example of the first area) and an enemy located outside the visual field of the virtual camera (an example of the second area) It is a figure which shows an avatar object. It is a figure which shows the self avatar object and friend avatar object which are located in the visual field of a virtual camera, and the enemy avatar object located out of the visual field of a virtual camera in case a self-avatar object and a sound source object are comprised integrally. . It is a flowchart for demonstrating the information processing method which concerns on the 2nd modification of 1st Embodiment. In the case where the virtual camera and the sound source object are integrally configured, an ally avatar object located in the gaze area (an example of the first area) and a field of view of the virtual camera other than the gaze area (an example of the second area) It is a figure which shows the enemy avatar object located. It is a flowchart for demonstrating the information processing method which concerns on the 3rd modification of 1st Embodiment. When a virtual camera and a sound source object are integrally configured, a teammate avatar object positioned in the visual axis region (an example of the first region) and a visual field of the virtual camera other than the visual axis region (an example of the second region) It is a figure which shows the enemy avatar object located in (). It is a flowchart for demonstrating the information processing method which concerns on the 4th modification of 1st Embodiment. When the self-avatar object and the sound source object are integrally configured, the teammate avatar object and the self-avatar object that are located inside the attenuation object (within the visual field of the virtual camera) and the outside of the attenuation object (out of the visual field of the virtual camera) It is a figure which shows the enemy avatar object located in. It is a flowchart for demonstrating the information processing method which concerns on 2nd Embodiment. It is a flowchart for demonstrating the information processing method which concerns on the 1st modification of 2nd Embodiment. It is a figure which shows the friend avatar object located inside the attenuation | damping object, and the enemy avatar object located outside the attenuation | damping object when a virtual camera and a sound source object are comprised integrally.

[Description of Embodiments Presented by the Present Disclosure]
An overview of an embodiment indicated by the present disclosure will be described.
(1) An information processing method in a system including a first user terminal having a first head mounted display and a voice input unit,
Generating virtual space data defining a virtual space including a virtual camera and a sound source object defined as a sound source of sound input to the sound input unit;
Determining a field of view of the virtual camera in response to movement of the first head mounted display;
Generating visual field image data based on the virtual camera visual field and virtual space data;
Displaying a field image on the first head mounted display based on the field image data;
In the first area of the virtual space, an attenuation coefficient that defines an attenuation amount per unit distance of sound propagating in the virtual space is set as the first attenuation coefficient, and the virtual space different from the first area In a second region, setting the attenuation coefficient to a second attenuation coefficient;
Including
The information processing method, wherein the first attenuation coefficient and the second attenuation coefficient are different from each other.

  According to the above method, the attenuation coefficient is set to the first attenuation coefficient in the first area of the virtual space, and the attenuation coefficient is set to the second attenuation coefficient different from the first attenuation coefficient in the second area of the virtual space. Is set. In this way, since different attenuation coefficients are set in the first and second regions, the sound output from the sound source object can be given directivity in the virtual space.

(2) The system further includes a second user terminal having a second head mounted display and an audio output unit,
The virtual space further includes an avatar object associated with the second user terminal,
The information processing method includes:
Obtaining voice data indicating voice input to the voice input unit;
Identifying a relative positional relationship between the sound source object and the avatar object;
Determining whether the avatar object is located in the first region;
Processing the audio data based on the specified relative positional relationship and the attenuation constant;
Outputting audio corresponding to the processed audio data to the audio output unit;
Further including
If it is determined that the avatar object is located in the first region, the attenuation coefficient is set to the first attenuation coefficient, and then the audio data includes the relative positional relationship and the first attenuation constant. Is processed based on
When it is determined that the avatar object is located in the second region, the attenuation coefficient is set to the second attenuation coefficient, and then the audio data includes the relative positional relationship and the second attenuation coefficient. Processed based on the
The information processing method according to item (1).

  According to the above method, when it is determined that the avatar object is located in the first region, the attenuation coefficient is set to the first attenuation coefficient, and thereafter, the audio data includes the relative positional relationship and the first attenuation constant. It is processed based on. On the other hand, when it is determined that the avatar object is located in the second region, the attenuation coefficient is set to the second attenuation coefficient, and then the audio data is processed based on the relative positional relationship and the second attenuation coefficient. Is done.

  Thus, the volume (sound pressure level) of the sound output from the sound output unit varies depending on the position of the avatar object in the virtual space. For example, when it is assumed that the first attenuation coefficient is smaller than the second attenuation coefficient, the sound volume output from the audio output unit when the avatar object is present in the first area is the avatar object is present in the second area. In this case, the volume is higher than the volume of the sound output from the sound output unit. For this reason, when the teammate avatar object exists in the first area and the enemy avatar object exists in the second area, the user of the first user terminal does not notice the friend avatar object without operating by the user operating the enemy avatar object. It is possible to give an instruction to the operating user by voice. Therefore, the entertainment property of the virtual space can be improved.

(3) the first region is within the field of view of the virtual camera;
The second region is outside the field of view of the virtual camera;
The information processing method according to item (1) or (2).

  According to the above method, when it is determined that the avatar object is located in the field of view, the attenuation coefficient is set to the first attenuation coefficient, and then the audio data is converted into the relative positional relationship and the first attenuation constant. Processed based on. On the other hand, when it is determined that the avatar object is located outside the field of view, the attenuation coefficient is set to the second attenuation coefficient, and then the audio data is processed based on the relative positional relationship and the second attenuation coefficient. The

  Thus, the volume (sound pressure level) of the sound output from the sound output unit varies depending on the position of the avatar object in the virtual space. For example, when it is assumed that the first attenuation coefficient is smaller than the second attenuation coefficient, the sound volume output from the audio output unit when the avatar object exists in the field of view is the case where the avatar object exists outside the field of view. Becomes larger than the volume of the sound output from the sound output unit. For this reason, when the friend avatar object exists in the field of view and the enemy avatar object exists outside the field of view, the user of the first user terminal operates the friend avatar object without being noticed by the user operating the enemy avatar object. It is possible to give an instruction to the user by voice. Therefore, the entertainment property of the virtual space can be improved.

(4) The first area is a gaze area defined by the line of sight of the user wearing the first head mounted display,
The second area is within the field of view of the virtual camera other than the gaze area.
The information processing method according to item (1) or (2).

  According to the above method, when it is determined that the avatar object is located in the gaze area, the attenuation coefficient is set to the first attenuation coefficient, and then the audio data is converted into the relative positional relationship and the first attenuation constant. Processed based on. On the other hand, when it is determined that the avatar object is located within the visual field of the virtual camera other than the gaze area (hereinafter simply referred to as the non-gaze area), the attenuation coefficient is set to the second attenuation coefficient, and then the voice The data is processed based on the relative positional relationship and the second attenuation coefficient.

  Thus, the volume (sound pressure level) of the sound output from the sound output unit varies depending on the position of the avatar object in the virtual space. For example, when it is assumed that the first attenuation coefficient is smaller than the second attenuation coefficient, the sound volume output from the audio output unit when the avatar object exists in the gaze area is the avatar object exists in the non-gaze area In this case, the volume is higher than the volume of the sound output from the sound output unit. For this reason, when an ally avatar object exists in the gaze area and an enemy avatar object exists in the non-gaze area, the user of the first user terminal operates the ally avatar object without being noticed by the user operating the enemy avatar object. It is possible to give an instruction to the user by voice. Therefore, the entertainment property of the virtual space can be improved.

(5) The first region is a visual axis region defined by the visual axis of the virtual camera,
The second region is within the field of view of the virtual camera other than the visual axis region.
The information processing method according to item (1) or (2).

  According to the above method, when it is determined that the avatar object is located in the visual axis region, the attenuation coefficient is set to the first attenuation coefficient, and thereafter, the audio data includes the relative positional relationship and the first attenuation constant. It is processed based on. On the other hand, when it is determined that the avatar object is located in the field of view of the virtual camera other than the visual axis region (hereinafter, simply referred to as the off-axial region), the attenuation coefficient is set to the second attenuation coefficient, and thereafter The audio data is processed based on the relative positional relationship and the second attenuation coefficient.

  Thus, the volume (sound pressure level) of the sound output from the sound output unit varies depending on the position of the avatar object in the virtual space. For example, when it is assumed that the first attenuation coefficient is smaller than the second attenuation coefficient, the sound volume output from the audio output unit when the avatar object exists in the visual axis region Is larger than the volume of the sound output from the sound output unit. For this reason, when the friend avatar object exists in the visual axis region and the enemy avatar object exists in the off-axis region, the user of the first user terminal is not noticed by the user operating the enemy avatar object. It is possible to give an instruction by voice to the user who operates. Therefore, the entertainment property of the virtual space can be improved.

(6) An information processing method in a system including a first user terminal having a first head mounted display and a voice input unit,
Generating virtual space data defining a virtual space including a virtual camera and a sound source object defined as a sound source of sound input to the sound input unit;
Determining a field of view of the virtual camera in response to movement of the first head mounted display;
Generating visual field image data based on the visual field of the virtual camera and the virtual space data;
Displaying a field image on the first head mounted display based on the field image data;
Including
The virtual space further includes an attenuation object that is disposed at a boundary between the first region and the second region of the virtual space, and that defines an attenuation amount of sound propagating in the virtual space.
Information processing method.

  According to the above method, the attenuation object that defines the attenuation amount of the sound propagating in the virtual space is arranged at the boundary between the first region and the second region of the virtual space. For this reason, for example, the attenuation amount of the sound output from the sound source object defined as the sound source is different in each of the first region and the second region of the virtual space. Thus, directivity can be given to the sound output from the sound source object in the virtual space.

  (7) The information processing method according to item (6), wherein the attenuation object is transparent and is not displayed in the visual field image.

  According to the above method, since the attenuation object is not displayed in the field-of-view image, the sound output from the sound source object has directivity without impairing the user's sense of immersion in the virtual space (the sense of presence in the virtual space). Can do.

  (8) The information processing method according to item (6) or (7), wherein the sound source object is arranged in a first region of the virtual space.

  According to the above method, the sound source object is arranged in the first area of the virtual space. For this reason, in the second region of the virtual space, the sound output from the sound source object is further attenuated by an attenuation amount defined by the attenuation object, as compared with the first region of the virtual space. In this way, the sound output from the sound source object can be given directivity.

(9) The system further includes a second user terminal having a second head mounted display and an audio output unit,
The virtual space further includes an avatar object associated with the second user terminal,
The information processing method includes:
Obtaining voice data indicating voice input to the voice input unit;
Identifying a relative positional relationship between the sound source object and the avatar object;
Determining whether the avatar object is located in a first region of the virtual space;
Processing the audio data;
Outputting audio corresponding to the processed audio data to the audio output unit;
Further including
When it is determined that the avatar object is located in the first region of the virtual space, the audio data is processed based on the relative positional relationship,
When it is determined that the avatar object is located in the second region of the virtual space, the audio data is processed based on the relative positional relationship and the attenuation amount defined by the attenuation object.
The information processing method according to item (8).

According to the above method, when it is determined that the avatar object is located in the first region of the virtual space where the sound source is located, the audio data is processed based on the relative positional relationship. On the other hand, when it is determined that the avatar object is located in the second region of the virtual space, the audio data is processed based on the relative positional relationship and the attenuation amount defined by the attenuation object.
Thus, the volume (sound pressure level) of the sound output from the sound output unit varies depending on the position of the avatar object in the virtual space. The sound volume output from the sound output unit when the avatar object exists in the first area of the virtual space is the sound volume of the sound output from the sound output unit when the avatar object exists in the second area of the virtual space. Bigger than. For this reason, when the friend avatar object exists in the first area of the virtual space and the enemy avatar object exists in the second area of the virtual space, the user of the first user terminal notices the user operating the enemy avatar object. Instead, it is possible to give a voice instruction to the user who operates the teammate avatar object. Therefore, the entertainment property of the virtual space can be improved.

(10) The first region is within the field of view of the virtual camera;
The second region is outside the field of view of the virtual camera;
The information processing method according to item (8) or (9).

  According to the above method, the sound source object is arranged in the visual field of the virtual camera, and the attenuation object is arranged at the boundary of the visual field of the virtual camera. For this reason, outside the visual field of the virtual camera, the sound output from the virtual camera is further attenuated by an attenuation amount defined by the attenuation object as compared to the visual field of the virtual camera. Thus, directivity can be given to the sound output from the sound source object in the virtual space.

  (11) A program for causing a computer to execute the information processing method according to any one of items (1) to (10) is provided.

  According to the above, it is possible to provide a program that can give directivity to the sound output from the sound source object defined as the sound source in the virtual space.

[Details of Embodiments Presented by the Present Disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In addition, about the member which has the same reference number as the member already demonstrated in description of this embodiment, the description is not repeated for convenience of explanation.

  A configuration of a game system 100 that realizes an information processing method according to an embodiment of the present disclosure (hereinafter simply referred to as the present embodiment) will be described with reference to FIG. FIG. 1 is a schematic diagram showing the configuration of the game system 100. As shown in FIG. 1, the game system 100 includes a head-mounted display (hereinafter simply referred to as an HMD) system 1A (an example of a first user terminal) operated by a user X and an HMD system 1B (first operation) operated by a user Y. An example of a two-user terminal), an HMD system 1C operated by a user Z (an example of a third user terminal), and a game server 2 that controls the HMD systems 1A to 1C in synchronization. The HMD systems 1A, 1B, 1C and the game server 2 are connected to each other via a communication network 3 such as the Internet so that they can communicate with each other. In the present embodiment, a client server system including the HMD systems 1A to 1C and the game server 2 is constructed. However, without providing the game server 2, the HMD system 1A, the HMD system 1B, and the HMD system 1C are mutually connected. Direct communication (P2P communication) may be performed. For convenience of explanation, the HMD systems 1A, 1B, and 1C may be simply referred to as the HMD system 1. Further, the HMD systems 1A, 1B, and 1C are assumed to have the same configuration.

  Next, the configuration of the HMD system 1 will be described with reference to FIG. FIG. 2 is a schematic diagram showing the HMD system 1. As shown in FIG. 2, the HMD system 1 includes an HMD 110 attached to the head of the user U, headphones 116 (an example of an audio output unit) attached to both ears of the user U, and the periphery of the mouth of the user U. A microphone 118 (an example of a voice input unit), a position sensor 130, an external controller 320, and a control device 120.

  The HMD 110 includes a display unit 112, an HMD sensor 114, and a gaze sensor 140. The display unit 112 includes a non-transmissive display device configured to completely cover the field of view (field of view) of the user U wearing the HMD 110. Thereby, the user U can immerse in the virtual space by viewing only the visual field image displayed on the display unit 112. The display unit 112 includes a display unit for the left eye configured to provide an image to the left eye of the user U and a display unit for the right eye configured to provide an image to the right eye of the user U. Also good.

  The HMD sensor 114 is mounted in the vicinity of the display unit 112 of the HMD 110. The HMD sensor 114 includes at least one of a geomagnetic sensor, an acceleration sensor, and a tilt sensor (such as an angular velocity sensor and a gyro sensor), and can detect various movements of the HMD 110 mounted on the head of the user U.

The gaze sensor 140 has an eye tracking function that detects the direction of the line of sight of the user U.
The gaze sensor 140 may include, for example, a right eye gaze sensor and a left eye gaze sensor. The right eye gaze sensor irradiates, for example, infrared light to the right eye of the user U, and detects reflected light reflected from the right eye (particularly the cornea and iris), thereby acquiring information related to the rotation angle of the right eye's eyeball. May be. On the other hand, the left eye gaze sensor irradiates the left eye of the user U with, for example, infrared light, and detects reflected light reflected from the left eye (particularly the cornea and iris), thereby providing information on the rotation angle of the left eye's eyeball. May be obtained.

  The headphones 116 are respectively attached to the left ear and the right ear of the user U. The headphones 116 are configured to receive audio data (electrical signals) from the control device 120 and output audio based on the received audio data. The sound output to the right ear speaker of the headphones 116 may be different from the sound output to the left ear speaker of the headphones 116. For example, the control device 120 obtains audio data input to the right ear speaker and audio data input to the left ear speaker based on the head-related transfer function, and the two different audio data are stored in the headphones 116. May be output to each of the left ear speaker and the right ear speaker. As another example of the audio output unit, two independent stationary speakers and earphones may be provided.

  The microphone 118 is configured to collect the voice uttered by the user U and generate voice data (electrical signal) based on the collected voice. Furthermore, the microphone 118 is configured to transmit audio data to the control device 120. The microphone 118 may have a function of AD (Analog-to-Digital) conversion of audio data. In addition, the microphone 118 may be physically connected to the headphones 116. The control device 120 may process the received audio data and transmit the processed audio data to another HMD system via the communication network 3.

  The position sensor 130 is configured by, for example, a position tracking camera, and is configured to detect the positions of the HMD 110 and the external controller 320. The position sensor 130 is communicably connected to the control device 120 by wireless or wired communication, and is configured to detect information on the position, inclination, or light emission intensity of a plurality of detection points (not shown) provided in the HMD 110. . Further, the position sensor 130 is configured to detect information on the position, inclination and / or light emission intensity of a plurality of detection points (not shown) provided in the external controller 320. The detection point is, for example, a light emitting unit that emits infrared light or visible light. The position sensor 130 may include an infrared sensor and a plurality of optical cameras.

  The external controller 320 is used, for example, to control the movement of the finger object displayed in the virtual space. The external controller 320 may include a right-handed external controller used by the user U with the right hand and a left-handed external controller used by the user U with the left hand. The external controller for the right hand is a device that detects the position of the right hand of the user U and the movement of the finger of the right hand. The left-hand external controller is a device that detects the position of the left hand of the user U and the movement of the finger of the left hand. The external controller 320 may include a plurality of operation buttons, a plurality of detection points, a sensor, and a transceiver. For example, the menu object may be displayed in the virtual space when the operation button of the external controller 320 is operated by the user U. Further, the field of view of the user U in the virtual space may be changed by operating the operation button of the external controller 320 by the user U (that is, the field of view image may be changed). In this case, based on the operation signal output from the external controller 320, the control device 120 may move the virtual camera to a predetermined position.

  The control device 120 acquires the position information of the HMD 110 based on the information acquired from the position sensor 130, and wears the position of the virtual camera in the virtual space and the HMD 110 in the real space based on the acquired position information. Thus, the position of the user U can be accurately associated. Further, the control device 120 acquires the position information of the external controller 320 based on the information acquired from the position sensor 130, and the position of the finger object displayed in the virtual space based on the acquired position information. And the relative position between the external controller 320 and the HMD 110 in the real space can be accurately associated.

  Further, the control device 120 specifies the gaze of the right eye and the gaze of the left eye of the user U based on the information transmitted from the gaze sensor 140, and determines the gaze point that is the intersection of the gaze of the right eye and the gaze of the left eye. Can be identified. Furthermore, the control device 120 can specify the line-of-sight direction of the user U based on the specified gaze point. Here, the line-of-sight direction of the user U is the line-of-sight direction of both eyes of the user U and coincides with the direction of a straight line passing through the middle point of the line segment connecting the right eye and the left eye of the user U and the gazing point.

  Next, with reference to FIG. 3, a method for acquiring information related to the position and inclination of the HMD 110 will be described. FIG. 3 is a diagram illustrating the head of the user U wearing the HMD 110. Information on the position and inclination of the HMD 110 that is linked to the movement of the head of the user U wearing the HMD 110 can be detected by the position sensor 130 and / or the HMD sensor 114 mounted on the HMD 110. As shown in FIG. 2, three-dimensional coordinates (uvw coordinates) are defined centering on the head of the user U wearing the HMD 110. The vertical direction in which the user U stands up is defined as the v-axis, the direction orthogonal to the v-axis and passing through the center of the HMD 110 is defined as the w-axis, and the direction orthogonal to the v-axis and the w-axis is defined as the u-axis. The position sensor 130 and / or the HMD sensor 114 is an angle around each uvw axis (that is, a yaw angle indicating rotation around the v axis, a pitch angle indicating rotation around the u axis, and a center around the w axis). The inclination determined by the roll angle indicating rotation) is detected. The control device 120 determines angle information for controlling the visual axis of the virtual camera based on the detected angle change around each uvw axis.

  Next, the hardware configuration of the control device 120 will be described with reference to FIG. FIG. 4 is a diagram illustrating a hardware configuration of the control device 120. As illustrated in FIG. 4, the control device 120 includes a control unit 121, a storage unit 123, an I / O (input / output) interface 124, a communication interface 125, and a bus 126. The control unit 121, the storage unit 123, the I / O interface 124, and the communication interface 125 are connected to each other via a bus 126 so as to communicate with each other.

  The control device 120 may be configured as a personal computer, a tablet, or a wearable device separately from the HMD 110, or may be built in the HMD 110. In addition, some functions of the control device 120 may be mounted on the HMD 110, and the remaining functions of the control device 120 may be mounted on another device separate from the HMD 110.

  The control unit 121 includes a memory and a processor. The memory includes, for example, a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) having a plurality of work areas in which various programs executed by the processor are stored, and the like. The processor is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and / or a GPU (Graphics Processing Unit), and a program specified from various programs embedded in the ROM is expanded on the RAM. It is comprised so that various processes may be performed in cooperation with.

  In particular, the processor 121 develops a program (described later) for causing the computer to execute the information processing method according to the present embodiment on the RAM, and executes the program in cooperation with the RAM. Various operations of the control device 120 may be controlled. The control unit 121 displays a virtual space (view image) on the display unit 112 of the HMD 110 by executing a predetermined application program (game program) stored in the memory or the storage unit 123. Thereby, the user U can be immersed in the virtual space displayed on the display unit 112.

  The storage unit (storage) 123 is a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a USB flash memory, and is configured to store programs and various data. The storage unit 123 may store a program that causes a computer to execute the information processing method according to the present embodiment. In addition, a user U authentication program, a game program including data on various images and objects, and the like may be stored. Furthermore, a database including tables for managing various data may be constructed in the storage unit 123.

  The I / O interface 124 is configured to connect the position sensor 130, the HMD 110, the external controller 320, the headphones 116, and the microphone 118 to the control device 120 so that they can communicate with each other, for example, a USB (Universal) The terminal includes a serial bus (D) terminal, a digital visual interface (DVI) terminal, a high-definition multimedia interface (HDMI) (registered trademark) terminal, and the like. The control device 120 may be wirelessly connected to each of the position sensor 130, the HMD 110, the external controller 320, the headphones 116, and the microphone 118.

  The communication interface 125 is configured to connect the control device 120 to a communication network 3 such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. The communication interface 125 includes various wired connection terminals for communicating with an external device such as the game server 2 via the communication network 3 and various processing circuits for wireless connection, and communicates via the communication network 3. It is configured to conform to the communication standard.

  Next, processing for displaying a field-of-view image on the HMD 110 will be described with reference to FIGS. FIG. 5 is a flowchart showing a process for displaying the visual field image on the HMD 110. FIG. 6 is an xyz space diagram showing an example of the virtual space 200. FIG. 7A is a yx plan view of the virtual space 200 shown in FIG. FIG. 7B is a zx plan view of the virtual space 200 shown in FIG. FIG. 8 is a diagram illustrating an example of the visual field image V displayed on the HMD 110.

  As illustrated in FIG. 5, in step S <b> 1, the control unit 121 (see FIG. 4) generates virtual space data indicating the virtual space 200 including the virtual camera 300 and various objects. As shown in FIG. 6, the virtual space 200 is defined as an omnidirectional sphere with the center position 21 as the center (in FIG. 6, only the upper half celestial sphere is shown). In the virtual space 200, an xyz coordinate system with the center position 21 as the origin is set. The virtual camera 300 defines a visual axis L for specifying the visual field image V (see FIG. 8) displayed on the HMD 110. The uvw coordinate system that defines the visual field of the virtual camera 300 is determined so as to be linked to the uvw coordinate system that is defined around the head of the user U in the real space. Further, the control unit 121 may move the virtual camera 300 in the virtual space 200 in conjunction with the movement of the user U wearing the HMD 110 in the real space.

  Next, in step S <b> 2, the control unit 121 specifies the field of view CV (see FIG. 7) of the virtual camera 300. Specifically, the control unit 121 acquires information on the position and inclination of the HMD 110 based on data indicating the state of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114. Next, the control unit 121 identifies the position and orientation of the virtual camera 300 in the virtual space 200 based on information regarding the position and tilt of the HMD 110. Next, the control unit 121 determines the visual axis L of the virtual camera 300 from the position and orientation of the virtual camera 300 and specifies the visual field CV of the virtual camera 300 from the determined visual axis L. Here, the visual field CV of the virtual camera 300 corresponds to a part of the virtual space 200 visible to the user U wearing the HMD 110 (in other words, a part of the virtual space 200 displayed on the HMD 110). Equivalent to Further, the visual field CV has a first region CVa set as an angular range of the polar angle α around the visual axis L in the xy plane shown in FIG. 7A and an xz plane shown in FIG. 7B. And a second region CVb set as an angular range of the azimuth angle β with the visual axis L as the center. The control unit 121 identifies the line of sight of the user U based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140, and determines the direction of the virtual camera 300 based on the line of sight of the user U. May be.

  As described above, the control unit 121 can specify the visual field CV of the virtual camera 300 based on the data from the position sensor 130 and / or the HMD sensor 114. Here, when the user U wearing the HMD 110 moves, the control unit 121 changes the visual field CV of the virtual camera 300 based on the data indicating the movement of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114. be able to. That is, the control unit 121 can change the visual field CV according to the movement of the HMD 110. Similarly, when the line-of-sight direction of the user U changes, the control unit 121 can move the visual field CV of the virtual camera 300 based on the data indicating the line-of-sight direction of the user U transmitted from the gaze sensor 140. That is, the control unit 121 can change the visual field CV according to the change in the user U's line-of-sight direction.

  Next, in step S <b> 3, the control unit 121 generates visual field image data indicating the visual field image V displayed on the display unit 112 of the HMD 110. Specifically, the control unit 121 generates visual field image data based on virtual space data defining the virtual space 200 and the visual field CV of the virtual camera 300.

  Next, in step S4, the control unit 121 displays the field image V on the display unit 112 of the HMD 110 based on the field image data (see FIG. 7). As described above, the visual field CV of the virtual camera 300 is changed according to the movement of the user U wearing the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110 is changed. I can immerse myself in 200.

  Note that the virtual camera 300 may include a left-eye virtual camera and a right-eye virtual camera. In this case, the control unit 121 generates left-eye view image data indicating the left-eye view image based on the virtual space data and the view of the left-eye virtual camera. Further, the control unit 121 generates right-eye view image data indicating a right-eye view image based on the virtual space data and the view of the right-eye virtual camera. Thereafter, the control unit 121 displays the left-eye view image and the right-eye view image on the display unit 112 of the HMD 110 based on the left-eye view image data and the right-eye view image data. In this way, the user U can visually recognize the visual field image as a three-dimensional image from the left-eye visual field image and the right-eye visual field image. In this specification, for convenience of explanation, the number of virtual cameras 300 is one. Of course, the embodiment of the present disclosure is applicable even when the number of virtual cameras is two.

(First embodiment)
Next, the information processing method according to the present embodiment will be described with reference to FIGS. FIG. 9 is a flowchart for explaining the information processing method according to the first embodiment. FIG. 10 shows a teammate avatar object FC located in the field of view CV (an example of the first region) of the virtual camera 300 and the field of view CV of the virtual camera 300 when the virtual camera 300 and the sound source object MC are integrally configured. It is a figure showing enemy avatar object EC located outside (an example of the 2nd field)

  First, as shown in FIG. 10, the virtual space 200 includes a virtual camera 300, a sound source object MC, a teammate avatar object FC, and an enemy avatar object EC. The control unit 121 generates virtual space data that defines the virtual space 200 including these objects.

  The virtual camera 300 is associated with the HMD system 1A operated by the user X. That is, the position and orientation of the virtual camera 300 (that is, the visual field CV of the virtual camera 300) are changed according to the movement of the HMD 110 worn by the user X. The sound source object MC is defined as the sound source of the sound of the user X (see FIG. 1) input to the microphone 118 and is configured integrally with the virtual camera 300. When the sound source object MC and the virtual camera 300 are integrally configured, the virtual camera 300 may be interpreted as having a sound source function. The sound source object MC may be transparent. In this case, the sound source object MC is not displayed on the visual field image V. Note that the sound source object MC may be separated from the virtual camera 300. For example, the sound source object MC may be close to the virtual camera 300 and may follow the virtual camera 300 (that is, the sound source object MC may move according to the movement of the virtual camera 300).

  The teammate avatar object FC is associated with the HMD system 1B operated by the user Y. That is, the teammate avatar object FC is the avatar object of the user Y and is controlled by the operation of the user Y. Further, the teammate avatar object FC may function as a sound collector that collects sound propagating on the virtual space 200. In other words, the teammate avatar object FC may be configured integrally with a sound collection object that collects sound propagating on the virtual space 200.

  The enemy avatar object EC is controlled by the operation of the user Z. That is, the enemy avatar object EC is controlled by the operation of the user Z. Further, the enemy avatar object EC may function as a sound collector that collects sound propagating in the virtual space 200. In other words, the enemy avatar object EC may be configured integrally with a sound collection object that collects sound propagating in the virtual space 200. In this embodiment, when the users X to Z are playing an online game in which a large number of people can participate, the user X and the user Y are allies, while the user Z is an enemy against the users X and Y. It is assumed that

  Next, how the voice uttered by the user X is output from the headphone 116 of the user Y will be described with reference to FIG. As illustrated in FIG. 9, when the user X utters a sound toward the microphone 118, the microphone 118 of the HMD system 1 </ b> A collects the sound uttered from the user X and the sound data indicating the collected sound. Is generated (step S10). Thereafter, the microphone 118 transmits audio data to the control unit 121, and the control unit 121 acquires audio data corresponding to the user X's audio. The control unit 121 of the HMD system 1A transmits information regarding the position and orientation of the virtual camera 300 and audio data to the game server 2 via the communication network 3 (step S11).

  The game server 2 receives the information and audio data regarding the position and orientation of the virtual camera 300 of the user X from the HMD system 1A, and then transmits the information and audio data to the HMD system 1B (step S12). Thereafter, the control unit 121 of the HMD system 1B receives information and audio data regarding the position and orientation of the virtual camera 300 of the user X via the communication network 3 and the communication interface 125 (step S13).

  Next, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1B specifies the position of the user Y avatar object (that is, the teammate avatar object FC) (step S14). Thereafter, the control unit 121 specifies a distance D (an example of a relative positional relationship) between the virtual camera 300 (sound source object MC) of the user X and the teammate avatar object FC (step S15). The distance D may be the shortest distance between the virtual camera 300 of the user X and the teammate avatar object FC. In the present embodiment, since the virtual camera 300 and the sound source object MC are integrally configured, the distance D between the virtual camera 300 and the teammate avatar object FC is the distance between the sound source object MC and the teammate avatar object FC. It corresponds to the distance between.

  Next, the control unit 121 specifies the field of view CV of the virtual camera 300 of the user X based on the position and orientation of the virtual camera 300 of the user X (Step S16). In step S <b> 17, the control unit 121 determines whether the teammate avatar object FC is located within the field of view CV of the user X virtual camera 300.

  When it is determined that the teammate avatar object FC is located within the field of view CV (an example of the first region) of the virtual camera 300 (YES in step S17), the control unit 121 is a unit of sound that propagates in the virtual space 200. The attenuation coefficient that defines the attenuation per distance is set to the attenuation coefficient α1 (an example of the first attenuation coefficient), and the audio is based on the distance D between the virtual camera 300 and the ally avatar object FC and the attenuation coefficient α1. Data is processed (step S18). Here, when the teammate avatar object FC is located in the visual field CV, the teammate avatar object FC is displayed with a solid line as shown in FIG.

  On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV of the virtual camera 300 (an example of the second region) (NO in step S17), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α2 (first 2 is an example of the attenuation coefficient, and the audio data is processed based on the distance D between the virtual camera 300 and the teammate avatar object FC and the attenuation coefficient α2 (step S19). Here, when the teammate avatar object FC is located outside the field of view CV, the teammate avatar object FC is displayed with a broken line as shown in FIG. Here, the attenuation coefficient α1 and the attenuation coefficient α2 are different, and α1 <α2.

  Next, in step S20, the control unit 121 causes the headphones 116 of the HMD system 1B to output sound corresponding to the processed sound data.

  In the present embodiment, the virtual camera 300 and the sound source object MC are integrally configured, and the ally avatar object FC has a sound collection function, so the distance between the virtual camera 300 of the user X and the ally avatar object FC. When D is large, the volume (sound pressure level) of the sound output to the headphones 116 of the HMD system 1B is small (in other words, the sound attenuation (dB) is large). On the other hand, when the distance D between the virtual camera 300 of the user X and the teammate avatar object FC is small, the sound volume (sound pressure level) output to the headphones 116 of the HMD system 1B increases (that is, the sound). (Attenuation amount (dB) is small).

  Furthermore, when the attenuation coefficient is large, the volume (sound pressure level) of the sound output to the headphones 116 of the HMD system 1B is small (in other words, the sound attenuation (dB) is large). On the contrary, when the attenuation coefficient is small, the volume (sound pressure level) of the sound output to the headphones 116 of the HMD system 1B increases (in other words, the sound attenuation (dB) is small). Thus, the control unit 121 determines the volume (sound pressure level) of the audio data based on the distance D between the virtual camera 300 of the user X and the teammate avatar object FC and the attenuation coefficient.

In this respect, the control unit 121 refers to a function indicating the relationship between the distance D between the virtual camera 300 of the user X and the teammate avatar object FC, the attenuation coefficient α, and the volume L of the audio data. The volume of the audio data may be determined. For example, when the volume at the reference distance D ₀ is known, the control unit 121 may determine the volume L of the audio data with reference to the following formula (1), for example. Expression (1) is merely an example, and the volume L of the audio data may be determined using another expression.

L = L ₀ −20 log (D / D ₀ ) −8.7α (D−D ₀ ) (1)

D: Distance D ₀ between the virtual camera 300 of the user X and the ally avatar object FC ₀ : Reference distance L between the virtual camera 300 of the user X and the ally avatar object FC: Volume of sound data (dB) at the distance D
L ₀ : Volume of sound data at the reference distance D ₀ (dB)
α: Damping coefficient (dB / distance)

  When the teammate avatar object FC is present within the field of view CV, the attenuation coefficient α is the attenuation coefficient α1, while when the teammate avatar object FC is present outside the field of view CV, the attenuation coefficient α is the attenuation coefficient α2. Further, the attenuation coefficient α1 is smaller than the attenuation coefficient α2. Therefore, the sound volume of the audio data when the friend avatar object FC is in the field of view CV is the distance D1 between the virtual camera 300 of the user X and the friend avatar object FC and the virtual camera 300 of the user X. It is the distance D1 between the teammate avatar object FC and is larger than the volume of the audio data when the teammate avatar object FC exists outside the field of view CV. That is, since different attenuation coefficients α1 and α2 are set inside and outside the visual field CV, the sound output from the sound source object MC (virtual camera 300) can be given directivity in the virtual space 200. .

  Furthermore, the control unit 121 determines a predetermined head-related transfer function from the relative positional relationship between the virtual camera 300 of the user X and the teammate avatar object FC, and based on the determined head-related transfer function and audio data Audio data may be processed.

  According to the present embodiment, when it is determined that the teammate avatar object FC is located in the field of view CV, the attenuation coefficient α is set to the attenuation coefficient α1, and then the audio data includes the distance D and the attenuation coefficient α1. It is processed based on. On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV, the attenuation coefficient α is set to the attenuation coefficient α2, and then the audio data is processed based on the distance D and the attenuation coefficient α2. Is done. As described above, the volume (sound pressure level) of the sound output from the headphones 116 differs depending on the position of the teammate avatar object FC on the virtual space 200. In this embodiment, since α1 <α2, as shown in FIG. 10, when the teammate avatar object FC exists in the field of view CV and the enemy avatar object EC exists outside the field of view CV, the teammate avatar object FC is displayed. The volume of the sound output from the headphones 116 attached to the user Y who operates is larger than the volume of the sound output from the headphones 116 attached to the user Z who operates the enemy avatar object EC. Therefore, the user X can give a voice instruction to the user Y who operates the teammate avatar object FC without being noticed by the user Z who operates the enemy avatar object EC. Therefore, the entertainment property of the virtual space 200 can be improved.

(First modification of the first embodiment)
Next, a first modification of the first embodiment will be described with reference to FIG. FIG. 11 shows that when the self-avatar object 400 and the sound source object MC are integrally formed, the self-avatar object 400 and the ally avatar object FC that are located within the visual field CV of the virtual camera 300 and the visual field CV of the virtual camera 300 are outside. It is a figure which shows the enemy avatar object EC located.

  First, as shown in FIG. 11, the virtual space 200A includes a virtual camera 300, a sound source object MC, a self-avatar object 400, a teammate avatar object FC, and an enemy avatar object EC. The control unit 121 generates virtual space data that defines the virtual space 200 including these objects. Here, the self-avatar object 400 is an avatar object (that is, an avatar object associated with the user X) controlled by the operation of the user X.

  A virtual space 200 </ b> A illustrated in FIG. 11 is different from the virtual space 200 illustrated in FIG. 10 in that the self-avatar object 400 is arranged and the sound source object MC is configured integrally with the self-avatar object 400. Thus, in the virtual space 200 shown in FIG. 10, the viewpoint of the virtual space provided to the user is the first person viewpoint, while in the virtual space 200A shown in FIG. 11, the viewpoint of the virtual space provided to the user is the third person viewpoint. It is. When the self-avatar object 400 and the sound source object MC are integrally configured as in the first modification, the self-avatar object 400 may be interpreted as having a sound source function.

  Next, an information processing method according to a first modification of the first embodiment will be described with reference to FIG. Hereinafter, only differences between the already described information processing method according to the first embodiment and the information processing method according to the first modification will be described in detail. In the information processing method according to the first modification, steps S10 to S13 shown in FIG. 9 are executed. In step S <b> 14, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1 </ b> B positions the user Y avatar object (friend avatar object FC) and the user X avatar object (self-avatar object 400). Identify each. Here, the HMD system 1A transmits the position information of the self-avatar object 400 to the game server 2 at a predetermined time interval, while the game server 2 transmits the position information of the self-avatar object 400 to the HMD system 1B at a predetermined time interval. You may send it with

  Next, in step S15, the control unit 121 specifies a distance Da (an example of a relative positional relationship) between the self-avatar object 400 (sound source object MC) and the teammate avatar object FC. The distance Da may be the shortest distance between the self-avatar object 400 and the teammate avatar object FC. In the first modification, since the self-avatar object 400 and the sound source object MC are integrally configured, the distance Da between the self-avatar object 400 and the teammate avatar object FC is equal to the sound source object MC and the teammate avatar object. Corresponds to the distance to FC.

  Then, the control part 121 performs the determination process prescribed | regulated by step S17, and when it determines with the ally avatar object FC being located in the visual field CV of the virtual camera 300 (it is YES at step S17), an attenuation coefficient is set. After setting the attenuation coefficient α1, the audio data is processed based on the distance Da between the self-avatar object 400 and the teammate avatar object FC and the attenuation coefficient α1 (step S18).

  On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV of the virtual camera 300 (NO in step S17), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α2 and then sets the self-avatar object. The voice data is processed based on the distance Da between 400 and the teammate avatar object FC and the attenuation coefficient α2 (step S19).

  Next, in step S20, the control unit 121 causes the headphones 116 of the HMD system 1B to output sound corresponding to the processed sound data.

(Second modification of the first embodiment)
Next, an information processing method according to a second modification of the first embodiment will be described with reference to FIG. The information processing method according to the second modification is different from the information processing method according to the first embodiment in that audio data is processed by the HMD system 1A. FIG. 12 is a flowchart for explaining the information processing method according to the second modification of the first embodiment.

  As shown in FIG. 12, in step S30, the microphone 118 of the HMD system 1A collects voices uttered from the user X, and generates voice data indicating the collected voices. Next, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1A specifies the field of view CV of the virtual camera 300 of the user X based on the position and orientation of the virtual camera 300 of the user X (step) S31). Thereafter, the control unit 121 specifies the position of the avatar object (friend avatar object FC) of the user Y (step S32). Here, while the HMD system 1B transmits the position information of the ally avatar object FC to the game server 2 at a predetermined time interval, the game server 2 transmits the position information of the ally avatar object FC to the HMD system 1B at a predetermined time interval. You may send it with

  In step S33, the control unit 121 specifies the distance D between the virtual camera 300 (sound source object MC) of the user X and the teammate avatar object FC. Next, the control unit 121 determines whether or not the teammate avatar object FC is located within the field of view CV of the virtual camera 300 of the user X. When it is determined that the teammate avatar object FC is located in the field of view CV of the virtual camera 300 (YES in step S34), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α1, and then the virtual camera 300 and the friend The audio data is processed based on the distance D between the avatar object FC and the attenuation coefficient α1 (step S35). On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV of the virtual camera 300 (NO in step S34), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α2 and then sets the virtual camera 300. The voice data is processed on the basis of the distance D and the attenuation coefficient α2 between the user and the teammate avatar object FC (step S36).

  Thereafter, the control unit 121 transmits the processed audio data to the game server 2 via the communication network 3 (step S37). After receiving the processed voice data from the HMD system 1A, the game server 2 transmits the processed voice data to the HMD system 1B (step S38). Next, the control unit 121 of the HMD system 1B receives the sound data processed from the game server 2, and then outputs the sound corresponding to the processed sound data to the headphones 116 of the HMD system 1B (step S39). ).

(Third Modification of First Embodiment)
Next, an information processing method according to a third modification of the first embodiment will be described with reference to FIGS. 13 and 14. FIG. 13 shows the field of view of the virtual camera 300 other than the ally avatar object FC and the gaze area R1 located in the gaze area R1 (an example of the first area) when the virtual camera 300 and the sound source object MC are integrally configured. It is a figure showing enemy avatar object EC located in CV (an example of the 2nd field). FIG. 14 is a flowchart for explaining an information processing method according to the third modification of the first embodiment.

  In the information processing method according to the first embodiment, when the teammate avatar object FC is located in the visual field CV of the virtual camera 300, the attenuation coefficient is set to the attenuation coefficient α1. On the other hand, when the teammate avatar object FC is located outside the field of view CV of the virtual camera 300, the attenuation coefficient is set to the attenuation coefficient α2.

  On the other hand, in the information processing method according to the third modification, as shown in FIG. 13, the teammate avatar object FC is located in the gaze area R1 (an example of the first area) defined by the line-of-sight direction S of the user X. The attenuation coefficient is set to the attenuation coefficient α3 (an example of the first attenuation coefficient). On the other hand, when the teammate avatar object FC is located in the field of view CV of the virtual camera 300 other than the gaze area R1 (an example of the second area), the attenuation coefficient is set to the attenuation coefficient α1 (an example of the second attenuation coefficient). Is done. Further, when the teammate avatar object FC is located outside the field of view CV of the virtual camera 300, the attenuation coefficient may be set to the attenuation coefficient α2. Here, the attenuation coefficient α1, the attenuation coefficient α2, and the attenuation coefficient α3 are different from each other, for example, α3 <α1 <α2. As described above, the information processing method according to the third modification is different from the information processing method according to the first embodiment in that two different attenuation coefficients α3 and α1 are set in the visual field CV of the virtual camera 300. .

  Further, the control unit 121 of the HMD system 1A specifies the line-of-sight direction S of the user X based on the data indicating the line-of-sight direction S of the user X transmitted from the gaze sensor 140 of the HMD system 1A. The gaze region R1 is set as an angle range of a predetermined polar angle centered on the line-of-sight direction S on the xy plane and an angle range of a predetermined azimuth angle centered on the line-of-sight direction S on the xz plane. Second region. The predetermined polar angle and the predetermined azimuth angle may be set as appropriate according to the specifications of the game program.

  Next, an information processing method according to a third modification will be described below with reference to FIG. In the following, only the difference between the information processing method according to the first embodiment described above and the information processing method according to the third modification will be described in detail.

  After the process of step S40, the HMD system 1A transmits information regarding the position and orientation of the virtual camera 300, information regarding the line-of-sight direction S, and audio data to the game server 2 via the communication network 3 (step S41). . The game server 2 receives the information regarding the position and orientation of the virtual camera 300 of the user X, the information regarding the line-of-sight direction S, and the audio data from the HMD system 1A, and then transmits the information and the audio data to the HMD system 1B. (Step S42). Thereafter, the control unit 121 of the HMD system 1B receives information regarding the position and orientation of the virtual camera 300 of the user X, information regarding the line-of-sight direction S, and audio data via the communication network 3 and the communication interface 125 ( Step S43).

  Next, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1B specifies the gaze region R1 based on the information regarding the line-of-sight direction S of the user X after executing the processing of steps S44 to S46. (Step S47). Next, the control unit 121 determines whether or not the teammate avatar object FC is located in the gaze region R1 (step S48). When it is determined that the teammate avatar object FC is located in the gaze region R1 (YES in step S48), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α3, and then the virtual camera 300 of the user X and the friend The audio data is processed based on the distance D between the avatar object FC and the attenuation coefficient α3 (step S49).

  On the other hand, when it is determined that the teammate avatar object FC is located outside the gaze area R1 (NO in step S48), the control unit 121 determines whether the teammate avatar object FC is located within the field of view CV ( Step S50). When the control unit 121 determines that the teammate avatar object FC is located within the field of view CV (YES in step S50), the controller 121 sets the attenuation coefficient to the attenuation coefficient α1, and then the virtual camera 300 of the user X and the teammate avatar. The audio data is processed based on the distance D between the object FC and the attenuation coefficient α1 (step S51). On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV (NO in step S50), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α2, and then the virtual camera 300 of the user X The audio data is processed based on the distance D between the ally avatar object FC and the attenuation coefficient α2 (step S52).

  Next, in step S53, the control unit 121 causes the audio corresponding to the processed audio data to be output to the headphones 116 of the HMD system 1B.

  According to the third modification, when it is determined that the teammate avatar object FC is located in the gaze region R1, the attenuation coefficient α is set to the attenuation coefficient α3, and then the audio data is the distance D and the attenuation. Processing is performed based on the coefficient α3. On the other hand, when it is determined that the teammate avatar object FC is located in the field of view CV of the virtual camera 300 other than the gaze region R1, the attenuation coefficient α is set to the attenuation coefficient α1, and then the audio data is the distance D And the damping coefficient α1.

  As described above, the volume (sound pressure level) of the sound output from the headphones 116 differs depending on the position of the teammate avatar object FC on the virtual space 200. In this modified example, α3 <α1, and therefore, as shown in FIG. 13, the teammate avatar object FC exists in the gaze area R1, and the enemy avatar object EC exists in the field of view CV other than the gaze area R1. The sound volume output from the headphones 116 attached to the user Y who operates the teammate avatar object FC is larger than the sound volume output from the headphones 116 attached to the user Z who operates the enemy avatar object EC. . Therefore, the user X can give a voice instruction to the user Y who operates the teammate avatar object FC without being noticed by the user Z who operates the enemy avatar object EC. Therefore, the entertainment property of the virtual space 200 can be improved.

(Fourth modification of the first embodiment)
Next, an information processing method according to a fourth modification of the first embodiment will be described with reference to FIGS. 15 and 16. FIG. 15 illustrates a virtual camera 300 other than the teammate avatar object FC and the visual axis region R2 positioned in the visual axis region R2 (an example of the first region) when the virtual camera 300 and the sound source object MC are integrally configured. It is a figure which shows enemy avatar object EC located in the visual field CV (an example of 2nd area | region). FIG. 16 is a flowchart for explaining an information processing method according to a fourth modification of the first embodiment.

  In the information processing method according to the first embodiment, when the teammate avatar object FC is located in the visual field CV of the virtual camera 300, the attenuation coefficient is set to the attenuation coefficient α1. On the other hand, when the teammate avatar object FC is located outside the field of view CV of the virtual camera 300, the attenuation coefficient is set to the attenuation coefficient α2.

  On the other hand, in the information processing method according to the fourth modified example, the teammate avatar object FC is positioned in the visual axis region R2 (an example of the first region) defined by the visual axis L of the virtual camera 300, as shown in FIG. In this case, the attenuation coefficient is set to the attenuation coefficient α3 (an example of the first attenuation coefficient). On the other hand, when the teammate avatar object FC is located within the field of view CV of the virtual camera 300 other than the visual axis region R2 (an example of the second region), the attenuation coefficient becomes the attenuation coefficient α1 (an example of the second attenuation coefficient). Is set. Further, when the teammate avatar object FC is located outside the field of view CV of the virtual camera 300, the attenuation coefficient may be set to the attenuation coefficient α2. Here, the attenuation coefficient α1, the attenuation coefficient α2, and the attenuation coefficient α3 are different from each other, for example, α3 <α1 <α2. As described above, in the information processing method according to the fourth modification, two different attenuation coefficients α3 and α1 are set in the field of view CV of the virtual camera 300, as in the information processing method according to the third modification. This is different from the information processing method according to the first embodiment.

  Further, the control unit 121 of the HMD system 1A specifies the visual axis L of the virtual camera 300 based on the position and orientation of the virtual camera 300. The visual axis region R2 is a first region set as a predetermined polar angle range centered on the visual line direction S on the xy plane, and an angular range of a predetermined azimuth angle centered on the visual line direction S on the xz plane. And a second region to be set. The predetermined polar angle and the predetermined azimuth angle may be set as appropriate according to the specifications of the game program. The predetermined polar angle is smaller than the polar angle α for specifying the visual field CV of the virtual camera 300, and the predetermined azimuth angle is smaller than the azimuth angle β for specifying the visual field CV of the virtual camera 300. .

  Next, an information processing method according to a fourth modification will be described below with reference to FIG. In the following, only the difference between the information processing method according to the first embodiment already described and the information processing method according to the fourth modification will be described in detail.

  Since the process of steps S60 to S66 corresponds to the process of steps S10 to S16 shown in FIG. 9, the description of these processes is omitted. In step S67, the control unit 121 specifies the visual axis region R2 based on the visual axis L of the virtual camera 300 (step S67). Next, the control unit 121 determines whether or not the teammate avatar object FC is located in the visual axis region R2 (step S68). When it is determined that the teammate avatar object FC is located in the visual axis region R2 (YES in step S68), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α3, and the virtual camera 300 of the user X The voice data is processed based on the distance D between the avatar object FC and the attenuation coefficient α3 (step S69).

  On the other hand, when it is determined that the teammate avatar object FC is located outside the visual axis region R2 (NO in step S68), the control unit 121 determines whether the teammate avatar object FC is located within the field of view CV. (Step S70). When the control unit 121 determines that the teammate avatar object FC is located within the field of view CV (YES in step S70), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α1, and then the virtual camera 300 of the user X and the teammate avatar. The audio data is processed based on the distance D between the object FC and the attenuation coefficient α1 (step S71). On the other hand, if the control unit 121 determines that the teammate avatar object FC is located outside the field of view CV (NO in step S70), the control unit 121 sets the attenuation coefficient to the attenuation coefficient α2, and the virtual camera 300 of the user X The voice data is processed based on the distance D between the ally avatar object FC and the attenuation coefficient α2 (step S72).

  Next, in step S73, the control unit 121 causes the audio corresponding to the processed audio data to be output to the headphones 116 of the HMD system 1B.

  According to the fourth modification, when it is determined that the teammate avatar object FC is located in the visual axis region R2, the attenuation coefficient α is set to the attenuation coefficient α3, and then the audio data is the distance D and Processing is performed based on the attenuation coefficient α3. On the other hand, when it is determined that the teammate avatar object FC is located in the visual field CV of the virtual camera 300 other than the visual axis region R2, the attenuation coefficient α is set to the attenuation coefficient α1, and then the audio data is the distance Processing is performed based on D and the attenuation coefficient α2.

  As described above, the volume (sound pressure level) of the sound output from the headphones 116 differs depending on the position of the teammate avatar object FC on the virtual space 200. In this modified example, α3 <α1, so that the teammate avatar object FC exists in the visual axis region R2 and the enemy avatar object EC exists in the field of view CV other than the visual axis region R2, as shown in FIG. In this case, the sound volume output from the headphones 116 attached to the user Y who operates the friend avatar object FC is higher than the sound volume output from the headphones 116 attached to the user Z who operates the enemy avatar object EC. Is also big. Therefore, the user X can give a voice instruction to the user Y who operates the teammate avatar object FC without being noticed by the user Z who operates the enemy avatar object EC. Therefore, the entertainment property of the virtual space 200 can be improved.

(Second Embodiment)
Next, an information processing method according to the second embodiment will be described with reference to FIGS. 17 and 18. FIG. 17 shows a friend avatar object FC and a self avatar object 400 that are located inside the attenuation object SA (within the field of view CV of the virtual camera 300) when the self avatar object 400 and the sound source object MC are integrally configured. It is a figure which shows enemy avatar object EC located in the outer side (outside the visual field CV of virtual camera 300) of attenuation object SA. FIG. 18 is a flowchart for explaining an information processing method according to the second embodiment.

  First, as shown in FIG. 17, the virtual space 200B includes a virtual camera 300, a sound source object MC, a self-avatar object 400, a friend avatar object FC, an enemy avatar object EC, and an attenuation object SA. The control unit 121 generates virtual space data that defines the virtual space 200 including these objects. The information processing method according to the second embodiment is different from the information processing method according to the first embodiment in that the attenuation object SA is arranged.

  The attenuation object SA is an object that defines an attenuation amount of sound propagating in the virtual space 200B, and is within the field of view CV (an example of the first area) and outside the field of view CV (an example of the second area) of the virtual camera 300. It is arranged at the boundary. Further, the attenuation object SA is transparent, and may not be displayed in the visual field image V (see FIG. 8) displayed on the HMD 110. In this case, the sound output from the sound source object MC can be given directivity without impairing the user's immersion in the virtual space 200B (a sense of presence in the virtual space 200B).

  In the virtual space 200B shown in FIG. 17, the sound source object MC is integrally formed with the self-avatar object 400, and these objects are arranged in the field of view CV of the virtual camera 300.

  Next, an information processing method according to the second embodiment will be described with reference to FIG. The processes in steps S80 to S83 shown in FIG. 18 correspond to the processes in S10 to S13 shown in FIG. In step S84, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1B determines the position of the user Y avatar object (ally avatar object FC) and the position of the user X avatar object (self-avatar object 400). Identify each.

  Next, in step S85, the control unit 121 specifies a distance Da (an example of a relative positional relationship) between the self-avatar object 400 (sound source object MC) and the teammate avatar object FC. The distance Da may be the shortest distance between the self-avatar object 400 and the teammate avatar object FC. The distance Da between the self-avatar object 400 and the teammate avatar object FC corresponds to the distance between the sound source object MC and the teammate avatar object FC. Next, the control part 121 performs the process of step S86 and S87. Here, the processes of steps S86 and S87 correspond to the processes of steps S16 and S17 shown in FIG. 9, respectively.

  Thereafter, when the control unit 121 determines that the teammate avatar object FC is located outside the field of view CV of the virtual camera 300 (NO in step S87), the distance Da between the self-avatar object 400 and the teammate avatar object FC is determined. The audio data is processed based on the attenuation amount T defined by the attenuation object SA. In this case, as shown in FIG. 17, the sound source object MC is located inside the attenuation object SA, and the teammate avatar object FC (shown by a broken line) is located outside the attenuation object SA. For this reason, since the sound traveling from the sound source object MC to the ally avatar object FC passes through the attenuation object SA, the sound volume (sound pressure level) of the sound is determined by the distance Da and the attenuation amount T defined by the attenuation object SA. The

  On the other hand, when the control unit 121 determines that the teammate avatar object FC is located within the field of view CV of the virtual camera 300 (YES in step S87), the control unit 121 sets the distance Da between the self-avatar object 400 and the teammate avatar object FC. Based on this, the voice data is processed. In this case, as shown in FIG. 17, the sound source object MC and the teammate avatar object FC (displayed with a solid line) are located inside the attenuation object SA. For this reason, since the sound heading from the sound source object MC to the ally avatar object FC does not pass through the attenuation object SA, the sound volume (sound pressure level) of the sound is determined by the distance Da.

  Next, in step S90, the control unit 121 causes the audio corresponding to the processed audio data to be output to the headphones 116 of the HMD system 1B.

  According to the present embodiment, when the sound source object MC is arranged in the field of view CV of the virtual camera 300 (inside the attenuation object SA), the sound output from the sound source object MC is outside the field of view CV of the virtual camera 300. Attenuation amount T is further attenuated by the attenuation amount T defined by the attenuation object SA as compared to the view CV. In this way, the sound output from the sound source object MC can be given directivity.

  Furthermore, when it is determined that the teammate avatar object FC is located within the visual field CV (first region) where the sound source object MC is located, the audio data is processed based on the distance Da. On the other hand, when it is determined that the teammate avatar object FC is located outside the field of view CV (second region), the audio data is processed based on the attenuation amount T defined by the distance Da and the attenuation object SA.

  As described above, the volume (sound pressure level) of the sound output from the headphones 116 varies depending on the position of the teammate avatar object FC on the virtual space 200B. The volume of the sound output from the headphones 116 when the ally avatar object FC exists in the field of view CV is larger than the volume of the sound output from the headphones 116 when the ally avatar object FC exists outside the field of view CV. . Therefore, when the teammate avatar object FC exists in the field of view CV and the enemy avatar object EC exists outside the field of view CV, the user X who operates the self-avatar object 400 becomes the user Z who operates the enemy avatar object EC. It becomes possible to give an instruction by voice to the user Y who operates the teammate avatar object FC without being noticed. Therefore, the entertainment property of the virtual space 200B can be improved.

(First Modification of Second Embodiment)
Next, an information processing method according to a first modification of the second embodiment will be described with reference to FIG. The information processing method according to the second modification is different from the information processing method according to the first embodiment in that audio data is processed by the HMD system 1A. FIG. 19 is a flowchart for explaining an information processing method according to a first modification of the second embodiment.

  As shown in FIG. 19, in step S <b> 100, the microphone 118 of the HMD system 1 </ b> A collects sound emitted from the user X, and generates sound data indicating the collected sound. Next, the control unit 121 (hereinafter simply referred to as the control unit 121) of the HMD system 1A specifies the field of view CV of the virtual camera 300 of the user X based on the position and orientation of the virtual camera 300 of the user X (step) S101). Thereafter, the control unit 121 identifies the position of the user Y avatar object (ally avatar object FC) and the position of the user X avatar object (self-avatar object 400) (step S102).

  In step S103, the control unit 121 specifies a distance Da between the self-avatar object 400 and the teammate avatar object FC. Next, the control unit 121 determines whether or not the teammate avatar object FC is located within the field of view CV of the virtual camera 300 of the user X (step S104). When determining that the teammate avatar object FC is located outside the field of view CV of the virtual camera 300 (NO in step S104), the control unit 121 determines that the distance Da between the self-avatar object 400 and the teammate avatar object FC and the attenuation object The audio data is processed based on the attenuation amount T defined by SA. On the other hand, when the control unit 121 determines that the teammate avatar object FC is located within the field of view CV of the virtual camera 300 (YES in step S104), the control unit 121 sets the distance Da between the self-avatar object 400 and the teammate avatar object FC. Based on this, the voice data is processed (step S106).

  Thereafter, the control unit 121 transmits the processed audio data to the game server 2 via the communication network 3 (step S107). After receiving the processed audio data from the HMD system 1A, the game server 2 transmits the processed audio data to the HMD system 1B (step S108). Next, the control unit 121 of the HMD system 1B receives the sound data processed from the game server 2, and then outputs the sound corresponding to the processed sound data to the headphones 116 of the HMD system 1B (step S109). ).

(Second Modification of Second Embodiment)
Next, an information processing method according to a second modification of the second embodiment will be described with reference to FIG. FIG. 20 is a diagram illustrating a teammate avatar object FC positioned inside the attenuation object SB and an enemy avatar object EC positioned outside the attenuation object SB when the virtual camera 300 and the sound source object MC are integrally configured. It is. As shown in FIG. 20, in the virtual space 200C, the attenuation object SB is arranged so as to surround the virtual camera 300 (the sound source object MC). When the teammate avatar object FC (solid line display) is arranged in the inner region R3 of the attenuation object SB, the audio data is processed based on the distance D between the virtual camera 300 and the teammate avatar object FC. On the other hand, when the teammate avatar object FC (indicated by a broken line) is arranged outside the attenuation object SB, the audio data is defined by the distance D between the virtual camera 300 and the teammate avatar object FC and the attenuation object SB. Processing is performed based on the attenuation amount T.

  In order to implement various processes executed by the control unit 121 by software, a display control program for causing a computer (processor) to execute the display control method according to the present embodiment is incorporated in the storage unit 123 or the ROM in advance. Also good. Alternatively, the display control program can be a magnetic disk (HDD, floppy disk), optical disk (CD-ROM, DVD-ROM, Blu-ray disk, etc.), magneto-optical disk (MO, etc.), flash memory (SD card, USB memory, It may be stored in a computer-readable storage medium such as SSD. In this case, the program stored in the storage medium is incorporated into the storage unit 123 by connecting the storage medium to the control device 120. Then, the display control program incorporated in the storage unit 123 is loaded on the RAM, and the control unit 121 executes the display control method according to the present embodiment by executing the loaded program.

  The display control program may be downloaded from a computer on the communication network 3 via the communication interface 125. Similarly in this case, the downloaded program is incorporated in the storage unit 123.

  As mentioned above, although embodiment of this indication was described, the technical scope of this invention should not be limitedly interpreted by description of this embodiment. This embodiment is an example, and it is understood by those skilled in the art that various modifications can be made within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the equivalents thereof.

1, 1A, 1B, 1C: HMD system 2: Game server 3: Communication network 21: Central position 100: Game system 112: Display unit 114: Sensor 116: Headphone 118: Microphone 120: Control device 121: Control unit 123: Storage Department (Storage)
123: storage unit 124: I / O interface 125: communication interface 126: bus 130: position sensor 140: gaze sensor 200: virtual space 300: virtual camera 320: external controller 400: self avatar object CV: visual field EC: enemy avatar object FC: ally avatar object MC: sound source object R1: gaze area R2: visual axis area R3: inner area S: line-of-sight direction SA, SB: attenuation object

Claims (8)

An information processing method in a system comprising a first user terminal having a first head mounted display, a voice input unit, and a second user terminal ,
Generating virtual space data defining a virtual space including a virtual camera, a sound source object defined as a sound source of sound input to the sound input unit, and an avatar object associated with the second user terminal ; ,
Here, the avatar object functions to collect sound propagating in the virtual space,
Determining a field of view of the virtual camera in response to movement of the first head mounted display;
Generating visual field image data based on the visual field of the virtual camera and the virtual space data;
Displaying a field image on the first head mounted display based on the field image data;
Obtaining voice data indicating voice input to the voice input unit;
Identifying a relative positional relationship between the sound source object and the avatar object ;
Processing the audio data based on the identified relative positional relationship and an attenuation coefficient defining an attenuation amount per unit distance of the audio propagating in the virtual space,
When it is determined that the avatar object is located in the first region of the virtual space, the attenuation coefficient is set to a first attenuation coefficient, and then the audio data is stored in the relative positional relationship and the first attenuation. Processed based on constants,
If it is determined that the avatar object is located in a second area of the virtual space different from the first area, the attenuation coefficient is set to a second attenuation coefficient different from the first attenuation constant; The audio data is processed based on the relative positional relationship and the second attenuation coefficient,
The information processing method, wherein at least one of the first area and the second area is determined based on a field of view of the virtual camera. The first region is within the field of view of the virtual camera;
The second region is outside the field of view of the virtual camera;
The information processing method according to claim 1. The first area is a gaze area defined by the line-of-sight direction of the user wearing the first head mounted display,
The second area is within the field of view of the virtual camera other than the gaze area.
The information processing method according to claim 1. The first region is a visual axis region defined by a visual axis of the virtual camera,
The second region is within the field of view of the virtual camera other than the visual axis region.
The information processing method according to claim 1. An information processing method in a system comprising a first user terminal having a first head mounted display, a voice input unit, and a second user terminal ,
Virtual space data defining a virtual space including a virtual camera, a sound source object defined as a sound source of sound input to the sound input unit, an avatar object associated with the second user terminal, and an attenuation object Generating, wherein the avatar object functions to collect sound propagating in the virtual space, and the attenuation object defines an attenuation amount of sound propagating in the virtual space;
Determining a field of view of the virtual camera in response to movement of the first head mounted display;
Generating visual field image data based on the visual field of the virtual camera and the virtual space data;
Displaying a field image on the first head mounted display based on the field image data;
Obtaining voice data indicating voice input to the voice input unit;
Identifying a relative positional relationship between the sound source object and the avatar object ;
Determining whether the avatar object is located inside the attenuation object;
Processing the audio data;
Including
When it is determined that the avatar object is located inside the attenuation object, the audio data is processed based on the relative positional relationship,
When it is determined that the avatar object is located outside the attenuation object, the audio data is processed based on the relative positional relationship and the attenuation amount defined by the attenuation object,
The sound source object is disposed inside the attenuation object;
Information processing method.   The information processing method according to claim 5, wherein the attenuation object is transparent and is not displayed in the visual field image.   The information processing method according to claim 5, wherein the attenuation object is arranged at a boundary between the field of view of the virtual camera and the field of view of the virtual camera.   The program for making a computer perform the information processing method as described in any one of Claims 1-7.