JP2025091817A - Program and audio control device - Google Patents

Abstract

To improve a user experience.SOLUTION: In a virtual space, a first area 51 and a second area 52 that are adjacent mutually are provided. An acoustic effect added to a voice in the virtual space is correlated with each of the first area 51 and the second area 52. A boundary sound source 65 that is a virtual sound source is disposed in a boundary between the first area 51 and the second area 52. An acoustic processing section adds an acoustic effect correlated with an area positioned on a side close to a virtual microphone 62 in a boundary sound source 65 in the first area 51 and the second area 52 to a boundary voice that is a voice of the boundary sound source 65 obtained by the virtual microphone 62 disposed in the virtual space. An acoustic output section allows a speaker to output the boundary voice to which an acoustic effect is added by the acoustic processing section.SELECTED DRAWING: Figure 2

Description

This disclosure relates to a program and an audio control device.

Some game programs play sounds within a virtual space. For example, in Patent Document 1, sounds such as gunshots are played in a shooting game.

JP 2014-094160 A

When various services (e.g. games) are provided in a virtual space, it is desirable to play audio appropriate to the situation from the perspective of user experience.

The purpose of this disclosure is to improve the user experience in services provided in virtual space.

A first aspect is a program for causing a computer system having one computer or a plurality of computers working together to function as an audio processing unit and an audio output unit,
The computer system is used together with a speaker for outputting sound within a virtual space;
The virtual space includes a first area and a second area adjacent to each other,
a sound effect to be added to a sound in the virtual space is associated with each of the first area and the second area;
A boundary sound source that is a virtual sound source is disposed at a boundary between the first area and the second area;
the sound processing unit adds a sound effect associated with an area of the first area and the second area that is located closer to the virtual microphone in the boundary sound source to a boundary sound that is a sound of the boundary sound source obtained by a virtual microphone arranged in the virtual space,
The sound output unit is a program that causes the speaker to output the boundary sound to which the sound effect has been added by the sound processing unit.

In a first aspect,
A determination area may be set in the virtual space, the determination area being a predetermined range based on the boundary sound source, the determination area having a first area portion belonging to the first area and a second area portion belonging to the second area,
The acoustic processing unit may add an acoustic effect associated with the first area to the boundary sound when the first area of the first area and the second area is closer to the virtual microphone, and may add an acoustic effect associated with the second area to the boundary sound when the second area of the first area and the second area is closer to the virtual microphone.

In a first aspect,
The computer system may function as a display processing unit,
The computer system may be utilized with a display for displaying an image of the virtual space;
The display processing unit may cause an image of the virtual space acquired by a virtual camera disposed in the virtual space to be displayed on the display,
The acoustic processing unit may move the virtual microphone in the virtual space so as to follow movement of the virtual camera in the virtual space.

The second aspect is
A storage unit that stores the program of the first aspect;
and a control unit for executing the program.

This disclosure can improve the user experience.

1 is a block diagram illustrating a configuration of a game device according to an embodiment. 1 is a schematic diagram illustrating a first area, a second area, a boundary sound source, and a determination region provided in a virtual space. FIG.

The following describes the embodiment in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are given the same reference numerals and their description will not be repeated. In addition, the following describes an example in which the program is implemented as a game program and the sound control device is realized as a game device. The game device is an example of a computer system having one computer or multiple computers working together.

(Game Description)
In the following description, a game refers to a game (video game) that is played in a virtual space. The virtual space is a three-dimensional game space, and is displayed as an image on a display. Objects are placed in the virtual space. Examples of objects include a player character operated by a player (a user who plays the game) and a non-player character operated by a computer. A player is an example of a user.

The objects also include sound source objects that act as virtual sound sources in the virtual space. Examples of sound source objects include objects that operate in the virtual space and objects that only output sound in the virtual space. Player characters and non-player characters can be sound source objects. In the games described below, sounds are played. Sounds played in the game include sounds output from sound source objects, background music (BGM), sound effects, and the like.

For ease of explanation, the following will use as an example of a game a "battle action game in which a player character attacks and defeats an enemy character." The enemy character is an example of a non-player character. The player character and the enemy character are examples of a sound source object. There are no limitations on the type of game.

(Gaming device)
1 illustrates an example of a configuration of a game device 10 according to an embodiment. The game device 10 executes a game in response to operations by a user. A display 21, a speaker 22, and a controller 23 are externally connected to or built into the game device 10. The game device 10 is an example of an acoustic control device.

The game device 10 can be, for example, a personal computer, PlayStation (registered trademark), XBox (registered trademark), PlayStation Vita (registered trademark), Nintendo Switch (registered trademark), or other commercially available device.

The game progresses on the game device 10 based on the installed game program and game data. Game devices 10 can communicate data with each other using a communication network (not shown) or a short-range wireless communication device (not shown).

[Hardware configuration of the game device]
The game device 10 includes a network interface 11, a graphics processing unit 12, an audio processing unit 13, an operation processing unit 14, a storage unit 15, and a control unit 16. The network interface 11, the graphics processing unit 12, the audio processing unit 13, the operation processing unit 14, and the storage unit 15 are electrically connected to the control unit 16 via a bus 17.

The network interface 11 is communicatively connected to a communication network (not shown) to transmit and receive various data, for example, between other game devices 10 and external server devices (not shown).

The graphics processing unit 12 is connected to the display 21 (e.g., a liquid crystal display) by wire or wirelessly. The graphics processing unit 12 renders game images including various objects in a virtual space in a moving image format according to game image information (image data) output from the control unit 16. The game images rendered in a moving image format (images in the virtual space) are displayed on the display 21 as a game screen.

The audio processing unit 13 is connected to the speaker 22 via a wired or wireless connection. The audio processing unit 13 plays digital game sounds according to instructions from the control unit 16. Specifically, the audio processing unit 13 converts the audio data output by the control unit 16 into an analog signal and outputs it to the speaker 22.

The audio processing unit 13 is configured to be capable of multi-channel audio output in order to reproduce three-dimensional sound. Accordingly, a number of speakers 22 are connected to the game device 10. These speakers 22 may be positioned to the left and right, in front and behind, above, etc., of a listener (e.g., a game player) in order to reproduce three-dimensional sound.

The operation processing unit 14 is connected to the controller 23 by wire or wirelessly. The operation processing unit 14 transmits and receives signals to and from the controller 23. The game player inputs signals (commands, data, etc.) to the game device 10 by operating various controls such as buttons on the controller 23.

The storage unit 15 stores various types of information and data. The storage unit 15 is composed of a HDD, SSD, RAM, ROM, etc. For example, the storage unit 15 stores game data, various programs including game programs, etc. Examples of game data include game media, user account information, etc.

The storage unit 15 also stores various types of audio data indicating various types of sounds that have been prepared in advance. The various types of audio data include audio data indicating sounds output from sound source objects, audio data indicating music (BGM) played in the virtual space, audio data indicating sound effects, and the like.

The control unit 16 controls the operation of the game device 10. The control unit 16 transmits and receives various information and data, and processes various information and data. The control unit 16 has a CPU (microcomputer) and a semiconductor memory. The semiconductor memory stores programs and data for operating the CPU.

[Functional configuration of the control unit]
The control unit 16 of the game device 10 has a game progression unit 100, a display processing unit 101, an audio processing unit 102, and an audio output unit 103. Specifically, the control unit 16 functions as the game progression unit 100, the display processing unit 101, the audio processing unit 102, and the audio output unit 103 by executing a game program.

<Game Progression Section (Virtual Space Control Section)>
The game progression unit 100 progresses a game that takes place in a virtual space. The game progression unit 100 is an example of a virtual space control unit that controls the virtual space.

In this example, the game progression unit 100 progresses the game by moving the player character in a virtual space in response to operations by the user (i.e., the game player) of the game device 10. The game progression unit 100 also moves non-player characters and predetermined objects (e.g., vehicles) in the virtual space. For example, the game progression unit 100 controls the movements of the non-player characters and predetermined objects using AI (artificial intelligence). The game progression unit 100 then progresses the game in response to the movements of the player characters and non-player characters.

For example, the game progression unit 100 progresses (controls) the game taking place in the virtual space by controlling (generating or updating) three-dimensional data representing the virtual space so that operations input to the controller 23 are reflected in the virtual space.

The three-dimensional data representing the virtual space includes well-known data for constructing the virtual space, such as data on a coordinate system (e.g., a world coordinate system) that serves as the reference for directions within the virtual space, data on the coordinate systems (e.g., a local coordinate system) of various objects placed within the virtual space, and data on the shapes of various objects. By controlling the three-dimensional data representing the virtual space, it is possible to control the movement of objects within the virtual space.

Display Processing Section
The display processing unit 101 generates an image (a game image in this example) of the virtual space controlled by the game progression unit 100. Specifically, the display processing unit 101 generates image data that represents the image of the virtual space, based on three-dimensional data (three-dimensional data representing the virtual space) controlled by the game progression unit 100. The process of generating an image from three-dimensional data is a well-known process (such as coordinate transformation or projection transformation).

Then, the display processing unit 101 reproduces (displays) the image generated as described above on the display 21. In this example, the display processing unit 101 supplies the image data generated as described above to the graphics processing unit 12. As a result, the image of the virtual space shown in the image data is reproduced (displayed) on the display 21.

As shown in FIG. 2, a virtual camera 61 is placed in the virtual space. The virtual camera 61 is placed near the player character C1. The virtual camera 61 is a virtual viewer (viewpoint), and is, for example, a reference point for reproducing the viewpoint of the player character C1. The display processing unit 101 changes the orientation of the virtual camera 61 so as to follow the orientation of the player character C1. The display processing unit 101 also moves the virtual camera 61 within the virtual space so as to follow the movement of the player character C1. Then, the display processing unit 101 displays an image of the virtual space obtained by the virtual camera 61 on the display 21.

Specifically, the display processing unit 101 generates image data showing the image according to the shooting conditions of the virtual camera 61 (e.g., the position and direction of the virtual camera 61 relative to an object in the virtual space, the orientation of the virtual camera 61, the orientation of the object, etc.) so that the image displayed on the display 21 is an image including an image obtained by the virtual camera 61. The image (image data) obtained by the virtual camera 61 is an image (image data) generated according to the shooting conditions of the virtual camera 61.

<Audio Processing Section>
The sound processing unit 102 generates sounds (game sounds in this example) in the virtual space controlled by the game progression unit 100. The sounds in the virtual space are sounds corresponding to the situation in the virtual space (such as the movement of objects). Specifically, the sound processing unit 102 selects sound data to be reproduced from the sound data (various types of sound data prepared in advance) stored in the storage unit 15 according to the situation in the virtual space (the progress of the game in this example), and generates sound data indicating sound corresponding to the situation in the virtual space based on the selected sound data (or new sound data obtained by synthesizing the selected multiple sound data).

Sound image localization processing
In this example, the sound processing unit 102 performs sound image localization processing, which is processing for localizing the position of a sound source as perceived by a human (player) at a predetermined virtual position (e.g., the position of a sound source object) in a virtual space.

As shown in FIG. 2, a virtual microphone 62 is placed in the virtual space. The virtual microphone 62 is placed near the player character C1 together with the virtual camera 61. The virtual microphone 62 is a virtual listener (sound collection point) and is a reference point for sound image localization in the sound image localization process. The sound processing unit 102 moves the virtual microphone 62 in the virtual space so as to follow the movements of the player character C1 and the virtual camera 61. The sound processing unit 102 then reproduces (outputs) the sound obtained by the virtual microphone 62 to the speaker 22.

Specifically, in the sound image localization process, the acoustic processing unit 102 generates audio data indicating the above-mentioned sound according to the sound collection conditions of the virtual microphone 62 (e.g., the position and direction of the virtual microphone 62 relative to the sound source object in the virtual space, the volume and direction of the sound output from the sound source object, the orientation of the virtual microphone 62, the orientation of the sound source object, etc.) so that the sound played (output) from the speaker 22 includes the sound obtained by the virtual microphone 62. The sound (audio data) obtained by the virtual microphone 62 is sound (audio data) generated according to the sound collection conditions of the virtual microphone 62. The sound output from the sound source object is sound associated with the sound source object (sound prepared in advance).

By processing the audio data generated by the sound image localization process by the audio processing unit 13, it becomes possible to output the sound output from the virtual sound source (sound source object) in the virtual space from the speaker 22 with an acoustic representation as if the sound was collected by the virtual microphone 62.

<Sound effect processing>
The sound processor 102 also performs sound effect applying processing. In this example, the sound processor 102 applies sound effects to the sound generated by the sound image localization processing. Examples of sound effects include volume adjustment and reverberation (adding reverberation and reflected sound).

As shown in FIG. 2, the virtual space is provided with a plurality of areas including a first area 51 and a second area 52. The first area 51 and the second area 52 are adjacent to each other.

In this example, a communication part (e.g., an opening) that connects the first area 51 and the second area 52 is provided at the boundary between the first area 51 and the second area 52. The player character C1 can move between the first area 51 and the second area 52 via the communication part between the first area 51 and the second area 52. For example, each of the first area 51 and the second area 52 is an area (closed space) surrounded by a shield (object) such as a wall.

The first area 51 and the second area 52 may be non-movable (non-trafficable). For example, an object such as a window may be provided to block the communication between the first area 51 and the second area 52.

A boundary sound source 65, which is a virtual sound source (sound source object), is placed at the boundary between the first area 51 and the second area 52. The boundary sound source 65 is placed on the boundary line between the first area 51 and the second area 52 at the communicating portion between the first area 51 and the second area 52. For example, the boundary sound source 65 is an object such as a door or a window.

Each of the multiple areas, including the first area 51 and the second area 52, is associated with a sound effect that is applied to the sound in the virtual space. In this example, the storage unit 15 stores sound effect information that is information indicating the sound effect associated with each of the multiple areas.

In the sound effect imparting process, the sound processing unit 102 imparts a sound effect corresponding to the "area located closer to the virtual microphone 62 in the boundary sound source 65" of the first area 51 and the second area 52 to the boundary sound, which is the sound of the boundary sound source 65 obtained by the virtual microphone 62 arranged in the virtual space. In this example, the sound processing unit 102 detects a sound effect corresponding to the "area located closer to the virtual microphone 62 in the boundary sound source 65" from the sound effect information stored in the storage unit 15, and performs processing to impart a sound effect to the sound data indicating the boundary sound so that the detected sound effect is imparted to the boundary sound.

In this example, a determination area 55 is set in the virtual space. The determination area 55 is an area of a predetermined range based on the boundary sound source 65. The determination area 55 has a first area portion 56 belonging to the first area 51 and a second area portion 57 belonging to the second area 52. For example, the determination area 55 is a rectangular parallelepiped area that spans the first area 51 and the second area 52.

In the sound effect applying process, the sound processing unit 102 applies the sound effect associated with the first area 51 to the boundary sound when the first area 56 of the first area 56 and the second area 57 is closer to the virtual microphone 62. In addition, the sound processing unit 102 applies the sound effect associated with the second area 52 to the boundary sound when the second area 57 of the first area 56 and the second area 57 is closer to the virtual microphone 62.

For example, the sound processing unit 102 performs ray casting using the virtual microphone 62 as a reference (starting point) to derive the distance from the virtual microphone 62 to the first area 56 (or the second area 57) and determines whether the first area 56 or the second area 57 is closer to the virtual microphone 62. Ray casting is a technique that extends straight lines (virtual rays) radially from the starting point and derives the distance from the virtual microphone 62 to an object based on the distance to the intersection of these straight lines with the object (the first area 56 and the second area 57).

<Audio output section>
The audio output unit 103 reproduces (outputs) the audio processed (generated) by the audio processing unit 102 to the speaker 22. In this example, the audio output unit 103 supplies the audio data generated by the audio processing unit 102 to the audio processing unit 13. As a result, the audio of the virtual space indicated by the audio data is reproduced (output) from the speaker 22.

The sound output unit 103 also outputs the boundary sound to which the sound effects have been added by the sound processing unit 102 to the speaker 22. Specifically, the sound output unit 103 supplies the audio processing unit 13 with audio data (audio data generated by the sound processing unit 102) indicating the sound including the boundary sound to which the sound effects have been added by the sound processing unit 102, thereby outputting the sound including the boundary sound to which the sound effects have been added to the speaker 22.

[Comparison between the embodiment and the comparative example]
Next, the embodiment and a comparative example will be compared. In the following, for convenience of explanation, the reference numerals of the components in the comparative example are used in the explanation of the components in the comparative example. Also, the comparative example in the following explanation is different from the embodiment in that the sound effect applying process does not determine "which of the first area 56 and the second area 57 is closer to the virtual microphone 62". Other processes in the comparative example are similar to those in the embodiment.

In the above comparative example, since it is uncertain whether the boundary sound source 65 belongs to the first area 51 or the second area 52, it is difficult to appropriately impart sound effects to the boundary sound output from the boundary sound source 65. For example, if the boundary sound source 65 is a "door" and the boundary sound is the "sound of a door opening," when the player character C1 located in the first area 51 opens the "door," if it is determined that the "door" belongs to the second area 52, it will be determined that the "sound of a door opening" has been output in the second area 52, which is on the other side of the "door" as seen by the player character C1. In this case, the "sound of a door opening" will not reach the virtual microphone 62 located in the first area 51 together with the player character C1.

On the other hand, in the embodiment, by determining whether the first area 56 or the second area 57 is closer to the virtual microphone 62, it is possible to appropriately determine whether the boundary sound source 65 should belong to the first area 51 or the second area 52. For example, when the player character C1 located in the first area 51 opens a "door", it is possible to determine that the "door" belongs to the first area 51, and therefore it is possible to deliver the "sound of the door opening" to the virtual microphone 62 located in the first area 51 together with the player character C1.

[Overview of the embodiment]
In summary, the program (game program) of the embodiment is a program that causes a computer system having one computer or a plurality of computers working together to function as an audio processing unit 102 and an audio output unit 103. The computer system is used together with a speaker 22 for outputting audio in a virtual space. A first area 51 and a second area 52 adjacent to each other are provided in the virtual space, and an audio effect to be added to the audio in the virtual space is associated with each of the first area 51 and the second area 52. A boundary sound source 65, which is a virtual sound source, is disposed at the boundary between the first area 51 and the second area 52. The audio processing unit 102 adds an audio effect associated with an area of the first area 51 and the second area 52 located closer to the virtual microphone 62 in the boundary sound source 65 to the boundary audio, which is the audio of the boundary sound source 65 obtained by the virtual microphone 62 disposed in the virtual space.
The audio output unit 103 causes the speaker 22 to output the boundary sound to which the audio effect has been added by the audio processing unit 102 .

The acoustic control device (game device 10) of the embodiment includes a storage unit 15 that stores the above program, and a control unit 16 that executes the above program.

[Effects of the embodiment]
As described above, in the embodiment, a sound effect associated with the area of the first area 51 or the second area 52 that is closer to the virtual microphone 62 in the boundary sound source 65 is added to the boundary sound, which is the sound of the boundary sound source 65 obtained by the virtual microphone 62 arranged in the virtual space. This makes it possible to appropriately determine whether the boundary sound source 65 should belong to the first area 51 or the second area 52, and therefore makes it possible to appropriately impart sound effects to the boundary sound output from the boundary sound source 65. As a result, the boundary sound to which an appropriate sound effect has been imparted can be reproduced (output) from the speaker 22, thereby improving the user experience.

Other Embodiments
In the above description, the following configuration or setting may be adopted.

The shape of the judgment region 55 may be other than a rectangular parallelepiped.

The virtual microphone 62 may be used to determine whether the first area 51 or the second area 52 is the "area located on the side of the boundary sound source 65 closer to the virtual microphone 62," or a virtual camera 61 (or player character C1) placed near the virtual microphone 62 may be used instead of the virtual microphone 62. For example, a raycast may be performed based on the virtual camera 61 (or player character C1).

The display 21 may be provided on a head-mounted display worn by the player. The virtual space may be a "virtual reality space in which virtual reality (VR) can be experienced."

The audio data does not necessarily need to be prepared in advance. For example, the acoustic processing unit 102 may synthesize or generate audio data each time based on some data (e.g., audio data stored in the memory unit 15) or information.

The speaker 22 may be a stationary speaker or a wearable speaker such as an earphone or a headphone.

The image of the virtual space may be from a first-person perspective or a third-person perspective.

The game program may also be implemented as a game program for a so-called online game. In this case, the processing performed in the control unit 16 of the game device 10 may be performed in a control unit of a server device (not shown), or may be shared between the game device 10 and the control unit of the server device. The game device 10 may be a mobile information terminal such as a smartphone or tablet.

In the above explanation, a virtual space in which a game progresses (game space) has been given as an example of a virtual space, but this is not limiting. For example, the virtual space may be a virtual space in which avatars communicate with each other (metaverse space). In other words, the services provided in the virtual space are not limited to games. For example, the services provided in the virtual space may be the provision of a place for commercial transactions, the provision of a place for communication, etc.

Even when these other embodiments are adopted, the effects of the present invention are achieved. In addition, this embodiment can be combined with other embodiments, and other embodiments can be combined with each other as appropriate. The above embodiments are essentially preferred examples, and are not intended to limit the scope of the present invention, its applications, or its uses.

10 Game device (sound device)
11 Network interface 12 Graphics processing unit 13 Audio processing unit 14 Operation unit 15 Memory unit 16 Control unit 17 Bus 21 Display 22 Speaker 23 Controller 51 First area 52 Second area 55 Determination area 56 First area unit 57 Second area unit 61 Virtual camera 62 Virtual microphone 65 Boundary sound source 100 Game progression unit (virtual space control unit)
101 Display processing unit 102 Sound processing unit 103 Sound output unit

Claims (4)

A program for causing a computer system having one computer or a plurality of computers working together to function as an audio processing unit and an audio output unit,
The computer system is used together with a speaker for outputting sound within a virtual space;
The virtual space includes a first area and a second area adjacent to each other,
a sound effect to be added to a sound in the virtual space is associated with each of the first area and the second area;
A boundary sound source that is a virtual sound source is disposed at a boundary between the first area and the second area;
the sound processing unit adds a sound effect associated with an area of the first area and the second area that is located closer to the virtual microphone in the boundary sound source to a boundary sound that is a sound of the boundary sound source obtained by a virtual microphone arranged in the virtual space,
The sound output unit is a program that causes the speaker to output the boundary sound to which the sound effect has been added by the sound processing unit. In the program of claim 1,
A determination area is set in the virtual space, the determination area being a predetermined range based on the boundary sound source, the determination area having a first area portion belonging to the first area and a second area portion belonging to the second area,
The sound processing unit is a program that adds sound effects associated with the first area to the boundary sound when the first area of the first area and the second area is closer to the virtual microphone, and adds sound effects associated with the second area to the boundary sound when the second area of the first area and the second area is closer to the virtual microphone. In the program of claim 1,
The computer system is caused to function as a display processing unit,
the computer system is utilized in conjunction with a display for displaying an image of the virtual space;
the display processing unit causes the display to display an image of the virtual space obtained by a virtual camera disposed in the virtual space;
The sound processing unit is a program that moves the virtual microphone in the virtual space so as to follow the movement of the virtual camera in the virtual space. A storage unit that stores the program according to any one of claims 1 to 3;
and a control unit for executing the program.