JP2008079748A - Image sound output device, image sound output method, and image sound output program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To permit natural sounds according to the situation of an object expressed in an image to be output. <P>SOLUTION: The image sound output device such as a game device which outputs images and sounds is provided with a physical engine 18 which computes the movement of the object in a three-dimensional space and detects the strength of collision and the site of collision of the object in the three-dimensional space. A CPU 10 displays the object in the image expressing the three-dimensional space on the basis of the result of computation by the physical engine 18. The CPU 10 acquires basic sound data of the object, the collision of which is detected by the physical engine 18 from a basic sound data base 16a, generates waveform data corresponding to the strength of collision and the site of collision on the basis of the basic sound data and outputs sounds corresponding to the generated sound waveform data from a speaker 24 by a sound processor 22. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video / audio output device that outputs video and audio, such as a game device, a video / audio output method in the same device, and a video / audio output program.

  In general, in a game device that outputs video and audio during a game, sound such as sound effects and BGM (back ground music) is output in accordance with changes in objects displayed in the video. For example, when an object collides with another object or the like in the game space, a sound effect corresponding to the object is output in accordance with a video where the object collides.

  In a conventional game device, in order to output a sound effect corresponding to an object, voice data to be output when an object collides is prepared in advance. Then, after adjusting the volume or pitch to the sound data according to the game situation, the sound is output.

For example, in the game device disclosed in Patent Document 1, pitch processing is performed on sound waveform data (sound data) read from a disc, volume adjustment of sound data and sound effects, and mixing with other tracks. It is processing and outputting sound.
JP 2005-321514 A

  As described above, conventionally, a predetermined voice for an object is output while changing a volume, a pitch, or the like according to a game situation. Usually, audio data is not prepared for all situations of an object expressed in a video space (game space). Therefore, even if the volume and pitch are adjusted, it is impossible to output a natural sound corresponding to all the situations of the object.

  The present invention has been made in consideration of the above-described circumstances, and an object of the present invention is to provide a video / audio output apparatus and a video / audio output method capable of outputting natural audio in accordance with the status of an object represented in the video. Another object is to provide a video / audio output program.

  The present invention relates to a video / audio output device that outputs video and audio, and calculates a motion of an object in a three-dimensional space and detects a collision strength and a place where the colliding object collides in the three-dimensional space. And object display means for displaying an object in a video representing a three-dimensional space based on a calculation result by the calculation means, and basic audio data for storing basic audio data corresponding to the object displayed in the video The basic voice data of the object whose collision is detected by the storage means and the calculation means is acquired from the basic voice data storage means, and the strength at the time of the collision and the location of the collision are determined based on the basic voice data. Sound waveform generation means for generating sound waveform data, and sound corresponding to the voice waveform data generated by the sound waveform generation means Characterized by comprising an audio output means for outputting.

  According to the present invention, a collision of an object in a video representing a three-dimensional space is detected, and sound waveform data corresponding to the strength when the object collides and the location of the collision is generated and output. The sound corresponding to the individual situation when the object displayed in is collided is output. For this reason, it is possible to output natural sound according to the state of the object expressed in the video.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a video / audio output apparatus according to the present embodiment. The video / audio output device in the present embodiment will be described as being realized as a game device.

  The game device shown in FIG. 1 includes a CPU 10, a storage device 12, a memory 14, a database 16, a physical engine 18, a display 20, a sound processor 22, a speaker 24, and an input device 26.

  The CPU 10 controls the entire game machine and executes game processing according to various programs stored in the memory 14. In the game process, video and audio in accordance with the progress of the game are output. The CPU 10 used in the present embodiment can execute a multi-thread program, and can launch and execute a sound waveform creation thread in parallel with a game process described later. The object waveform creation thread is executed to generate sound waveform data of a sound to be output when an object collides in the game space.

  The storage device 12 stores various programs and data in a storage medium such as a hard disk, a CD-ROM, and a DVD. In addition to an OS (operating system) that controls the basic system, a game main processing program for realizing a game , Various programs such as image processing programs, sound waveform creation programs, various data such as data (parameter data, image data, audio data, etc.) related to various objects (including player characters and non-player characters) appearing in the game, etc. Remembered. The programs and data stored in the storage device 12 are read as necessary and stored in the memory 14. The parameter data is data indicating characteristics of the object in the game space, and includes data indicating characteristics that change the sound generated when the object collides, such as the mass, shape, and frictional force of the object. .

  The memory 14 includes a RAM, a ROM, and the like, and stores programs executed by the CPU 10 and various data. The memory 14 (RAM) is used as a buffer area for processing executed by the CPU 10, and is used for, for example, work for generating sound waveform data corresponding to an object in a video displayed on the display 20.

  The database 16 is for registering data used in the game process, and includes, for example, a basic voice database 16a. In the basic audio database 16a, audio data for outputting audio corresponding to an object displayed in a video representing the game space is registered (see FIG. 3).

  The physics engine 18 is used to display an object (3D object) in a 3D image representing a game space so as to move in the same manner as an actual 3D space. A physical processing processor that specializes in processing only (physics processing unit (PPU)). Based on the characteristics of each object defined in the game space, the physics engine 18 determines how the objects in the game space relate to each other, for example, for a certain event executed in the game process. Yes, how the situation changes (exercises) is calculated according to the laws of physics. By the arithmetic processing by the physics engine 18, a collision with another object or the like in the game space is detected. The physics engine 18 outputs, to the CPU 10, data representing the strength of the collision of the object and the location of the collision (hereinafter referred to as hit data (see FIG. 4)) for the object in which the collision is detected.

  Although not shown, in addition to the physics engine 18, a GPU (Graphic Processing Unit) or the like may be provided to display a three-dimensional game image. For example, in response to a command from the CPU 10, the GPU performs coordinate calculation processing such as rotation, movement, and deformation of each object (including characters) in a game space of a three-dimensional coordinate system, or in a video representing the game space. A process for generating three-dimensional image data to be displayed is executed.

The display 20 displays a screen (video) corresponding to the display data generated by the object drawing process.
The sound processor 22 causes the speaker 24 to output sound corresponding to the sound waveform data generated by the sound waveform creation process executed by the CPU 10. The sound processor 22 outputs sound according to sound waveform data generated according to the state of the object in the game space, so that the natural state corresponding to the state of the object of the video representing the game space displayed on the display 20 is obtained. Sound can be output. In addition, the sound processor 22 performs processing such as pitch modulation, mixing with BGM audio, etc., and mixing with other tracks.

  The speaker 24 outputs sound corresponding to the sound signal generated by the sound processor 22.

  FIG. 2 is a diagram showing a memory map of the memory 14. As shown in FIG. 2, the memory 14 includes a program storage area 30, an object data area 32, an object image data area 34, a basic audio data area 36, and the like.

  The program storage area 30 is an area where various programs for executing game processing are stored. In addition to the OS (operating system), a game main processing program 30a, an image processing program 30b, a sound waveform creation program 30c, and the like. include.

  The game main processing program 30a is a program for controlling the entire game processing, and outputs a sound corresponding to an image (video) representing a three-dimensional space including objects and the same video as the game is executed.

  The image processing program 30b is a program for causing the display 20 to display an image (video) associated with the game processing by the game main processing program 30a.

  The sound waveform creation program 30c generates speech waveform data of speech corresponding to each object in order to output natural speech corresponding to the image (video) displayed by the image processing program 30b. It is a program for executing. Based on the hit data (indicating the strength when the object collides and the location where the object collided) obtained by the sound waveform creation program 30c by the arithmetic processing in the physics engine 18, the basic audio data corresponding to the object is converted into the basic audio database 16a. The sound waveform data corresponding to the strength at the time of collision and the location of the collision is generated based on the basic voice data.

  The object data area 32 is an area for storing data related to various objects displayed in the game space, and stores data for each of the objects 1, 2,. For example, the object 1 includes position data 40 indicating the position of the object in the game space, and parameter data 42 indicating characteristics in the game space (for example, indicating the mass, shape, frictional force, etc. of the object). Similarly, it is assumed that position data 40 and parameter data 42 are set for each object 2.

  The object image data area 34 is an area for storing image data for displaying an object in the game space, and stores data for each of the objects 1, 2,.

  The basic audio data area 36 is an area for storing basic audio data corresponding to an object displayed in the game space, and the basic audio data of the object to be displayed is read from the basic audio database 16a. Remembered.

FIG. 3 is a diagram showing an example of the basic voice database 16 a stored in the database 16.
As shown in FIG. 3, in the basic audio database 16a, basic audio data (basic audio) to be output when an object collides (hits) with another object or the like for each object material defined in the game space. Data) is registered. The basic audio data is created by, for example, PCM (pulse cord modulation) data. Sound waveform data for outputting sound can be generated by the sound processor 22 based on the basic sound data stored in the basic sound database 16a.

FIG. 4 is a diagram illustrating an example of hit data acquired from the physics engine 18.
The hit data is data detected by physics processing in the physics engine 18, hit power data indicating the strength when an object whose collision has been detected in the game space, and hit location data representing the location of the collision. It is included. The physics engine 18 notifies the CPU 10 of hit data (hit power data, hit location data) for each of all objects displayed in the video representing the game space as a calculation result by physical processing.

Next, the operation of the game device in this embodiment will be described.
FIG. 5 is a flowchart for explaining the game processing in the present embodiment.
The CPU 10 activates the game main process program 30a loaded in the memory 14 and starts the game process.

  The CPU 10 creates a game space by executing the game main processing program 30a, and advances the game in this game space. In the game space, a player character that is operated by the player operating the input device 26, a non-player character that operates in the game space according to a predetermined algorithm, and a predetermined motion in the game space according to the game situation Various objects are defined in the game space.

  When the game process is started, the CPU 10 executes an operation input process and inputs an instruction input by an operation on the input device 26 by the user (step A1).

  CPU10 performs the following processes according to the user instruction input by the operation input process (steps A2 to A6).

  First, the CPU 10 executes character action processing in accordance with a user instruction (step A2). The CPU 10 executes a process for moving the player character in response to a user instruction, a process for operating a related non-player character (for example, an enemy character, a stationary character, etc.) in accordance with the action of the player character, and the like. .

  For example, when the user operates the input device 26 (cross key or joystick) and designates a direction, the CPU 10 executes a character action process for moving the player character in the designated direction in the game space. In addition, when a predetermined button provided on the input device 26 is operated by the user, the CPU 10 executes a process for causing an attacking action using a weapon (gun, sword, etc.) provided on the player character. .

  Next, the CPU 10 uses the image processing program 30b to execute object drawing processing for displaying an image (video) in which characters and objects in the game space change according to the game execution status (step A3). At this time, the CPU 10 outputs object data (position data, parameter data) of each object to be displayed in the game space to the physics engine 18 to calculate the movement of each object in the three-dimensional game space.

  Based on the object data input from the CPU 10, the physics engine 18 calculates the relationship between the objects in the game space and how the situation changes (exercises) according to the laws of physics. Ask. The CPU 10 displays the object in the game space based on the calculation result by the physics engine 18, that is, the position data indicating the position of each object in the game space. Since the physics engine 18 calculates the movement of the object in accordance with the laws of physics, by displaying the object based on the calculation result, it is possible to display the object so as to perform the same movement as in the actual three-dimensional space.

  When the physics engine 18 detects a collision (hit) between the objects by physical processing for each object, the hit data of each object in which the collision has occurred, that is, the strength when the object in which the collision is detected collides. The hit power data indicating the height and the hit location data representing the colliding location in the object are output to the CPU 10. Since the physics engine 18 performs an operation in accordance with the laws of physics, the physics engine 18 can detect not only the presence / absence of collision of the object but also the strength at the time of collision and from which part of the object the collision occurs.

  When the collision of the object is detected by the physics engine 18 and hit data for each object is received, the CPU 10 activates a sound waveform creation thread by the sound waveform creation program 30c and executes a sound waveform creation process (step A5). The CPU 10 outputs a natural sound corresponding to the situation (intensity of collision, collision location) of each object when the object collides in the three-dimensional space by executing the sound waveform creation process.

FIG. 6 is a flowchart for explaining the sound waveform creation processing in the present embodiment.
First, the CPU 10 acquires hit data of an object collided in the game space from the physics engine 18 (step B1), and acquires basic audio data for creating sound waveform data of the collided object from the basic audio database 16a. And stored in the memory 14 (step B2).

  In accordance with the hit data, the CPU 10 generates sound waveform data for outputting a sound when the object collides based on the basic voice data (step B3). The basic sound data is created according to the material (for example, metal, wood, glass) defined in the game space of the object. In addition, when the material of the object is different, the sound changes due to the difference in strength at the time of collision. The change in sound also varies depending on the material of the colliding opponent's object. Furthermore, the change in sound differs depending on where the object collides, for example, from a corner portion or a surface portion. Furthermore, the change in sound varies depending on the collision direction (frontal collision, etc.) of the object.

  The CPU 10 deforms the sound waveform indicated by the basic sound data according to the situation when the object collides as described above, and generates sound waveform data representing the sound when the object collides.

  The CPU 10 generates sound waveform data for each object detected to have collided in the physics engine 18, sends the sound waveform data to the sound processor 22, and the sound processor 22 generates sound corresponding to the sound waveform data from the speaker 24. (Step B4). Since the sound output from the speaker 24 is generated in accordance with the movement of the object similar to the reality detected by the calculation according to the physical law by the physics engine 18, a natural sound corresponding to the change of the object in the video is obtained. It has become.

  The CPU 10 outputs other sounds according to the state of each object by the sound waveform creation process, and then executes other game processes (step A6). The CPU 10 determines whether or not to end the game. If the game process is to be continued (No at Step A7), the CPU 10 returns to Step A1 and repeatedly executes the same process as described above (Steps A1 to A6). On the other hand, when it is determined that the game is to be ended (step A7, Yes), the CPU 10 executes a predetermined game end process and ends the game process.

  In this manner, according to the game device (video / audio output device) in the present embodiment, the sound waveform data for the object collided in the game space is generated using the calculation result of the physical processing by the physics engine 18. Since it can be output, it is possible to output a natural sound in accordance with the state of the object in association with the object displayed so as to change in the same manner as an actual three-dimensional space using the physics engine 18.

  In the above description, collision between objects is detected by the physics engine 18, but this collision includes collision between a solid and a fluid (gas, liquid) in addition to collision between objects representing the solid. In addition, contact between objects (moving in contact) is also included.

  In the above description, the game apparatus is described as an example. However, the present invention can be applied to various video / audio output apparatuses that output audio corresponding to video together with video including objects.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  In addition, the method described in the embodiment uses, as a program (software means) that can be executed by a computer, for example, a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, MO, etc.), and a semiconductor memory. It can also be stored in a recording medium such as (ROM, RAM, flash memory, etc.), or transmitted and distributed via a communication medium. The program stored on the medium side also includes a setting program that configures in the computer software means (including not only the execution program but also a table and data structure) that is executed by the computer. A computer that implements the present apparatus reads the program recorded on the recording medium, constructs software means by a setting program in some cases, and executes the above-described processing by controlling the operation by the software means. The recording medium referred to in this specification is not limited to distribution, but includes a storage medium such as a magnetic disk or a semiconductor memory provided in a computer or a device connected via a network.

The block diagram which shows the structure of the audio-video output apparatus in this embodiment. The figure which shows the memory map of the memory. The figure which shows an example of the basic audio | voice database 16a memorize | stored in the database 16. The figure which shows an example of the hit data acquired from the physics engine. The flowchart for demonstrating the game process in this embodiment. The flowchart for demonstrating the sound wave formation process in this embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... CPU, 12 ... Storage device, 14 ... Memory, 16 ... Database, 16a ... Basic voice database, 18 ... Physical engine, 20 ... Display, 22 ... Sound processor, 24 ... Speaker, 26 ... Input device, 30 ... Program storage Area 30a ... Game main processing program 30b ... Image processing program 30c ... Sound waveform creation program 32 ... Object data area 34 ... Object image data area 36 ... Basic audio data area 40 ... Position data 42 ... Parameter data.

Claims (6)

In a video / audio output device that outputs video and audio,
A calculation means for calculating the movement of the object in the three-dimensional space, detecting the strength of the collision of the colliding object in the three-dimensional space, and the location of the collision;
An object display means for displaying an object in a video representing a three-dimensional space, based on a calculation result by the calculation means;
Basic audio data storage means for storing basic audio data corresponding to an object displayed in a video;
The basic voice data of the object whose collision is detected by the computing means is acquired from the basic voice data storage means, and the sound waveform data corresponding to the strength at the time of the collision and the location of the collision is obtained based on the basic voice data. Sound waveform generating means for generating;
An audio / video output apparatus comprising: audio output means for outputting audio corresponding to the audio waveform data generated by the sound waveform generating means.   The video / audio output apparatus according to claim 1, wherein the calculation unit is realized by a physical processing processor that calculates the movement of an object in a three-dimensional space according to a physical law.   2. The video / audio output apparatus according to claim 1, wherein the computing means detects the strength when the object collides and the location where the object collides, based on parameter data indicating the characteristics of the object. A CPU that executes a program for outputting audio corresponding to the video representing the three-dimensional space including the object;
A physical processor that calculates the movement of an object defined in a three-dimensional space by executing the program of the CPU according to a physical law;
Basic voice data storage means for storing basic voice data corresponding to the object,
Based on the calculation result by the physical processor, the collision strength of the colliding object in the three-dimensional space and the location of the collision are detected.
Obtaining the basic audio data of the object in which this collision is detected from the basic audio data storage means, and generating sound waveform data according to the strength and location of the collision based on the basic audio data,
An audio / video output device that outputs audio corresponding to the generated sound waveform data. In a video / audio output method for outputting video and audio,
A calculation step of calculating the movement of the object in the three-dimensional space, detecting the strength of the collision of the colliding object in the three-dimensional space, and the location of the collision;
An object display step for displaying an object in a video representing a three-dimensional space based on the calculation result of the calculation step;
A sound waveform generating step for generating sound waveform data corresponding to the strength and the location of the collision based on the basic audio data corresponding to the object in which the collision is detected by the calculation step;
An audio / video output method comprising: an audio output step for outputting audio corresponding to the audio waveform data generated by the sound waveform generation step. Computer
In a video / audio output device that outputs video and audio,
A calculation means for calculating the movement of the object in the three-dimensional space, detecting the strength of the collision of the colliding object in the three-dimensional space, and the location of the collision;
An object display means for displaying an object in a video representing a three-dimensional space, based on a calculation result by the calculation means;
Basic audio data storage means for storing basic audio data corresponding to an object displayed in a video;
The basic sound data of the object whose collision is determined by the determining means is acquired from the basic sound data storage means, and the sound waveform data corresponding to the strength at the time of the collision and the location of the collision is obtained based on the basic sound data. Sound waveform generating means for generating;
A video / audio output program for functioning as an audio output unit that outputs audio corresponding to the audio waveform data generated by the sound waveform generation unit.

Images