JP2009246600A - Production control program of sound generated from sound source in virtual space - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a program of controlling the generation of a sound from a sound source object in virtual reality in response to a state change of an object as an obstacle of a sound source and a sound. <P>SOLUTION: The size of the object in a predetermined three-dimensional shape is decided in response to sound data correlated with the sound source object, and the object in the three-dimensional shape of the decided size is arranged in a virtual three-dimensional space. An angle of view of the object in the three-dimensional shape is set from a position of a second viewpoint inside the virtual three-dimensional space, and the object in the three-dimensional shape is perspectively converted into a two-dimensional plane so as to achieve the preset image size. Two-dimensional image data for deciding the magnitude of a sound is produced, the magnitude of a sound to generate from the sound source object is decided based on a size ratio of an overlapping portion between an image area of the produced two-dimensional image data and an image area in a two-dimensional shape for the other objects barring the sound, and a sound to generate by the sound source object is output at the decided magnitude of the sound. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a sound generation control program generated from a sound source in a virtual space.

  In an image processing system such as a game device, control is performed to generate sound from a virtual sound source in conjunction with virtual three-dimensional or two-dimensional image processing.

  As a prior art related to such control, there is an invention described in Patent Document 1. In the invention described in Patent Document 1, when generating a three-dimensional sound from the position information of the object and the sound type information given from the host system, there is a fine control of controlling the way of hearing according to the direction in which the sound comes out. It points out that it was impossible.

In order to solve this problem, an object table for holding information on the vertex coordinates of each of one or more polygons constituting the target image and the sound to be generated from each polygon is prepared. Based on the position and orientation of each polygon when viewed from the (virtual viewpoint) and the information in the object table, it controls how the sound is heard according to the direction of the sound coming out of the polygon.
JP-A-8-149600

  Here, when the image displayed on the game device is an image obtained by perspective-transforming an image in the virtual three-dimensional space into a two-dimensional plane, there is an obstacle between the sound source placed in the virtual three-dimensional space. When present, the way the sound is heard (the volume of the sound) is different from when the sound from the sound source is heard directly.

  Furthermore, the form of the obstacle that blocks the sound changes in the virtual three-dimensional space in accordance with the progress of the game. Along with this, the way the sound from the sound source is heard should also change.

  However, in the invention described in Patent Document 1, there is no description about the generation of sound in consideration of such an obstacle.

  Accordingly, an object of the present invention is to provide a sound source and a sound failure according to the progress of a game in a game device that controls the movement of an object placed in a virtual three-dimensional space and displays the state of the object on a two-dimensional screen. An object of the present invention is to provide a control program for controlling the generation of sound from a sound source object in response to a change in state of an object to be a thing.

A basic aspect of the present invention that achieves the above object is that a first viewpoint and an object are arranged in a virtual three-dimensional coordinate space, and the state of the virtual three-dimensional space including the object viewed from the first viewpoint is perspective-transformed. A program for causing a computer to execute a process of generating a two-dimensional image, displaying the generated two-dimensional image on a display unit, and outputting a sound preset for the object to a sound output unit There,
In the computer,
Means for reading sound data stored in a storage unit in association with a sound source object set in advance to generate sound, and determining a size of a predetermined three-dimensional object corresponding to the sound source data;
The three-dimensional object having the determined size is arranged in a virtual three-dimensional space in correspondence with the sound source object, and the viewing direction with respect to the three-dimensional object from the position of the second viewpoint in the virtual three-dimensional space The two-dimensional image data for determining the volume of sound is generated by perspective-transforming the three-dimensional object into a two-dimensional plane so that the angle of view is set and the image size is set in advance. Means,
The size ratio of the overlapping part of the two-dimensional shape image region corresponding to the three-dimensional shape object in the generated two-dimensional image data and the two-dimensional shape image region corresponding to another object that blocks sound Means for determining a loudness generated from the sound source object, based on
The sound output unit may function as a means for outputting a sound generated by the sound source object from a sound output unit at the determined sound volume.

  Thereby, it is possible to virtually control the generation of sound from the sound source object in response to the state change of the sound source and the object that becomes a sound obstacle.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a functional block diagram of a configuration example of the game apparatus main body. Each functional unit transmits and receives data through the bus 11.

  In accordance with the game program stored in the external memory or the ROM 2 functioning as the internal memory, the main CPU 1 as the control means functions as each functional unit necessary for realizing the execution of the game.

  In the game execution, the image data creation function unit 100 by the main CPU 1 performs coordinate conversion on the object data stored in the ROM 2 to form object data that is controlled to move according to the game program in the virtual three-dimensional space of the world coordinate system. To do.

  Further, the data of the object having the virtual three-dimensional space coordinates whose movement is controlled is perspective-transformed into two-dimensional coordinates to be displayed on the display device 5 by the rendering processor 4.

  At this time, the rendering processor 4 reads the texture data from the texture memory 6 and pastes the read texture on the object data coordinate-converted to the two-dimensional coordinates in accordance with the game program, and draws it in the video memory 7 as image data. To do.

  The image data drawn in the video memory 7 is read at a predetermined cycle and sent to the display device 5, whereby a game image including an object whose movement is controlled according to the game program is displayed on the display device 5.

  Furthermore, the sound source device 9 is provided as a sound output unit, and the sound source device 9 generates sound and outputs from the speaker 10 as the sound generation control function unit 101 by the main CPU 1 according to the game program.

  Among sounds generated by the sound source device 9, sounds generated from objects that are controlled to move according to the game program are included.

  In contrast to the basic operation of such a game apparatus, the present invention is characterized by a method for controlling generation of sound generated from an object processed in accordance with a game program stored in a ROM 2 as an embodiment.

  FIG. 2 is a diagram showing an example of two hostile character objects C1 and C2 placed in a virtual three-dimensional space according to a battle game program by such a game device. The character objects C1 and C2 are formed by a plurality of polygons, and three-dimensional coordinates, normal vectors, transparency, etc. are defined for the vertices constituting each polygon.

  The example shown in FIG. 2 shows a state where the enemy character object C2 is sounding toward the character object C1 controlled by the player as a sound source object.

  The rendering processor 4 converts the three-dimensional image data including the character objects C1 and C2 from the first viewpoint 20 in the virtual three-dimensional space to the two-dimensional plane 21 within the range of the front clipping plane 22A and the rear clipping plane 22B. Perspective transformation is performed to generate two-dimensional surface image data.

  As the two-dimensional surface image data, the rendering processor 4 pastes the texture read from the texture memory 6 and draws it in the video memory 7 according to the polygon data attached to each of the plurality of polygons constituting the character objects C1 and C2. To do. The two-dimensional plane image data of the game image drawn in the video memory 7 is sequentially read out and displayed on the display device 5.

  Further, as will be described later in detail in synchronization with the processing of the two-dimensional plane image data of the game image, the sound generation control function of the main CPU 1 is used to control the volume of sound generated by the character object that becomes the sound source object. The unit 101 generates sound source image data in a virtual three-dimensional space.

  The rendering processor 4 perspective-converts the sound source image data in the virtual three-dimensional space generated by the sound generation control function unit 101 into two-dimensional plane image data and draws it in the video memory 7 or an appropriate RAM.

  FIG. 3 shows a state in which the two-dimensional surface image data of the game image is displayed on the display surface 22 of the display device 5.

  The sound output by setting the sound generation control function unit 101 of the CPU 1 is set so that the sound generation from the enemy character object C2 as the sound source object is generated according to the program in synchronization with the display on the display surface 22. The sound source device 9 serving as a unit outputs a sound with a set magnitude from the speaker 10.

  On the other hand, FIG. 4 is a diagram illustrating an example of two character objects C1 and C2 placed in the virtual three-dimensional space corresponding to FIG. A sound obstacle object C3 serving as a sound shielding object is positioned between the character object C1 operated by the player and the enemy character object C2 serving as a sound source object. The sound obstacle object C3 is also composed of a plurality of polygons.

  FIG. 5 shows a state in which the three-dimensional image of FIG. 4 is converted into two-dimensional surface image data and displayed on the display surface 22 of the display device 5.

  Here, assuming the virtual three-dimensional space shown in FIG. 4, when the pronunciation from the enemy character object C2 that is a sound source object reaches the character object C1, in the example of FIG. 2, there is a failure between the character objects C1 and C2. There is no object and the pronunciation from the enemy character object C2 arrives as it is.

  On the other hand, in the example shown in FIG. 4, there is an obstacle object C3 that is an obstacle between the character objects C1 and C2. In this case, the pronunciation from the enemy character object C2 is blocked by the obstacle object C3, and when reaching, it is more virtual reality to control the amount of pronunciation so that it can be attenuated and heard.

  Therefore, in order to give such a preferable virtual reality, the pronunciation is controlled in accordance with the game screen situation according to the present invention.

  The sound generation control is executed by the sound generation control function unit 101 of the CPU 1 according to a program.

  FIG. 6 is an overall operation flow of control by the sound generation control function unit 101 of the main CPU 1.

  For such control, character data shown in FIGS. 7 and 8 is prepared as a table in advance in the ROM 2 as game data.

  In the table shown in FIG. 7, parameters of all characters displayed during the progress of the game program are listed. FIG. 7 shows only the sound data portion related to the present invention among the listed parameter items.

  That is, in FIG. Correspondingly, the non-sound source object is a character that does not pronounce sound, including the constituent polygon, transparency of the sound of the constituent polygon, and parameters of whether or not it is a sound source object (1 is a sound source object, 0 is a non-sound source object).

  In FIG. The character with “1” is a sound source object, and is composed of polygons P11, P12,... P1n, and the sound transparency of each component polygon is P11 = 50, P12 = 56,. Is registered.

  Here, the sound transparency is set corresponding to what structure the character is. For example, if the character is hard like a rocky mountain, it is difficult to pass through the sound, so the sound transparency is a small value. On the other hand, in the thin shape, the sound is easy to pass and the sound transparency is a large numerical value.

  In FIG. 8, for the sound source object of FIG. 7 (character distinction is set to “1” in FIG. 7), the tone color, tone pitch, and volume level are further defined as sound data. For example, the character No. shown as the sound source object in FIG. For 1, the tone color is set to AA, the pitch of the tone is set to +1 with respect to the default level, and the volume level is set to +1 with respect to the default level.

  Returning to FIG. 6, the main CPU 1 reads the sound data of the character object to be displayed from the character objects registered in the list of FIG. 7 and arranges it in the virtual three-dimensional space in accordance with the progress of the game program. The position to be performed is determined (step S1).

  At this time, the number of character objects to be displayed is not limited to one and may be plural. That is, the sound generation control unit 101 determines whether a plurality of sound source objects exist within a predetermined range from the position of the second viewpoint set in the virtual three-dimensional space. When the plurality of sound source objects exist within the predetermined range, the volume of the sound is determined for each sound source object.

  In the following description, for the sake of simplification, also in the following description of the embodiments, the character object C1 that is the operation target of the player and the one enemy character object C2 are displayed on the display device 5 so as to face each other. Take the case as an example.

  Therefore, the following processing is repeated for the number of character objects when there are a plurality of character objects to be displayed.

  The sound generation control function unit 101 refers to the table in FIG. 7 and determines whether or not the character object that is the target to be displayed is a sound source object.

  If it is a sound source object, the level of the sound of the corresponding sound source object is further extracted with reference to the table of FIG.

  Then, based on the sound level of the sound source object in which the size of a predetermined three-dimensional shape, for example, a sphere having a default diameter (hereinafter referred to as sound reference character SO) is set in the sound data. The change is determined (step S2).

  For example, the sound reference character SO displayed as a sphere having the diameter determined as described above is replaced with an enemy character object C2 located in the three-dimensional space.

  FIG. 9A and FIG. 9B are diagrams showing a state when the sound reference character SO replaced with the enemy character object C2 is placed in a three-dimensional space.

  9A shows a case where the pitch level (see FIG. 8) of the enemy character object C2 serving as the sound reference character SO1 is larger than the pitch level SS of the reference sound, and FIG. 9B shows an enemy character object serving as the sound reference character SO2. This is a case where the level of the sound C2 (see FIG. 8) is smaller than the level SS of the reference sound.

  Next, in FIG. 6, the sound generation control function unit 101 sends the rendering processor 4 to FIG. 9A (FIG. 9B) in parallel with the processing by the image data creation function unit 100 that creates image data for game screen display of the CPU 1. The sound reference character SO1 (or SO2) placed in the three-dimensional space shown is instructed for sound image generation processing for perspective transformation into a two-dimensional plane (step S3).

  The rendering processor 4 perspective-converts the sound reference character SO1 (SO2) into a two-dimensional plane corresponding to the sound source object placed in the three-dimensional space based on the sound image generation processing command (step S3).

  At that time, as shown in FIG. 10A (corresponding to FIG. 9A) and FIG. 10B (corresponding to FIG. 9B), the shape in which the two-dimensional plane is in contact with the sound reference character SO1 (SO2) (the shape of the sound reference character is a sphere). In the case of a square), for example, perspective transformation of 3D spatial data to 2D data so as to project onto a square with an area of 32 × 32 pixels to obtain 2D surface image data for sound volume determination (Step S4).

  At this time, the sound reference character SO1 (SO2) is in contact with the periphery of the square having an area of 32 × 32 pixels in the viewing direction of the sound reference character O1 (SO2) based on the second viewpoint 20 ′ in the virtual three-dimensional space. The angle of view is set to. The second viewpoint 20 ′ may be the same as the first viewpoint 20.

  Specifically, when the sound reference character SO1 (SO2) is a spherical object, the size of a circle obtained by projecting on the projection plane is the size of a 32 × 32 pixel square having the same center as the circle. Is smaller, the angle of view of the second viewpoint 20 ′ may be gradually decreased, and zoomed up until the outer circumference of the circle touches the square. On the other hand, when it is larger, the angle of view of the second viewpoint 20 'may be gradually increased until the outer periphery of the circle touches the periphery of the square.

  FIG. 11A shows the result of perspective transformation into a two-dimensional surface of 32 × 32 pixels in FIG. 10A. FIG. 11B shows the result of perspective transformation into a two-dimensional surface of 32 × 32 pixels in FIG. 10B.

  As is apparent from the comparison between FIG. 11A and FIG. 11B, as shown in FIG. 10A, the sound obstacle character object C3 is projected on the sound reference character SO1 whose sound level is larger than the reference level SS. It can be understood that the area is small.

  Returning to FIG. 6, the sound generation control function unit 101 of the CPU 1 then determines the loudness based on the sound image data that is perspective-transformed into the two-dimensional plane in step S4 (step S5).

  That is, the two-dimensional image region corresponding to the three-dimensional object in the two-dimensional image data generated by the perspective transformation overlaps with the two-dimensional image region corresponding to another object that blocks sound. Based on the volume ratio of the portion, the volume of sound generated from the sound source object is determined.

  Next, the sound having the determined magnitude is output from the speaker 10 by the sound source device 9 (step S6).

  In the above processing, step S4 for performing image conversion for sound volume determination and a procedure for determining the sound volume (step S5) will be described in more detail below.

  The details of step S4 for performing image conversion for sound volume determination are shown in the flow of FIG.

  As shown in FIG. 11A (FIG. 11B), the area 25A obtained by perspective-transforming the sound reference character SO1 (SO2) into a two-dimensional plane is in contact with the side of the square area 24 of 32 × 32 pixels.

  In FIG. 12, the rendering processor 4 has a circular shape corresponding to the sound source object SO1 (SO2) in the “image data for sound volume determination” projected onto the two-dimensional surface as shown in FIG. 11A (FIG. 11B). The texture is pasted on the perspective transformation area 25A in an arbitrary color, for example, red (step S40).

  Next, with respect to an object having high opacity with respect to sound such as terrain that blocks sound, an object having the shape of a house roof in the example shown in FIG. 11A (FIG. 11B), complete sound shielding for each polygon constituting the object. The portion of the polygon (opaque polygon) to be formed is rendered black on the “image data for sound volume determination” rendered in red in step S40 (step S41).

  Furthermore, for each polygon constituting the object that blocks the sound, a polygon (semi-transparent polygon) that is incompletely sound-blocking, for example, in the example shown in FIG. 11A (FIG. 11B), the shape of the main body of the house Black translucency is drawn on the portion of the “image data for determining sound volume” rendered in red in step S40 (step S42).

  Here, let us consider the addition of colors. In RGB display, the red level is represented by (255,0,0) and the black level is represented by (0,0,0). Here, when the color level is normalized to 0 to 1 for easy understanding, red is represented by (1,0,0) and black is represented by (0,0,0).

  Therefore, the color addition process is as follows.

  When a color to be filled with a certain opacity is superimposed on the original color (already written color), the resulting color is [original color (already written color) * (1-opacity) + Color to fill * Opacity].

“Dest” the original color (already written)
“Source” the color to fill
When the opacity (0 to 1) of the fill color is expressed as “sourceα”,
The color S 'as a result of the addition is
S ′ = dest * (1-sourceα) + source * sourceα. This is the same as a normal translucent calculation.

For example, if the original color is red (1), black (0), opaque (1),
S ′ = 1 * (1-1) + 0 * 1 = 0 + 0 = 0 (result is written as 0)
If the original color is red (1), black (0), opaque (0.5),
S ′ = 1 * (1-0.5) + 0 * 0.5 = 0.5 + 0 = 0.5 (result is written at 0.5)
Then, regarding the addition of this color, only red is simply added from black (0,0,0) to red (1,0,0). The process of translucency α is also performed at the added portion of steps S41 and S42.

  Next, the red R used in steps S41 and S42 is added to the “virtual image data for determining the sound volume” for 32 × 32 pixels (step S43).

  That is, the addition value of the pixel in the area rendered in red is 255, and the addition value of the pixel in the area rendered in black is 0.

  Further, details of the procedure for determining the loudness (step S5) are shown in the flow of FIG.

  With regard to the processing in step S5, the rendering value is added in red in the circular area 25A of the sound reference character in the 32 × 32 dot two-dimensional image when there is nothing to block the sound in FIG. 11A (FIG. 11B). The value S is registered in the ROM 2 in advance.

  The sound generation control function unit 101 reads “a value S corresponding to a circular area in a 32 × 32 dot image when there is nothing to block” from the storage unit (ROM 2) (step S50).

  Next, in step S43, a value S 'obtained by adding a color component to the rendered "image data for determining sound volume" is read (step S51).

  Then, a ratio S ′ / S between the value S ′ and the corresponding reference value S registered in advance in the ROM 2 corresponding to the circular area 25A of FIG. 11A (FIG. 11B) is calculated.

  With this ratio S ′ / S, it is possible to obtain the transmittance PS that has not finally blocked the sound (step S52).

  Next, a result obtained by multiplying the “reference volume level” set for the character object set first by this transmittance PS is obtained (step S53).

  Therefore, the result of step S53 is sent to the sound source device 10 as the volume level determined to be output from the speaker, and the sound generation of the controlled volume is output from the speaker 10 (FIG. 6: step S6).

It is a functional block diagram of the structural example of the game device main body to which this invention is applied. It is a figure which shows the example of two hostile character objects C1 and C2 located in virtual three-dimensional space according to a battle game program with a game device. It is a figure which shows the state by which the two-dimensional surface image data of the game image was displayed on the display surface 22 of the display apparatus 5. FIG. It is a figure which shows the example in which a sound disorder | damage | failure object is located between the two character objects C1 and C2 located in the virtual three-dimensional space corresponding to FIG. It is a figure which shows the state displayed on the display surface of a display apparatus corresponding to FIG. It is the whole operation | movement flow of the Example of the control of the pronunciation according to this invention. It is a figure which shows the partial data of the table which lists the parameter of all the characters displayed during the progress of a game program. FIG. 8 is a diagram showing data of a table that further defines the tone color, tone pitch, and volume level of the sound source character of FIG. It is a figure which shows the state when the sound reference character SO replaced with the enemy character object C2 in the case where it is larger than the high and low level SS of the reference sound is placed in the three-dimensional space. It is a figure which shows a state when the sound reference character SO replaced with the enemy character object C2 in the case where it is smaller than the high and low level SS of the reference sound is placed in the three-dimensional space. FIG. 9B is a diagram for explaining that image conversion for sound volume determination is performed by perspective-transforming three-dimensional space data into two-dimensional data, corresponding to FIG. 9A. FIG. 9C is a diagram for explaining that image conversion for sound volume determination is performed by perspective-transforming three-dimensional space data into two-dimensional data, corresponding to FIG. 9B. FIG. 10A shows the result of perspective transformation into a two-dimensional surface of 32 × 32 pixels. FIG. 10B shows the result of perspective return to a two-dimensional surface of 32 × 32 pixels in FIG. 10B. It is a flowchart which shows the detail of step S4 which performs image conversion for the loudness determination in FIG. It is a flowchart which shows the detail of step S5 which determines the loudness in FIG.

Explanation of symbols

1 Main CPU
100 Image data creation function unit 101 Sound generation control function unit 2 ROM
3 RAM
4 Rendering processor 5 Display device 6 Texture memory 7 Video RAM (VRAM)
8 Input device 9 Sound source device 10 Speaker 11 Bus

Claims (5)

The two-dimensional image generated by arranging a first viewpoint and an object in a virtual three-dimensional coordinate space, generating a two-dimensional image obtained by perspective-transforming the state of the virtual three-dimensional space including the object viewed from the first viewpoint. Is a program for causing a computer to execute a process of displaying a sound on a display unit and a process of outputting a sound preset for the object to a sound output unit,
In the computer,
Means for reading sound data stored in a storage unit in association with a sound source object set in advance to generate sound, and determining a size of a predetermined three-dimensional object corresponding to the sound source data;
The three-dimensional object having the determined size is arranged in a virtual three-dimensional space in correspondence with the sound source object, and the viewing direction with respect to the three-dimensional object from the position of the second viewpoint in the virtual three-dimensional space The two-dimensional image data for determining the volume of sound is generated by perspective-transforming the three-dimensional object into a two-dimensional plane so that the angle of view is set and the image size is set in advance. Means,
The size ratio of the overlapping part of the two-dimensional shape image region corresponding to the three-dimensional shape object in the generated two-dimensional image data and the two-dimensional shape image region corresponding to another object that blocks sound Means for determining a loudness generated from the sound source object, based on
Function as means for outputting the sound generated by the sound source object from the sound output unit at the determined sound volume;
A program characterized by that. In claim 1,
The predetermined three-dimensional object has a spherical shape, and the two-dimensional image obtained by perspective transformation corresponding to the predetermined three-dimensional object is circular. In claim 1,
The position of the second viewpoint is set to coincide with the position of the first viewpoint,
The viewing direction from the second viewpoint is set to the direction toward the object,
The angle of view of the second viewpoint is set so that a circle touches the periphery of the image size.
A program characterized by that. In claim 2 or 3,
Means for determining whether the computer further includes a plurality of sound source objects within a predetermined range from the position of the second viewpoint set in the virtual three-dimensional space;
When the plurality of sound source objects exist within the predetermined range, the volume of the sound is determined for each sound source object.
A program characterized by that. In claim 1,
The means for determining the magnitude of the sound generated from the sound source object is a transparency with respect to the sound of the other object that blocks the sound with respect to an overlapping portion with a two-dimensional image area corresponding to the other object that blocks the sound. A program characterized by multiplying as a coefficient.