JPH06233395A - Video game authoring system - Google Patents

Abstract

PURPOSE:To obtain a game cassette from which an acoustic effect conforming to a real sound is obtained through the efficient edit. CONSTITUTION:A signal subjected to sound image processing by a sound image localization processing means comprising a convolver is reproduced from a couple of separated speakers together with a picture and a listener feels in a manner that the sound image is localized to a position different from the position of the speaker and is used by a visual game equipment in this authoring system editing a game software suitable for the visual game equipment. Then a pattern production section 100, an audio data production section 200, a sound image localization production section 300 and a conversion information generating section 600 supply audio data, audio data conversion information to revise the time base of the audio data, sound image localization information and picture data respectively to an editer 400, from which a gate software arranged in a prescribed format is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video game authoring system, and more particularly to a reproduced image and a listener using a pair of transducers (speakers) for reproduction so that the listener differs from the position of the transducer. 180
The present invention relates to a video game authoring system for editing game software used in a television game machine or the like which makes a user feel that a sound image is localized at a desired position more than one degree.

[0002]

2. Description of the Related Art Conventionally, there is a sound image localization method using a binaural technique in which a sound source is felt at a specific position (direction) by the level difference and phase difference (time difference) of signals in both ears. As a sound image localization method using an analog circuit, for example, Japanese Patent Application Laid-Open No. 53-140001 (Japanese Patent Publication No.
58-3638) There is a sound image localization method described in the official gazette. This is because the analog filter emphasizes / attenuates the level of a specific frequency band (amplitude control) to give a sense of front and back of the sound source, and the analog delay causes a time difference between the left and right sounds (phase control). It was intended to give a sense of left and right.

Further, with the recent development of digital processing technology, there is a sound image localization method realized by a digital circuit, for example, there is a "sound image forming method and its device" described in Japanese Patent Laid-Open No. 2-298200. In the sound image localization method using this digital circuit, the signal from the sound source is processed by FFT (Fast Fourier Transform).
Transform), processing on the frequency axis and giving a level difference and a phase difference depending on the frequency to the left and right channel signals to digitally control the localization of the sound image. The frequency-dependent level difference and phase difference at each sound image localization position of this method are collected as experimental data using an actual listener.

However, in the sound image localization method using such a circuit, if the sound image is to be localized accurately and precisely,
The circuit scale was extremely large, and it was only used as a recording system for special business purposes. The authoring system for editing the software used in these devices also naturally supplies and edits audio data and image data corresponding to these.

[0005]

By the way, recently,
Amusement game machines and computer terminals using virtual reality have appeared. In these game machines and terminals, more realistic sound image localization is being demanded together with images. Moreover, for example, in a game machine (or terminal device), the flight course (position) of the airplane varies depending on the operation of the operator, and the sound image is changed in real time according to the operation of the operator. It becomes necessary to perform a moving process and reproduce it. This is a point that is significantly different from the sound image localization processing for records described above.

[0006] The applicant has responded to such a demand as follows.
First, the circuit scale of the device is small, the cost is low, and
We proposed a sound image localization control method that can localize a sound image together with an image in a wide space exceeding 180 degrees.
The present invention provides a video game authoring system for editing suitable game software which is used in a device adopting such a sound image localization control method.

[0007]

In order to solve the above problems, the present invention is to provide a video game authoring system having the following constitutions 1) to 8). 1) Sound image localization processing means including a convolver to which a predetermined image is reproduced based on incoming video display data, and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a sound-image-processed signal and makes a listener feel as if the sound image is localized at an arbitrary moving position different from the pair of transducers, and includes sound source data. The supplied audio data, the audio data conversion information for changing the audio data and the sound image localization information indicating the localization position of the sound image are supplied, and a predetermined image is provided together with the game software. A television game authoring system for editing, corresponding to sound image information designating means for designating the sound image localization information, and corresponding to the moving direction and speed of the sound image with respect to the listener. And a conversion information generating means for generating the audio data conversion information.

2) Incoming video display data
A predetermined image is reproduced on the basis of the audio data, and a pair of transducers separated from each other reproduces a signal subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied, and reproduces the signal to a listener. The pair of trans
It is used in a television game machine that makes a user feel that a sound image is localized at an arbitrary moving position different from that of the sound source, and includes audio data including sound source data and sound image localization information indicating the localization position of the sound image. A television game operating system for editing game software together with a predetermined image supplied thereto, comprising sound image information designating means for designating the sound image localization information, and listening to the sound image movement information. And a conversion processing means for giving the audio data a time-axis change corresponding to the moving direction and speed of the sound image with respect to the person.

3) Incoming video display data
A predetermined image is reproduced based on the audio signal, and a pair of transducers separated from each other reproduces a signal subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied, and reproduces the signal to a listener. The pair of trans
This is used for a television game machine that makes the user feel that a sound image is localized at an arbitrary moving position different from that of the sounder, and includes audio data including sound source data and an audio data for changing this audio data. A television game audio for editing the game software together with a predetermined image supplied with the video data conversion information and the sound image localization information indicating the localization position of the sound image.
In the saling system, the starting point and the ending point of the sound source are specified by means for specifying the sound image localization information and the moving speed between them, and the starting point is based on a relationship predetermined by an input from the specifying means. And sound image localization information generating means for generating sound image localization information at an intermediate point of the end point, and conversion information generating means for generating the audio data conversion information corresponding to the sound image localization information at the intermediate point. A TV game authoring system characterized by being made up.

4) Incoming video display data
A predetermined image is reproduced based on the audio signal, and a pair of transducers separated from each other reproduces a signal subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied, and reproduces the signal to a listener. The pair of trans
It is used in a television game machine that makes a user feel that a sound image is localized at an arbitrary moving position different from that of the sound source, and includes audio data including sound source data and sound image localization information indicating the localization position of the sound image. A television game authoring system for editing game software together with a predetermined image supplied, for specifying sound image localization information of the start point and end point of the sound source and a moving speed between them. Of the sound image localization information generating means for generating sound image localization information at the intermediate point between the start point and the end point, and listening to the sound image movement information at the intermediate point, based on the relationship predetermined by the input from the designating means. And a game processing means for imparting to the audio data a time base change corresponding to the direction and speed of movement of the sound image with respect to a person. ·system.

5) Incoming video display data
A predetermined image is reproduced based on the audio signal, and a pair of transducers separated from each other reproduces a signal subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied, and reproduces the signal to a listener. The pair of trans
This is used for a television game machine that makes the user feel that a sound image is localized at an arbitrary moving position different from that of the sounder, and includes audio data including sound source data and an audio data for changing this audio data. A television game audio for editing the game software together with a predetermined image supplied with the video data conversion information and the sound image localization information indicating the localization position of the sound image.
A saling system, wherein the sound image localization information and sound image designating means for designating the distance of the sound image to the listener, and the audio data conversion information correspond to the moving direction and speed of the sound image to the listener. And a converted information generating means for generating the generated information.

6) Incoming video display data
A predetermined image is reproduced based on the audio signal, and a pair of transducers separated from each other reproduces a signal subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied, and reproduces the signal to a listener. The pair of trans
It is used in a television game machine for making a user feel that an arbitrary moving position different from that of a sound source is localized, and is supplied with audio data including sound source data and sound image localization information indicating the localization position of the sound image. A television game authoring system for editing game software together with a predetermined image, and sound image specifying means for specifying the sound image localization information and the distance of the sound image to a listener, Television game operating, comprising conversion processing means for giving the audio data time-axis changes corresponding to the moving direction and speed of the sound image with respect to the listener of the sound image localization information. ·system.

7) Incoming video, display data
A predetermined image is reproduced on the basis of the audio signal, and a sound image localization signal having a convolver to which a monaural sound source is supplied is reproduced from a pair of transducers that are separated from each other, and the pair of transformers is reproduced to the listener. It is used for a television game machine that makes the user feel that the sound image is localized at an arbitrary moving position different from the user, and to change the audio data including the sound source data and this audio data. Oh
The television game authoring for editing the game software together with the predetermined image supplied with the audio data localization information indicating the localization position of the sound image and the gain information for changing the sound volume. In the system, the sound source localization information of the start point and the end point of the sound source, the designation means for designating the movement locus between them and the movement speed between them, and based on the relationship predetermined by the input from the designation means. , Sound image localization information generating means for generating sound image localization information at the intermediate point between the start point and the end point, and conversion information generating means for generating the audio data conversion information corresponding to each of the sound image localization information, And a gain information generating means for generating the gain information corresponding to each of the sound image localization information.

8) Incoming video display data
A predetermined image is reproduced on the basis of the audio signal, and a signal which has been subjected to sound image processing by a sound image localization processing means equipped with a convolver to which a monaural sound source is supplied is reproduced from a pair of spaced transducers, and a listener is reproduced. Used for a television game machine that makes the user feel that a sound image is localized at an arbitrary moving position different from the pair of transducers, and the audio data including sound source data and the localization position of the sound image are used. Is a television game operating system for editing the game software together with the sound image localization information indicating the sound image localization information of the start point and the end point of the sound source and between them. Based on the relationship between the locus of movement and the moving speed between these points, and the relationship predetermined by the input from the above-mentioned specifying means, the sound image localization information of the intermediate point between the start point and the end point is generated. Sound image localization information generating means, and a change in time axis corresponding to the moving direction and speed of the sound image to the listener of the sound image movement information at the intermediate point and a gain change for changing the sound volume are given to the audio data. A television game authoring system comprising a conversion processing means.

[0015]

EXAMPLES Video game authoring according to the present invention
An embodiment of the system will be described below with reference to the drawings.
First, a sound image localization control method capable of localizing a sound image in a wide space exceeding 180 degrees by using game software edited by the video game authoring system of the present invention. The basic principle of is explained. This is a technique for localizing a sound image at an arbitrary position in space by using a pair of transducers (which will be described below by taking a speaker as an example) arranged apart from each other.

FIG. 6 is a principle diagram of sound image localization. sp1,
sp2 is a speaker arranged in front of and on the left and right of a listener (sometimes referred to as a listener in the embodiment), sp2
1 to h1L the head-related transfer characteristic (impulse response) from the listener's left ear to h1R, sp
The head-related transfer characteristics from 2 to the left and right ears are h2L and h2R. In addition, the head-related transfer characteristics to the left and right ears of the listener when the actual speaker is placed at the target localization position x are pL.
x, pRx. Here, each transfer characteristic is obtained by actually measuring with a speaker in a sound space and a microphone at both ears of a dummy head (or human head), and subjected to appropriate waveform processing.

Next, signals obtained by passing the sound source X to be localized through signal converters cfLx and cfRx (transfer characteristics by a convolver) are sp1 and sp, respectively.
Consider playing in 2. At this time, when the signals obtained in the left and right ears of the listener are eL and eR, eL = h1L · cfLx · X + h2L · cfRx · X (Equation 1) eR = h1R · cfLx · X + h2R · cfRx · X (〃)

On the other hand, when the signals obtained at the left and right ears of the listener when the source X is reproduced from the target localization position are dL and dR, dL = pLxX (Equation 2) dR = pRxX (〃)

Here, if the signals obtained in the left and right ears of the listener by reproduction of sp1 and sp2 match the signal when the source is reproduced from the target position, the listener is as if the speaker exists at the target position. The sound image will be recognized. From these conditions eL = dL, eR = dR and (Equation 1) and (Equation 2), X is deleted and h1L · cfLx + h2L · cfRx = pLx (Equation 3) h1R · cfLx + h2R · cfRx = pRx (Equation 3) CfLx, cfRx is calculated from cfLx = (h2R · pLx−h2L · pRx) / H (Equation 4a) cfRx = (− h1R · pLx + h1L · pRx) / H (〃) where H = h1L · h2R-h2L · h1R (Formula 4b)

Therefore, if a signal to be localized is processed by a convolver (convolution operation processing circuit) or the like using the transfer characteristics cfLx and cfRx calculated by (Equation 4a) and (Equation 4b), a sound image is obtained at a target position x. Can be localized. There are various concrete methods for realizing the signal conversion apparatus, but it is sufficient to use an asymmetric FIR digital filter (convolver). The final transfer characteristic when used in the FIR digital filter is a time response function.

That is, as the transfer characteristics cfLx and cfRx at the required localization position x, (Equation 4a) and (Equation 4b)
The coefficients cfLx and cfRx are created in advance as the coefficients for realizing the one obtained in step 1 by the FIR filter processing, and are prepared as ROM data.
If the coefficient of the required sound image localization position is transferred from the ROM to the FIR digital filter and the signal from the sound source is subjected to the convolutional arithmetic processing and reproduced from the pair of speakers, the sound image is localized at any desired position.

Next, a method of controlling the sound image localization based on such a principle will be described in detail with reference to FIG. FIG. 7 shows the steps of the method for controlling the sound image localization. Measurement of Head Related Transfer Function (hereinafter referred to as HRTF) (Step 1)

This will be described with reference to FIGS.
FIG. 8 shows an HRTF measurement system. A pair of microphones ML and MR are installed in both ears of the dummy head (or human head) DM, the measurement sound from the speaker SP is received, and the source sound (reference data) r is recorded in the recorder DAT.
efL and refR and the to-be-measured sound (measurement data) L and R are recorded in synchronization.

The source sound XH may be impulse sound, white noise, or other noise. In particular, from the viewpoint of statistical processing, white noise is
Since it is a continuous sound and the energy distribution is constant over the audio band, it is possible to use SN by using white noise.
The ratio is improved. The position of the speaker SP is set to a plurality of angles θ (for example,
As shown in FIG. 10, it is installed at 12 points every 30 degrees and continuously recorded for a predetermined time.

HRTF impulse response (Impulse Re
sponse; hereinafter referred to as IR) (step 2) In step 1, the source sounds (reference data) refL and refR and the to-be-measured sounds (measurement data) L and R, which are synchronously recorded, are transferred to the workstation ( (Not shown).

Letting X (S) be the frequency response of the source sound (reference data), Y (S) be the frequency response of the sound to be measured (measurement data), and IR (S) be the frequency response of the HRTF at the measurement position. There is an input / output relationship shown in Expression 5). Y (S) = IR (S) · X (S) (Equation 5) Therefore, the frequency response of the HRTF is IR (S) = Y (S) / X (S) (Equation 6) is there. Therefore, the frequency response X (S) of the reference,
The frequency response Y (S) of the measurement data is obtained by cutting out the data obtained in step 1 using a window synchronized in time, and
A finite Fourier series expansion is performed by FT transformation to calculate as a discrete frequency. From (Equation 6), the frequency response I of the HRTF is
R (S) is calculated by a known calculation method.

In this case, the accuracy of IR (S) is increased (S
In order to improve the N ratio), IR (S) may be calculated for several hundred windows that are temporally different, and they may be averaged. Then, the calculated HRTF frequency response IR (S)
Is subjected to inverse FFT conversion to obtain a time axis response (impulse response) IR (first IR) of HRTF.

IR (impulse response) shaping processing (step 3) Here, the IR obtained in step 2 is shaped. First, for example, by FFT conversion, the first IR obtained in step 2 is expanded at discrete frequencies over the audio spectrum,
An unnecessary band (a large dip occurs in the high range, but this is unnecessary that does not affect the sound image localization),
It is removed by BPF (band pass filter). By limiting the band in this way, unnecessary peaks and dips on the frequency axis are removed and unnecessary coefficients do not occur in the cancel filter, so that the convergence is improved and the coefficients can be shortened.

Then, the band-limited IR (S) is inverted F
FT conversion is performed, and IR (impulse response) is cut out on the time axis by a window (for example, a cosine function window) and subjected to window processing (second IR). By performing the window processing, the effective length of the IR is not long, the convergence of the cancel filter is improved, and the sound quality is not deteriorated.

Cancellation filters cfLx, cfRx
(Step 4) The cancel filters cfLx and cfRx, which are convolvers (convolutional integration circuits), are cfLx = (h2R · pLx−h2L · pRx), as shown in (Equation 4a) and (Equation 4b) described above. / H (Formula 4a) cfRx = (-h1R.pLx + h1L.pRx) / H (〃) However, it is H = h1L.h2R-h2L.h1R (Formula 4b).

Here, the speakers sp1 and sp to be arranged are arranged.
Head related transfer characteristics h1L, h1R, h2L, h2R
And, as the head-related transfer characteristics pLx, pRx when the actual speaker is placed at the target localization position x, the second IR (which has been shaped and processed for each angle θ obtained in steps 1 to 3 above). Impulse response).

The head-related transfer characteristics h1L and h1R correspond to the position of the L-channel speaker in FIG. 11, and if they are installed at 30 degrees (θ = 330 degrees) from the front to the left, θ = 330. The IR of the degree is used. Head-related transfer characteristic h2
R and h2L correspond to the position of the R channel speaker in the figure, and are, for example, 30 degrees (θ = 30) from the front to the right.
, The IR of θ = 30 degrees is used (that is, the system at the time of actual sound image reproduction (for example, FIG. 9).
Select one that is close to)).

The head-related transfer characteristics pLx and pRx are 90 degrees from the front, which is the target sound source localization position.
The cfLx of the entire space corresponding to that by substituting IR for every 30 degrees in a wide space (overall space) beyond that of 180 °
cfRx, that is, 12 sets of cancellation filters cfLx and cfRx are obtained every 30 degrees (in FIG. 11, a position of 240 degrees is taken as an example). The cancel filters cfLx and cfRx group are finally obtained as IR (impulse response) which is a response on the time axis.
Note that the cancel filter cfLx according to (Equation 4a),
The calculation of cfRx is as follows. First, H ^-1 , which is a kind of inverse filter for H in (Equation 4b), is obtained by the least squares method, and this is subjected to inverse FFT conversion to obtain a time function h (t). In addition, the terms h1L, h1R, h2L, and p of (Equation 4a) are
By expressing Rx, pLx, and h2R by time functions, respectively, the following equation is established.

CfLx (t) = (h2R · pLx−h2L · pRx) · h (t) (Formula 7) cfRx (t) = (− h1R · pLx + h1L · pRx) · h (t) (〃)

Therefore, the coefficients of the cancel filters cfLx and cfRx can be obtained from these (formula 7). As is clear from (Equation 7), in order to shorten the coefficients of the cancel filters cfLx and cfRx, the head-related transfer characteristics h1L, h1R, h2L, pRx, pLx,
It is extremely important to shorten each h2R. Therefore, as described above, various processes such as the window process and the shaping process are performed in steps 1 to 3 to shorten the head-related transfer characteristics h1L, h1R, h2L, pRx, pLx, h2R.

Scaling of Cancellation Filter at Each Localization Point x (Step 5) Further, the spectrum distribution of the sound source (source sound) actually processed by the convolver (cancellation filter) is like pink noise statistically. There are things that are distributed or that drop gently in the high range.In any case, since the sound source is different from a single sound, there is a risk that overflow will occur when convolution calculation (integration) is performed and distortion will occur. is there. Therefore, in order to prevent the overflow, the maximum gain (for example, the cancel filter cfL) among the coefficients of the cancel filters cfLx and cfRx.
x, cfRx), and all coefficients are scaled so that overflow does not occur when the coefficient and the white noise of 0 db are convoluted.

Scaling is performed in this way,
Finally, a group of data supplied to the convolver as coefficients (in this example, 12 sets of convolver coefficient groups capable of sound image localization every 30 degrees) cfLx and cfRx are obtained.

A signal from a sound source is convoluted and reproduced (step 6) For example, as a sound reproducing device for a game machine, as shown in FIG. 9, a pair of speakers are spaced 30 degrees left and right around a listener M. Sp1 and sp2 are provided, and the pair of speakers sp1 and sp2 are configured to reproduce the acoustic signals processed by the pair of convolvers (convolution operation processing circuits).

A signal from the same sound source X (for example, a monaural sound such as a flight sound from a game synthesizer) is supplied to the pair of convolvers, and the above-mentioned step 1 is performed.
The IR coefficients cfLx and cfRx created in 05 (for example, when the sound image is to be localized at a position of 120 degrees to the left rear (θ = 240 degrees), the coefficient of θ = 240 degrees) are selected and the convolver is selected. Is set to. For example, based on a sound image localization command from a main CPU (central processing unit) of a game machine or the like, the control sub CPU transfers the coefficient of the desired localization position from the coefficient ROM to the pair of convolvers.

In this way, the signal from the sound source X is subjected to the convolution operation processing on the time axis by the pair of convolvers, and the pair of speakers sp1 and sp1 arranged apart from each other.
Played from 2. The sound reproduced from the pair of speakers sp1 and sp2 has its sound image localized such that the crosstalk to both ears is canceled and the sound source is located at a desired position, and is heard by the game operator (listener) M. Played as an extremely realistic sound. As for the coefficient of the convolver, the optimum sound image position is sequentially selected and switched with the transition of the movement of the airplane according to the operation of the operator. Further, when the flight sound is changed to, for example, a missile sound, the source sound from the sound source X is changed from the flight sound to the missile sound. In this way, the control method can freely localize the sound image at any position. Incidentally, the sound image is reproduced according to the image like a general game machine, but the explanation of the image processing system is omitted.

Further, in addition to the sound image localization described above, it is possible to make the sound more realistic by taking into consideration the change of the pitch (pitch) and the change of the sound volume due to the Doppler effect accompanying the movement of the sound image. Television game authoring of the present invention
The system also includes data for utilizing this Doppler effect, and next, this Doppler effect will be described in advance. FIG. 5 is a drawing for explaining the Doppler effect. In the same figure, let us consider a case where the sound source passes through the side (distance d) of the listener M at a constant velocity (velocity v). First, with respect to the change in direction localization, the change becomes faster as it approaches the side of the listener M. If the passing time right beside is set to t = 0 and the direction localization θ is represented counterclockwise with respect to the side just next to the listener M, the direction localization θ and the time t are expressed by θ = −arctan (vt / d) Relationship is established.

For changes in pitch (pitch),
Under the influence of the Doppler effect, letting f be the frequency of the sound actually emitted and F is the frequency audible to the ear, then F = fc / (cv-sin θ), where c is the speed of sound. That is, the pitch (pitch) ratio N
Becomes N = c / (cv-sin θ). Further, regarding the sound volume, if the sound volume at the time of passing directly beside is g, the audible sound volume G is G = g · cos θ
Becomes

Therefore, as shown in FIG. 5, as the sound source moves, the direction localization θ is represented by a discrete value every 22.5 degrees, and line segments A, B, C, D, E, F, The same pitch conversion is applied to G and the like. The pitch conversion ratio N is given by the above equation.
The length of each line segment corresponds to the length of time that pitch conversion is performed. In order to bring the moving sound image closer to the real sound, it is sufficient to take the above points into consideration, and the television game authoring system of the present invention aims to achieve this Doppler effect. -It is designed to include data.

The game software created from the television game authoring system of the present invention is used in a game machine or the like equipped with the sound image localization processing means composed of the above convolver. Each embodiment of the TV game authoring system will be sequentially described. FIG. 1 is a schematic block diagram showing a first embodiment of the television game authoring system. Reference numeral 100 is a screen production unit for generating image data of characters such as airplanes, and 200 is audio data for generating audio data such as music, sound effects, and voice. The production unit 300 is the moving angle θ and the distance d of the sound image with respect to the listener (operator).
A sound image localization production unit for designating a sound source, a conversion information generation unit 600 for generating change information (pitch ratio N) on the time axis of the sound source based on information from the sound image localization production unit 300,
Reference numeral 400 is an editor for obtaining each of the above-mentioned information and for arranging the information in a predetermined arrangement. 500 is a soft recorder for FDD, ROM writer, CD, magneto-optical disk drive, etc.
It's Da. This television game authoring system has the above configuration.

Next, this operation will be described. Image data such as an airplane from the screen production unit 100, and sound source data such as music and sound effects from the audio / data production unit 200.
Data is supplied to the editor 400, respectively. On the other hand, in the sound image localization producing section 300, sound image localization information indicating the moving movement angle θ of the sound image with respect to the listener of the sound source data and the distance d of the sound image with respect to the operator is generated. The sound image localization information is supplied to the editor 400 and the conversion information generation unit 600. The conversion information generation unit 600 detects movement of the sound image or relative position change based on the incoming sound image localization information, and changes information on the time axis,
That is, the pitch (pitch) ratio information is generated, and the information
Supply to 00. In this editor 400, the above-mentioned data
The data is obtained, arranged in sequence according to the format of a predetermined data structure which can be reproduced synchronously, and supplied to the soft recorder 500 together with the program representing the story of the game. In the soft recorder 500, it is recorded on a desired recording medium mounted on the soft recorder 500.

Therefore, on this recording medium, image data for reproducing a picture of an airplane, audio data as sound source information such as musical instrument sound and sound effect, and sound image are presented to the listener. Sound image localization information indicating a moving position and the like when moving, and time axis conversion information (pitch ratio information, time length information for pitch conversion, etc.) corresponding to this sound image movement are recorded.

Then, this recording medium is packaged in a cartridge for a game or the like and used in, for example, a game machine having the above-mentioned structure. Based on the above, the predetermined signal processing is performed. As a result, the sound corresponding to the image is localized and reproduced from the pair of speakers over 180 degrees. Moreover, since the sound image at this time is reproduced with a change in the pitch assuming the Doppler effect, the sound more realistically reproduced is reproduced. Further, the audio data producing section 200 may be synchronized with the sound image localization producing section 300 by a control signal such as MIDI.

FIG. 2 is a schematic block diagram showing a second embodiment of the television game authoring system. The same components as those of the first embodiment are designated by the same reference numerals and the description thereof will be omitted. Also in this embodiment, as in the case of the first embodiment, an audio data production unit 201 for generating audio data such as music, sound effects, and voice is provided. But their functions are different. That is, the audio data production section 201 generates sound source data such as music and sound effects, and supplies this data to the editor 400 and also to the sound image localization production section 301, respectively. It The sound image localization producing unit 301 detects movement of the sound image or relative position change based on the supplied sound image localization information, and generates change information on the time axis, that is, pitch (pitch) ratio information, etc. Is supplied to the audio data production section 201.
On the other hand, the moving angle θ and the distance d of the sound image with respect to the listener are specified based on the sound image localization information, and this information is edited
Supply to. The audio data producing unit 201 receives the pitch (pitch) ratio information, and the internal conversion processing unit generates movement data of the sound image or generates change data on the time axis corresponding to the relative position change, and the editor 400 Supply to.

That is, the audio data production section 201 creates new audio data corresponding to this change information by time-axis converting sound sources such as music and sound effects. For example, when the ratio (N: n) of the sound source to the frequency n as shown in FIG. 4A is smaller than 1 (when the sound source is approaching the operator), it is shown in FIG. If the ratio (N: n) of the original sound source to the frequency n is greater than 1 (when the sound source moves away from the operator), it is converted into compressed data as shown in FIG. Converted to decompressed data. Thus, the conversion of the time axis involves a change in pitch (pitch) and a change in data length. Then, this time-axis converted data is supplied to the editor 400, where the incoming data is arranged so as to have a predetermined format, as described above, and the next stage. The specified game software is created by being recorded on the recording medium in the soft recorder 500. According to this embodiment, since the pitch (pitch) ratio information is not used in the editor 400, it is effective when the pitch (pitch) conversion means is not provided inside the game machine. In addition, here, the screen production unit 1
00 to the audio data production section 201 may be synchronized with a synchronization signal such as a tie code.

FIG. 3 is a schematic block diagram showing a third embodiment of the TV game authoring system. Reference numeral 202 is an audio data production section, and 303 is a sound image localization production section. Particularly, in this embodiment, the sound image localization information at the start point and the end point of the sound image movement and the start point / end point designating means 404 for designating the speed of movement therebetween, and the sound image localization information intermediate between the start point and the end point in the editor 400 are provided. Similarly to the sound image localization information generating means 402 for generating, in the editor 400, the time base conversion information corresponding to the intermediate sound image localization information and the sound image localization information of the start point and the end point is produced as audio data. Conversion processing means 4 for supplying to the section 202
03 and 03 are provided respectively.

Here, the start point / end point designating means 404 may be means for designating the sound image localization information of the start point and the end point of the sound image, the moving speed between them, and the path (moving locus) between them. That is, it is a path of the direction localization θ of the sound image to the listener and a constant radius r when forming a circular trajectory, or the direction localization θ and the distance d (straight line) from the listener. in this case,
The editor 400 is provided with a gain data generating means 401 for giving a predetermined gain to the sound source data in order to determine a sound volume with respect to a change in sound pressure, and is supplied to the audio data production section 202. Is also good. Thereby, the sound image localization information (direction localization θ, distance d) from the start point to the end point can be automatically generated.

Further, the conversion information generating means 600 and the sound image designating means 405 may be provided. In this case, the sound image designating means 405 drives the gain data generating means 401 and the sound image localization information generating means 402 of the editor 400,
Audio data production unit 202, sound image localization control unit 303
The changed data can be supplied to each of the
It is possible to supply the data as the conversion information from the conversion information generating means 600 without converting the data by the conversion processing means 403. Here, the sound image designating means 405 may be input by a remote joystick. Also,
The screen production unit 100 and the audio data production unit 202 may be synchronized with each other by a synchronizer (not shown).

Further, it is also possible to separately provide gain data separately. The gain data in this case is used to change the gain of the convolver of the game machine, and the volume is changed by changing the gain of the convolver. In the above description, the term "television game" generally refers to not only an ordinary television game but also a device in which images and sounds are interactively operated.

[0054]

[Detailed Description of the Invention] [Brief description of drawings]

FIG. 1 is a schematic block diagram showing a first embodiment of a television game authoring system according to the present invention.

FIG. 2 is a schematic block diagram showing a second embodiment of the television game authoring system according to the present invention.

FIG. 3 is a schematic block diagram showing a third embodiment of a television game authoring system according to the present invention.

FIG. 4 is a diagram for explaining a time axis change of a sound source.

FIG. 5 is a diagram for explaining a Doppler effect.

FIG. 6 is a configuration diagram for explaining the principle of sound image localization control.

FIG. 7 is a chart showing steps of a sound image localization control method of a sound image localization control device for a video game according to the present invention.

FIG. 8 is a configuration diagram showing an HRTF (head related transfer function) measurement system.

FIG. 9 is a basic configuration diagram of a sound image localization control device.

FIG. 10 is a diagram for explaining the points of HRTF measurement.

FIG. 11 is a diagram for explaining a calculation example of a cancel filter.

[Explanation of symbols]

1 Step of measuring head related transfer function (HRTF) 2 Step of calculating impulse response of HRTF 3 Step of shaping IR (impulse response) 4 Step of calculating cancel filters cfLx and cfRx 5 Step of scaling cancel filter 6 Steps for convoluting and playing back the signal from the sound source 100 Screen production section 200, 201 Audio data production section 300, 301, 303 Sound image localization production section 400 Editor 401 Gain data generation means 404 Start point / End point Designating means 405 Sound image designating means 500 Soft recorder 600 Conversion information designating part sp1, sp2 Speakers h1L, h1R Head-to-ear transfer characteristics from the speaker sp1 to the listener's left and right ears h2L, h2R Speaker sp2 to the listener's left and right ears Transmission PLx, pRx Head-to-head transfer characteristics to the listener's left and right ears when an actual speaker is placed at the desired localization position x cfLx, cfRx Cancel filter (convolver) and its coefficient DM Dummy head (or human head) M Listening Person (game operator, listener) X sound source x target sound image localization position

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 5, 1993

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] Full text

[Correction method] Change

[Correction content]

[Document name] Statement

Title of invention Video game authoring system

[Claims]

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video game authoring system, and more particularly to a reproduced image together with a pair of transducers (speakers) for reproduction so that the listener is different from the position of the transducer.
The present invention relates to a video game authoring system for editing game software used in a video game machine or the like that makes a user feel that a sound image is localized at a desired position more than one degree.

[0002]

2. Description of the Related Art Conventionally, there is a sound image localization method using a binaural technique in which a sound source is felt at a specific position (direction) by the level difference and phase difference (time difference) of signals in both ears. As a sound image localization method using an analog circuit, for example, there is a sound image localization method described in Japanese Patent Application Laid-Open No. 53-140001 (Japanese Patent Publication No. 58-3638). This is because the analog filter emphasizes / attenuates the level of a specific frequency band (amplitude control) to give a sense of front and back of the sound source, and the analog delay causes a time difference between the left and right sounds (phase control). It was intended to give a sense of left and right.

Further, along with the recent progress of digital processing technology, there is a sound image localization method realized by a digital circuit, for example, there is a "sound image forming method and its device" described in Japanese Patent Laid-Open No. 2-298200. In the sound image localization method using this digital circuit, the signal from the sound source is FFT (Fa
st Fourier Transform), performs processing on the frequency axis, and gives a level difference and a phase difference depending on the frequency to both left and right channel signals to digitally control the localization of the sound image. The frequency-dependent level difference and phase difference at each sound image localization position of this method are collected as experimental data using an actual listener.

However, in the sound image localization method using such a circuit, if the sound image is to be localized accurately and precisely,
The circuit scale was extremely large, and it was only used as a recording system for special business purposes. Further, the authoring system for editing the software used in these devices also naturally supplies and edits audio data and image data corresponding to these.

[0005]

By the way, recently,
Amusement game machines and computer terminals using virtual reality have appeared. In these game machines and terminals, more realistic sound image localization is being demanded together with images. Moreover, for example, in a game machine (or terminal device), the flight course (position) of the airplane varies depending on the operation of the operator, and the sound image is changed in real time according to the operation of the operator. It becomes necessary to perform a moving process and reproduce it. This is a point that is significantly different from the sound image localization processing for records described above.

[0006] The applicant has responded to such a demand as follows.
First, the circuit scale of the device is small, the cost is low, and
We proposed a sound image localization control method that can localize a sound image together with an image in a wide space exceeding 180 degrees. The present invention provides a video game authoring system for editing suitable game software which is used in an apparatus adopting such a sound image localization control method.

[0007]

In order to solve the above problems, the present invention is to provide a video game authoring system having the following constitutions 1) to 8). That is, 1) a predetermined image is reproduced on the basis of incoming video display data, and a sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers is subjected to sound image processing. Used for a video game machine that reproduces a signal and makes the listener feel that a sound image is localized at an arbitrary moving position different from the pair of transducers, in order to change the audio data including the sound source data and the audio data. A video game authoring system for editing game software, which is supplied with audio data conversion information and sound image localization information indicating the localization position of the sound image, and specifies the sound image localization information. Corresponding to the direction and speed of sound image movement with respect to the listener And a conversion information generating means for generating the audio data conversion information.

2) A predetermined image is reproduced on the basis of incoming video display data, and sound image processing is performed by sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. It is used in a video game machine that reproduces a signal that has been heard and makes the listener feel as if the sound image is localized at an arbitrary moving position different from the pair of transducers.The audio data including the sound source data and the localization position of the sound image are used. A video game authoring system for editing game software, together with a predetermined image supplied with sound image localization information, and sound image information designating means for designating the sound image localization information, and the sound image movement information. Changes in the time axis corresponding to the moving direction and speed of the sound image with respect to the listener of the Video game authoring, characterized in that formed by and a Kumise occupied conversion processing means
system.

3) A predetermined image is reproduced on the basis of incoming video display data, and sound image processing is performed by a sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. It is used for a video game machine that reproduces the generated signal and makes the listener feel that a sound image is localized at an arbitrary moving position different from the pair of transducers, and changes audio data including sound source data and this audio data. A video game authoring system for editing game software together with a predetermined image supplied with audio data conversion information and sound image localization information indicating the localization position of the sound image, the start point and the end point of the sound source Specifying means for specifying the sound image localization information and the moving speed between them, and Sound image localization information generating means for generating sound image localization information at the intermediate point between the start point and the end point, and sound image localization information for the intermediate point, based on a predetermined relationship by input from the designating means. A video game authoring system, comprising: a conversion upper table generating means for generating information.

4) A predetermined image is reproduced on the basis of the incoming video display data, and sound image processing is performed by sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. It is used in a video game machine that reproduces a signal that has been heard and makes the listener feel as if the sound image is localized at an arbitrary moving position different from the pair of transducers.The audio data including the sound source data and the localization position of the sound image are used. A video game authoring system for editing game software, together with a predetermined image supplied with sound image localization information, which specifies sound image localization information of the start point and end point of the sound source and a moving speed between them. The starting point based on a predetermined means for inputting and a predetermined relationship by the input from the specifying means. And a sound image localization information generating means for generating sound image localization information at an intermediate point between the end point and the end point, and a conversion process for imparting to the audio data a time axis change corresponding to a sound image moving direction and a velocity of a listener of the sound image moving information at the intermediate point. A video game authoring system comprising:

5) A predetermined image is reproduced on the basis of the incoming video display data, and sound image processing is performed by sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. It is used for a video game machine that reproduces the generated signal and makes the listener feel that a sound image is localized at an arbitrary moving position different from the pair of transducers, and changes audio data including sound source data and this audio data. A video game authoring system for editing game software together with a predetermined image supplied with audio data conversion information and sound image localization information indicating a localization position of the sound image, wherein the sound image localization information and listening Sound image designating means for designating the distance of the sound image to the user, Video game authoring system characterized by comprising; and a conversion information generating means for generating in response to the moving direction and speed of the sound image of Odeta conversion information for the listener.

6) A predetermined image is reproduced based on incoming video display data, and sound image processing is performed by sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. Used for a video game machine that reproduces the recorded signal and makes the listener feel as if the listener is localizing an arbitrary movement position different from the pair of transducers, and audio data including sound source data and a sound image indicating the localization position of the sound image A video game authoring system for editing game software, which is supplied with localization information and a predetermined image, and sound image specifying means for specifying the sound image localization information and the distance of the sound image to a listener. , The change of the time axis corresponding to the moving direction and speed of the sound image to the listener of the sound image localization information Video game authoring system characterized by comprising comprises a conversion processing means allowed to impart to the audio data.

7) A predetermined image is reproduced based on the incoming video and display data, and a sound image localization signal having a convolver supplied with a monaural sound source is reproduced from a pair of spaced transducers. Audio data containing sound source data and audio data for changing this audio data, which is used for a video game machine to make a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. A video game authoring system for editing game software, which is supplied with conversion information, sound image localization information indicating a localization position of the sound image, and gain information for changing the volume, and is a sound source. Image localization information of the start and end points of the Specifying means for specifying, sound image localization information generating means for generating sound image localization information at the intermediate point between the start point and the end point, and the audio data conversion information based on the relationship predetermined by the input from the specifying means. And conversion information generating means for generating the gain information corresponding to the sound image localization information, and gain information generating means for generating the gain information corresponding to the sound image localization information. Video game authoring system.

8) A predetermined image is reproduced on the basis of incoming video display data, and sound image processing is performed by a sound image localization processing means having a convolver to which a monaural sound source is supplied from a pair of spaced transducers. It is used in a video game machine that reproduces a signal that has been reproduced and makes a listener feel that a sound image is localized at an arbitrary moving position different from the pair of transducers.Audio data including sound source data and the localization position of the sound image are used. With a predetermined image supplied with the sound image localization information, a video game authoring system for editing the game software, the sound image localization information of the start point and the end point of the sound source, and the movement trajectory between them. A designation means for designating the moving speed during this period and a relationship predetermined by the input from the designation means Then, the sound image localization information generating means for generating the sound image localization information at the intermediate point between the start point and the end point, and the time axis change and volume change corresponding to the moving direction and speed of the sound image of the sound image movement information at the intermediate point with respect to the listener. A video game authoring system, comprising: a conversion processing means for adding a change in gain for the audio data to the audio data.

[0015]

Embodiments Video game authoring according to the present invention
An embodiment of the system will be described below with reference to the drawings.
First of all, the basic principle of a sound image localization control method capable of localizing a sound image in a wide space exceeding 180 degrees by using the game software edited by the video game authoring system of the present invention. Will be described. This is a technique for localizing a sound image at an arbitrary position in space by using a pair of transducers (which will be described below by taking a speaker as an example) arranged apart from each other.

FIG. 6 is a principle diagram of sound image localization. sp1,
sp2 is a speaker arranged in front of and on the left and right of a listener (sometimes referred to as a listener in the embodiment), sp2
1 to h1L the head-related transfer characteristic (impulse response) from the listener's left ear to h1R, sp
The head-related transfer characteristics from 2 to the left and right ears are h2L and h2R. In addition, the head-related transfer characteristics to the left and right ears of the listener when the actual speaker is placed at the target localization position x are pL.
x, pRx. Here, each transfer characteristic is obtained by actually measuring with a speaker in a sound space and a microphone at both ears of a dummy head (or human head), and subjected to appropriate waveform processing.

Next, signals obtained by passing the sound source X to be localized through signal converters cfLx and cfRx (transfer characteristics by a convolver) are sp1 and sp, respectively.
Consider playing in 2. At this time, letting the signals obtained in the left and right ears of the listener be eL and eR, eL = h1L · cfLx · X + h2L · cfRx · X (Equation 1) eR = h1R. cfLx. X + h2R / cfRx / X (〃)

On the other hand, when the signals obtained in the left and right ears of the listener when the source X is reproduced from the target localization position are dL and dR, dL = pLxX (Equation 2) dR = pRxX (〃)

Here, if the signals obtained in the left and right ears of the listener by reproduction of sp1 and sp2 match the signal when the source is reproduced from the target position, the listener is as if the speaker exists at the target position. The sound image will be recognized. From these conditions eL = dL, eR = dR and (Equation 1) and (Equation 2), X is deleted to obtain h1L · cfLx + h2L · cfRx = pLx (Equation 3) h1R · cfLx + h2R · cfRx = pRx (Equation 3) CfLx, cfRx is calculated from cfLx = (h2R · pLx−h2L · pRx) / H (equation 4a) cfRx = (− h1R · pLx + h1L · pRx) / H (〃) where H = h1L · h2R-h2L · h1R (Formula 4b)

Therefore, if a signal to be localized is processed by a convolver (convolution operation processing circuit) or the like using the transfer characteristics cfLx and cfRx calculated by (Equation 4a) and (Equation 4b), a sound image is obtained at a target position x. Can be localized. There are various concrete methods for realizing the signal conversion apparatus, but it is sufficient to use an asymmetric FIR digital filter (convolver). The final transfer characteristic when used in the FIR digital filter is a time response function.

That is, as the transfer characteristics cfLx and cfRx at the required localization position x, (Equation 4a) and (Equation 4b)
The coefficients cfLx and cfRx are created in advance as the coefficients for realizing the one obtained in step 1 by the FIR filter processing, and are prepared as ROM data.
If the coefficient of the required sound image localization position is transferred from the ROM to the FIR digital filter and the signal from the sound source is subjected to the convolutional arithmetic processing and reproduced from the pair of speakers, the sound image is localized at any desired position.

Next, a method of controlling the sound image localization based on such a principle will be described in detail with reference to FIG. FIG. 7 shows the steps of the method for controlling the sound image localization. Head related transfer function
sfer Function (hereinafter referred to as HRTF) measurement (step 1)

This will be described with reference to FIGS.
FIG. 8 shows an HRTF measurement system. A pair of microphones ML and MR are installed in both ears of the dummy head (or human head) DM, the measurement sound from the speaker SP is received, and the source sound (reference data) r is recorded in the recorder DAT.
efL and refR and the to-be-measured sound (measurement data) L and R are recorded in synchronization.

The source sound XH may be impulse sound, white noise, or other noise. In particular, from the viewpoint of statistical processing, white noise is
Since it is a continuous sound and the energy distribution is constant over the audio band, it is possible to use SN by using white noise.
The ratio is improved. The position of the speaker SP is set to a plurality of angles θ (for example,
As shown in FIG. 10, it is installed at 12 points every 30 degrees and continuously recorded for a predetermined time.

HRTF impulse response (Impul
se response; hereinafter referred to as IR) (step 2) In step 1, the source sounds (reference data) refL, refR and the sounds to be measured (measurement data) L, R recorded in synchronization with each other are stored in the workstation. Processing (not shown).

Letting X (S) be the frequency response of the source sound (reference data), Y (S) be the frequency response of the sound to be measured (measurement data), and IR (S) be the frequency response of the HRTF at the measurement position. There is an input / output relationship shown in Expression 5). Y (S) = IR (S) · X (S) (Equation 5) Therefore, the frequency response of the HRTF is IR (S) = Y (S) / X (S) (Equation 6) is there. Therefore, the frequency response X (S) of the reference,
The frequency response Y (S) of the measurement data is obtained by cutting out the data obtained in the above step 1 using a window that is time-synchronized.
A finite Fourier series expansion is performed by FT transformation to calculate as a discrete frequency, and from (Equation 6), the frequency response I of the HRTF is
R (S) is calculated by a known calculation method.

In this case, the accuracy of IR (S) is increased (S
In order to improve the N ratio), IR (S) may be calculated for several hundred windows that are temporally different, and they may be averaged. Then, the calculated HRTF frequency response IR (S)
Is subjected to inverse FFT conversion to obtain a time axis response (impulse response) IR (first IR) of HRTF.

IR (impulse response) shaping processing (step 3) Here, the IR obtained in step 2 is shaped. First, for example, by FFT conversion, the first IR obtained in step 2 is expanded at discrete frequencies over the audio spectrum,
An unnecessary band (a large dip occurs in the high range, but this is unnecessary that does not affect the sound image localization),
It is removed by BPF (band pass filter). By limiting the band in this way, unnecessary peaks and dips on the frequency axis are removed and unnecessary coefficients do not occur in the cancel filter, so that the convergence is improved and the coefficients can be shortened.

Then, the band-limited IR (S) is inverted F
FT conversion is performed, and IR (impulse response) is cut out on the time axis by a window (for example, a cosine function window) and subjected to window processing (second IR). By performing the window processing, the effective length of the IR is not long, the convergence of the cancel filter is improved, and the sound quality is not deteriorated.

Cancellation filters cfLx, cfRx
(Step 4) The cancel filters cfLx and cfRx, which are convolvers (convolutional integration circuits), are cfLx = (h2R · pLx−h2L · pRx), as shown in (Equation 4a) and (Equation 4b) described above. / H (Formula 4a) cfRx = (-h1R.pLx + h1L.pRx) / H (〃) However, it is H = h1L.h2R-h2L.hIR (Formula 4b).

Here, the speakers sp1 and sp to be arranged are arranged.
Head related transfer characteristics h1L, hIR, h2L, h2R
And, as the head-related transfer characteristics pLx, pRx when the actual speaker is placed at the target localization position x, the second IR (which has been shaped and processed for each angle θ obtained in steps 1 to 3 above). Impulse response).

The head-related transfer characteristics h1L and h1R correspond to the position of the L-channel speaker in FIG. 11, and if they are installed at 30 degrees (θ = 330 degrees) from the front to the left, θ = 330. The IR of the degree is used. Head-related transfer characteristic h2
R and h2L correspond to the position of the R channel speaker in the figure, and are, for example, 30 degrees (θ = 30) from the front to the right.
, The IR of θ = 30 degrees is used (that is, the system at the time of actual sound image reproduction (for example, FIG. 9).
Select one that is close to)).

The head-related transfer characteristics pLx and pRx are 90 degrees from the front, which is the target sound source localization position.
The cfLx of the entire space corresponding to that by substituting IR for every 30 degrees in a wide space (overall space) beyond that of 180 °
cfRx, that is, 12 sets of cancellation filters cfLx and cfRx are obtained every 30 degrees (in FIG. 11, a position of 240 degrees is taken as an example). The cancel filters cfLx and cfRx group are finally obtained as IR (impulse response) which is a response on the time axis.
Note that the cancel filter cfLx according to (Equation 4a),
The calculation of cfRx is as follows. First, H ⁻¹ , which is a kind of inverse filter for H in (Equation 4b), is obtained by the least squares method, and this is inverse FFT-transformed to obtain a time function h (t)
And Also, the terms h1L, h1R, and h2 of (Equation 4a)
By expressing L, pRx, pLx, and h2R by a time function, respectively, the following equation is established.

CfLx (t) = (h2R · pLx−h2L · pRx) · h (t) (Equation 7) cfRx (t) = (− hIR · pLx + h1L · pRx) · h (t) (〃)

Therefore, the coefficients of the cancel filters cfLx and cfRx can be obtained from these (formula 7). As is clear from (Equation 7), in order to shorten the coefficients of the cancel filters cfLx and cfRx, the head-related transfer characteristics h1L, hIR, h2L, pRx, pLx,
It is extremely important to shorten each h2R. Therefore, as described above, various processes such as the window process and the shaping process are performed in steps 1 to 3 to shorten the head-related transfer characteristics h1L, hIR, h2L, pRx, pLx, h2R.

Scaling of Cancellation Filter at Each Localization Point x (Step 5) Further, the spectrum distribution of the sound source (source sound) actually processed by the convolver (cancellation filter) is like pink noise statistically. There are things that are distributed or that drop gently in the high range.In any case, since the sound source is different from a single sound, there is a risk that overflow will occur when convolution calculation (integration) is performed and distortion will occur. is there. Therefore, in order to prevent the overflow, the maximum gain (for example, the cancel filter cfL) among the coefficients of the cancel filters cfLx and cfRx.
x, cfRx), and all coefficients are scaled so that overflow does not occur when the coefficient and the white noise of 0 db are convoluted.

Scaling is performed in this way,
Finally, a group of data supplied to the convolver as coefficients (in this example, 12 sets of convolver coefficient groups capable of sound image localization every 30 degrees) cfLx and cfRx are obtained.

A signal from the source is convoluted and reproduced (step 6) For example, as a sound reproducing device for a game machine, as shown in FIG. 9, a pair of speakers are spaced 30 degrees left and right centering on a listener M. Sp1 and sp2 are provided, and the pair of speakers sp1 and sp2 are configured to reproduce the acoustic signals processed by the pair of convolvers (convolution operation processing circuits).

A signal from the same sound source X (for example, a monaural sound such as a flight sound from a game synthesizer) is supplied to the pair of convolvers, and the above-mentioned step 1 is performed.
The IR coefficients cfLx and cfRx created in 05 (for example, when the sound image is to be localized at a position 120 ° to the left rear (θ ÷ 240 °), θ = 240 °), the convolver is selected. Is set to. For example, based on a sound image localization command from a main CPU (central processing unit) of a game machine or the like, the control sub CPU transfers the coefficient of the desired localization position from the coefficient ROM to the pair of convolvers.

In this way, the signal from the sound source X is subjected to the convolution operation processing on the time axis by the pair of convolvers, and the pair of speakers sp1 and sp1 arranged apart from each other.
Played from 2. The sound reproduced from the pair of speakers sp1 and sp2 has its sound image localized such that the crosstalk to both ears is canceled and the sound source is located at a desired position, and is heard by the game operator (listener) M. Played as an extremely realistic sound. As for the coefficient of the convolver, the optimum sound image position is sequentially selected and switched with the transition of the movement of the airplane according to the operation of the operator. Also, from the flight sound,
For example, when changing to a missile sound, the source sound from the sound source x is changed from a flight sound to a missile sound. In this way, the control method can freely localize the sound image at any position. Note that the sound image is reproduced along with the image as in a general game machine, but the description of the image processing system is omitted.

Furthermore, in addition to the above-mentioned sound image localization, it is possible to make the sound more realistic by taking into consideration the change of the pitch (pitch) and the change of the sound volume due to the Doppler effect accompanying the movement of the sound image. The video game authoring of the present invention
The system also includes data for utilizing this Doppler effect, and next, this Doppler effect will be described in advance. FIG. 5 is a drawing for explaining the Doppler effect. In the same figure, let us consider a case where the sound source passes through the side (distance d) of the listener M at a constant velocity (velocity v). First, with respect to the change in direction localization, the change becomes faster as it approaches the side of the listener M. If the passage time right beside is set to t = 0, and the direction localization θ is represented counterclockwise with respect to the side just next to the listener M, the direction localization θ and the time t are expressed by θ = −arctan (vt / d). Relationship is established.

For changes in pitch (pitch),
If the frequency of the sound actually emitted under the influence of the Doppler effect is f and the frequency audible to the ear is F, then F = fc / (cv-sin θ), where c is the speed of sound. That is, the pitch (pitch) ratio N
Becomes N = c / (cv-sin θ). Further, regarding the sound volume, if the sound volume at the time of passing directly beside is g, the audible sound volume G is G = g · cos θ
Becomes

Therefore, as shown in FIG. 5, as the sound source moves, the direction localization θ is represented by a discrete value every 22.5 degrees, and line segments A, B, C, D, E, F, The same pitch conversion is applied to G and the like. The pitch conversion ratio N is given by the above equation.
The length of each line segment corresponds to the length of time that pitch conversion is performed. In order to bring the moving sound image closer to the real sound, it is necessary to take the above points into consideration, and the video game authoring system of the present invention includes the data aiming at the Doppler effect. Is what you are doing.

The game software created from the video game authoring system of the present invention is used in a game machine or the like equipped with the sound image localization processing means composed of the above convolver. Next, this video game authoring system is used. -Each embodiment of the system will be sequentially described. FIG. 1 is a schematic block diagram showing a first embodiment of the video game authoring system. 100 is a screen production unit for generating image data of characters such as airplanes, 200 is an audio data production unit for generating audio data of music, sound effects, voices, etc., 300 is a listener (operator). ) Moving angle θ of sound image and distance d
A sound image localization production unit for designating a sound source, a conversion information generation unit 600 for generating change information (pitch ratio N) on the time axis of the sound source based on information from the sound image localization production unit 300,
Reference numeral 400 is an editor for obtaining each of the above-mentioned information and for arranging the information in a predetermined arrangement. Reference numeral 500 is a soft recorder such as an FDD, a ROM writer, a CD, and a magneto-optical disk drive. This video game authoring system has the above configuration.

Next, this operation will be described. Image data such as an airplane from the screen production unit 100 and sound source data such as music and sound effects from the audio data production unit 200 are supplied to the editor 400, respectively. On the other hand, in the sound image localization producing section 300, sound image localization information indicating the moving movement angle θ of the sound image with respect to the listener of the sound source data and the distance d of the sound image with respect to the operator is generated. The sound image localization information is supplied to the editor 400 and the conversion information generation unit 600. The conversion information generation unit 600 detects movement of the sound image or relative position change based on the incoming sound image localization information, and changes information on the time axis,
That is, pitch (pitch) ratio information is generated, and this is edited by the editor 4
Supply to 00. The editor 400 obtains the above-mentioned data, sequentially arranges them in accordance with a format of a predetermined data structure that can be reproduced synchronously, and supplies them to the soft recorder 500 together with a program representing the story of the game. In the soft recorder 500, the data is recorded on a desired recording medium mounted on the soft recorder 500.

Therefore, on this recording medium, image data for reproducing an image of an airplane, audio data which is sound source information such as musical instrument sound and sound effect, and a moving position when a sound image moves to a listener. And so on, and time-axis conversion information (pitch ratio information, time length information for pitch conversion, etc.) corresponding to this sound image movement are recorded, respectively.

The recording medium is packaged in a game cartridge or the like and used in, for example, a game machine having the above-described structure. In the game machine, the above-described predetermined signal processing is performed based on the information. It As a result, the sound corresponding to the image is localized and reproduced by the pair of speakers over 180 degrees or more. Moreover, since the sound image at this time is reproduced with a change in the pitch assuming the Doppler effect, a sound more realistically reproduced is reproduced. Further, the audio / data production unit 200 may be synchronized with the sound image localization production unit 300 by a control signal such as MIDI.

FIG. 2 is a schematic block diagram showing a second embodiment of the video game authoring system. The same components as those of the first embodiment are designated by the same reference numerals and the description thereof will be omitted. Similar to the first embodiment, this embodiment also includes an audio data production unit 201 for generating audio data such as music, sound effects, and voice, but the function is different. That is, sound source data such as music and sound effects is generated in the audio / data producing unit 201, and this data is supplied to the editor 400 and also to the sound image localization producing unit 301. The sound image localization production unit 301 detects movement of the sound image or relative position change based on the supplied sound image localization information, and generates change information on the time axis, that is, pitch (pitch) ratio information, etc. Is supplied to the audio data production unit 201.
On the other hand, the moving angle θ and the distance d of the sound image with respect to the listener are specified based on the sound image localization information, and this information is edited.
Supply to. The audio data production unit 201 receives the pitch (pitch) ratio information, and the internal conversion processing unit generates change data on the time axis corresponding to the movement of the sound image or the relative position change, and supplies it to the editor 400.

That is, the audio data production section 201 creates new audio data corresponding to the change information by time-axis conversion of a sound source such as music or sound effect. For example, when the ratio (N: n) of the sound source to the frequency n as shown in FIG. 4A is smaller than 1 (when the sound source is approaching the operator), it is shown in FIG. If the ratio (N: n) of the original sound source to the frequency n is greater than 1 (when the sound source moves away from the operator), it is converted into compressed data as shown in FIG. Converted to decompressed data. Thus, the conversion of the time axis involves a change in pitch (pitch) and a change in data length. Then, the time-axis converted data is supplied to the editor 400, where the incoming data are arranged so as to have a predetermined format, as described above, and stored in the next soft recorder 500. The game software is recorded on a recording medium and a predetermined game software is created. According to this embodiment, since the pitch (pitch) ratio information is not used in the editor 400, it is effective when the pitch (pitch) conversion means is not provided inside the game machine. In addition, here, the screen production unit 1
00 to the audio data production section 201 may be synchronized with a synchronization signal such as a tie code.

FIG. 3 is a schematic block diagram further showing a third embodiment of the video game authoring system. Reference numeral 202 is an audio data production section, and 303 is a sound image localization production section. Particularly, in this embodiment, the sound image localization information at the start point and the end point of the sound image movement and the start point / end point designating means 404 for designating the speed of movement therebetween, and the sound image localization information intermediate between the start point and the end point in the editor 400 are provided. Similarly to the sound image localization information generating means 402 for generating, in the editor 400, the intermediate sound image localization information and time axis conversion information corresponding to the sound image localization information of the start point and the end point are supplied to the audio data production unit 202. Conversion processing means 4 for
03 and 03 are provided respectively.

Here, the start point / end point designating means 404 may be means for designating the sound image localization information of the start point and the end point of the sound image, the moving speed between them, and the path (moving locus) between them. That is, it is a path of the direction localization θ of the sound image to the listener and a constant radius r when forming a circular trajectory, or the direction localization θ and the distance d (straight line) from the listener. in this case,
The editor 400 may be provided with a gain data generating means 401 for giving a predetermined gain to the sound source data for determining the volume with respect to the change of the sound pressure, and the gain data generating means 401 may be supplied to the audio data production unit 202. Thereby, the sound image localization information (direction localization θ, distance d) from the start point to the end point can be automatically generated.

Further, the conversion information generating means 600 and the sound image designating means 405 may be provided. In this case, the sound image designating means 405 drives the gain data generating means 401 and the sound image localization information generating means 402 of the editor 400,
Audio data production unit 202, sound image localization control unit 303
Can supply modified data to each of the
It is possible to supply the data as the conversion information from the conversion information generating means 600 without converting the data by the conversion processing means 403. Here, the sound image designating means 405 may be input by a remote joystick. Also,
The screen production unit 100 and the audio data production unit 202 may be synchronized with each other by a synchronizer (not shown).

Further, the gain data may be separately provided. The gain data in this case is used to change the gain of the convolver of the game machine, and the volume is changed by changing the gain of the convolver. In the above description, a video game or the like generally refers to a device in which not only a normal video game but also an image and a sound are interactively operated.

[0054]

As described above in detail, according to the video game authoring system of the present invention, it is possible to provide the optimum game software to be used in the video game machine provided with the sound image localization processing means having the convolver. it can. In particular,
By using this game software, the pitch of the sound source (compression / expansion on the time axis) corresponding to the positional relationship between the sound image and the listener and the moving speed / direction of both (synchronous with the movement of the sound image) Is added), a sound effect corresponding to a so-called Doppler effect can be obtained, and game software closer to a real sound can be edited. Also, for audio data, the sound image localization information at the start and end points of the sound image and the speed of movement between them are specified, and the intermediate sound image localization information and conversion information are converted into the sound image localization information based on a predetermined relationship. The editing efficiency is good because it corresponds and is generated. Furthermore, the sound image localization information at the start and end points of the sound image and the speed of movement between them are specified (that is, the positional relationship between the sound image and the listener and the moving speed and moving direction of both are specified from the start point to the end point) in advance. Since the intermediate sound image localization information and the audio data obtained by converting the change corresponding to the sound image movement of the sound image localization information are obtained based on the determined relationship, the editing efficiency is good. Furthermore, the volume can be changed synchronously to control the movement between the start point and the end point of the sound image in a straight line or a curved line. When the data for changing the volume is included, the sound becomes even more realistic. A sound effect is obtained.

[Brief description of drawings]

FIG. 1 is a schematic block diagram showing a first embodiment of a video game authoring system according to the present invention.

FIG. 2 is a schematic block diagram showing a second embodiment of the video game authoring system according to the present invention.

FIG. 3 is a schematic block diagram showing a third embodiment of the video game authoring system according to the present invention.

FIG. 4 is a diagram for explaining a time axis change of a sound source.

FIG. 5 is a diagram for explaining a Doppler effect.

FIG. 6 is a configuration diagram for explaining the principle of sound image localization control.

FIG. 7 is a chart showing steps of a sound image localization control method of a sound image localization control device for a video game according to the present invention.

FIG. 8 is a configuration diagram showing an HRTF (head related transfer function) measurement system.

FIG. 9 is a basic configuration diagram of a sound image localization control device.

FIG. 10 is a diagram for explaining the points of HRTF measurement.

FIG. 11 is a diagram for explaining a calculation example of a cancel filter.

[Description of Reference Signs] 1 step of measuring head related transfer function (HRTF) 2 step of calculating HRTF impulse response 3 step of shaping IR (impulse response) 4 step of calculating cancel filters cfLx and cfRx 5 cancel filter 6 Scaling the signal from the sound source and playing back by convolutional calculation 100 Screen production unit 200, 201 Audio data production unit 300, 301, 303 Sound image localization production unit 400 Editor 401 Gain data generation means 404 Start point / End point Designating unit 405 Sound image designating unit 500 Soft recorder 600 Conversion information designating unit sp1, sp2 Speakers h1L, h1R Head transfer characteristics from the speaker sp1 to the listener's left and right ears h2L, h2R Speaker sp2 to the listener's left and right ears Head transfer characteristics at pLx, pRx Head transfer characteristics up to the listener's left and right ears when an actual speaker is placed at the desired localization position x cfLx, cfRx Cancel filter (convolver) and its coefficient DM Dummy head (or Human head) M Listener (game operator, listener) X Sound source x Target sound image localization position

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI Technical display location H04N 5/76 Z 7916-5C H04S 5/00 8421-5H 7/00 F 8421-5H

Claims (8)

[Claims] 1. A sound image localization process comprising a convolver in which a predetermined image is reproduced on the basis of incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. The audio data included and audio data conversion information for changing the audio data and sound image localization information indicating the localization position of the sound image are supplied together with a predetermined image,
A television game authoring system for editing game software, comprising sound image information designating means for designating the sound image localization information, and the sound image information designating means corresponding to the moving direction and speed of the sound image to the listener. And a conversion information generating means for generating the conversion information of the video data, a television game operating system. 2. A sound image localization process comprising a convolver in which a predetermined image is reproduced on the basis of incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. A television game authoring system for editing game software, which is supplied with included audio data and sound image localization information indicating a localization position of the sound image, and a predetermined image. Sound image information designating means for designating sound image localization information, and a time axis change corresponding to the moving direction and speed of the sound image to the listener of the sound image moving information is given to the audio data. A television game authoring system comprising a conversion processing means. 3. A sound image localization process comprising a convolver in which a predetermined image is reproduced on the basis of incoming video display data, and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. The audio data included and audio data conversion information for changing the audio data and sound image localization information indicating the localization position of the sound image are supplied together with a predetermined image,
A television game authoring system for editing game software, comprising: designation means for designating the sound image localization information of the start and end points of the sound source and the moving speed between them; and the designating means. Corresponding to the sound image localization information generating means for generating the sound image localization information of the intermediate point between the start point and the end point, and the audio image localization information of the intermediate point, based on a predetermined relationship by the input from And a conversion information generating means for generating conversion information. 4. A sound image localization process comprising a convolver in which a predetermined image is reproduced based on incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. A television game authoring system for editing game software, which is supplied with included audio data and sound image localization information indicating a localization position of the sound image, and a predetermined image. Based on the sound source localization information of the start point and the end point of the sound source and designating means for designating the moving speed between them, based on the relationship predetermined by the input from the designating means, the start point and the Sound image localization information generating means for generating sound image localization information at the middle point of the end point, and time-axis changes corresponding to the direction and speed of the sound image movement to the listener of the sound image movement information at the middle point are given to the audio data. A television game authoring system comprising a conversion processing means. 5. A sound image localization process comprising a convolver in which a predetermined image is reproduced based on incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used in a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if a sound image is localized at an arbitrary moving position different from the pair of transducers. The audio data included and audio data conversion information for changing the audio data and sound image localization information indicating the localization position of the sound image are supplied together with a predetermined image,
A television game authoring system for editing game software, comprising sound image localization information and sound image designating means for designating the distance of the sound image to a listener, and the audio data conversion. And a conversion information generating means for generating information corresponding to the moving direction and speed of a sound image with respect to a listener. 6. A sound image localization process comprising a convolver in which a predetermined image is reproduced based on incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used for a television game machine that reproduces a signal subjected to sound image processing by means, and makes a listener feel as if the listener moves the moving position different from the pair of transducers, and includes sound source data. Oh
A television game operating system for editing game software, which is supplied with a video image and sound image localization information indicating a localization position of the sound image, and which is the sound image localization information. Sound image designating means for designating the distance of the sound image to the listener, and conversion processing means for giving the audio data a time axis change corresponding to the moving direction and speed of the sound image of the sound image localization information with respect to the listener. And a video game system characterized by comprising:
Saling system. 7. A sound image localization process comprising a convolver in which a predetermined image is reproduced based on incoming video and display data and a monaural sound source is supplied from a pair of spaced transducers. Play back the signal
Used in a television game machine for making a listener feel that a sound image is localized at an arbitrary moving position different from the pair of transducers, and audio data including sound source data and this audio Audio data conversion information for changing the audio data, sound image localization information indicating the localization position of the sound image, and gain information for changing the volume are supplied to edit the game software together with a predetermined image. And a specifying means for specifying the sound image localization information of the start point and the end point of the sound source, the moving locus between them, and the moving speed between them, and the specifying means, Based on the predetermined relationship by input of,
Sound image localization information generating means for generating sound image localization information at the intermediate point between the start point and the end point, conversion information generating means for generating the audio data conversion information corresponding to each sound image localization information, and And a gain information generating means for generating gain information corresponding to each sound image localization information. 8. A sound image localization system comprising a convolver in which a predetermined image is reproduced on the basis of incoming video display data and a monaural sound source is supplied from a pair of spaced transducers. It is used for a television game machine which reproduces a sound image processed signal by the processing means and makes a listener feel as if the sound image is localized at an arbitrary moving position different from the pair of transducers. A television game authoring system for editing the game software, which is supplied with audio data including the sound image localization information indicating the localization position of the sound image and a predetermined image. Based on the sound image localization information of the start point and the end point of the sound source, a moving locus between them, a designating unit for designating a moving speed therebetween, and a relationship predetermined based on an input from the designating unit. The sound image localization information generating means for generating the sound image localization information at the intermediate point between the start point and the end point, and the time axis change and the volume change corresponding to the moving direction and speed of the sound image to the listener of the sound image movement information at the intermediate point. And a conversion processing means for imparting a gain change to the audio data, the television game operating system.