JP4584203B2 - Voice simulation apparatus, voice simulation method, and program - Google Patents

Description

  The present invention relates to an audio simulation apparatus, an audio simulation method, and a program for realizing these on a computer.

  In general, the Doppler effect depends on the source of a wave (sound wave, radio wave, light wave, etc.) (sound source, radio wave source, light source, etc.), the relative speed of both, and the speed at which the observer moves relative to the wave medium. A phenomenon that is observed with different wave frequencies. In general, when the source approaches the observer, the vibration of the wave is reduced and the frequency is increased, and when the source is moved away, the vibration is extended and the frequency is decreased.

A technique related to such a Doppler effect is disclosed in, for example, the following documents.
Japanese Patent Laid-Open No. 10-042399 `` Doppler effect '' from the free encyclopedia Wikipedia http://en.wikipedia.org/wiki/%E3%83%89%E3%83%83%E3%83%97%E3%83%A9% E3% 83% BC% E5% 8A% B9% E6% 9E% 9C

Here, [Patent Document 1] discloses that a technique for simulating the Doppler effect is applied to an audio spatialization system.
On the other hand, [Non-Patent Document 1] discloses a technique for calculating the Doppler effect for sound waves. The observer and the sound source move on the same straight line, the direction from the sound source S to the observer O is corrected, the frequency of the sound wave emitted by the sound source is f, the speed of sound is c, the speed of movement of the observer is v _o , and the speed of movement of the sound source Is represented as v _s , the frequency f ′ of the sound wave that can be heard by the observer is obtained as follows.
f '= f × (cv _o ) / (cv _s )

However, the case where the sound source and the observer freely move in the space or the case where the moving speed exceeds the sound speed is not disclosed in these documents, and a simple simulation technique is strongly demanded.
The present invention solves the above-described problems, and an object thereof is to provide an audio simulation apparatus, an audio simulation method, and a program that realizes these on a computer that can easily simulate the Doppler effect.

  In order to achieve the above object, the following invention is disclosed in accordance with the principle of the present invention.

The sound simulation apparatus according to the first aspect of the present invention is a virtual space in which a sound source object that emits sound and an observation object that observes sound move, from the time t ₁ to the time t _{2 in the} observation object. The observed voice waveform is simulated and includes a history storage unit, a time calculation unit, and a voice output unit, and is configured as follows.

  Here, the history storage unit stores a movement history of the sound source object in the virtual space.

  Typically, the position of a sound source object in the virtual space is acquired at regular time intervals, and sequentially stored in a form such as an array or a ring buffer, so that the position at a certain time can be acquired. If you want to know the position of the sound source object at the exact time you want, and you do not store the position corresponding to that time, obtain the position at the time before and after that time and perform appropriate interpolation. Just do it.

On the other hand, the time calculation unit, and the speed of sound c in the virtual space, a position b ₁ at time t ₁ of the observation object, and the position b ₂ at time t _2, the from the movement history of the stored the sound source object, Time u ₁ and time u ₂ are calculated. At this time, with respect to the position s _{1 at} the time u ₁ of the sound source object and the position s _{2 at} the time u ₂ , the condition “the distance between the position b ₁ and the position s ₁ is c (t ₁ -u ₁ ) equally, and the distance between the position b ₂ and the position s ₂ calculates the time u ₁ and the time u ₂ satisfying "equal to c (t ₂ -u _2).

  In the normal calculation of the Doppler effect, the frequency change is calculated based on the moving speed of the sound source object and the observation object. In the present invention, the time when the positional relationship between the sound source object and the observation object satisfies a predetermined condition. Thus, it is possible to perform a simulation in consideration of the time delay until the sound wave reaches from the sound source object to the observation object as well as the change in frequency.

Further, the sound output unit generates a sound waveform having a time length (u ₂ -u ₁ ) generated by the sound source object between time u ₁ and time u ₂ calculated in the virtual space, as a time length (t ₂ The speech waveform expanded and contracted to -t ₁ ) and expanded in time is output as a speech waveform observed from time t ₁ to time t ₂ by the observation object.

By changing the sound waveform of time length (u ₂ -u ₁ ) to the sound waveform of time length (t ₂ -t ₁ ) in the time direction, it is possible to appropriately simulate a change in frequency due to the Doppler effect. In addition, the sound wave waveform that the sound source object emits from time u ₁ to time u ₂ is the sound wave waveform observed by the observation object from time t ₁ to time t ₂ , The time delay based on the distance between the two can be appropriately simulated.

  According to the present invention, the Doppler effect in the case where the sounding body and the observation body move freely can be appropriately simulated by simple calculation.

Further, in the sound simulation device of the present invention, the sound output unit has the time length expanded and contracted based on the distance between the position b ₁ and the position s ₁ and the distance between the position b ₂ and the position s ₂ . The audio waveform can be expanded and contracted, and the audio waveform whose amplitude is expanded and contracted can be output as a speech waveform observed from time t ₁ to time t ₂ by the observation object.

That is, it simulates a situation in which the intensity (volume) of a sound wave waveform that arrives due to a change in distance is attenuated. For example, the sound volume can be adjusted by using an audio waveform observed from time t ₁ to time t ₂ , an attenuation factor 1 based on the distance between position b ₁ and position s _1, and the distance between position b ₂ and position s _2. also may be outputted at an average gain of the attenuation factor 2 based on, in the attenuation factor 1 at time t _1, with the attenuation factor 2 in time t _2, the smooth during which the attenuation factor 1 to the attenuation factor 2 It is also possible to output at an amplification factor that changes to

  According to the present invention, the attenuation rate based on the distance between the sounding body and the observing body can be appropriately simulated by a simple calculation in accordance with the Doppler effect. Become.

In the sound simulation apparatus of the present invention, when u ₁ ≧ u ₂ , the sound output unit generates a predetermined shock sound waveform at the time t (t ₂ -t ₁ ) at the observation object at time t. It may be configured to output a voice waveform which is observed from ₁ to time t _2.

  In the above invention, it is possible to cope with a situation where both of them move at a speed lower than the speed of sound. In the technology disclosed in the prior art, under supersonic movement, the frequency becomes negative and the sound is played back in reverse. However, in the present invention, such reverse playback actually occurs. First, the situation where a shock wave is generated is appropriately simulated.

  According to the present invention, a shock wave generated under supersonic movement is approximated by a predetermined shock sound waveform, and a situation where the shock wave reaches can be easily simulated by appropriately adjusting the playback time and playback time. You will be able to

The speech simulation apparatus of the present invention further includes a repeat control unit, and t ₂ = t ₁ + ΔT with respect to a predetermined time step width ΔT. The repeat control unit applies the observation object to the observation object. When the sound waveform observed from time t ₁ to time t ₂ is output, time t ₁ and time t ₂ are each increased by ΔT, time u ₂ is set at time u ₁ , and time calculation unit Can be configured to cause the time u ₂ to be calculated again.

The present invention relates to a preferred embodiment of the above invention, wherein a time step on the observation object side is Δt, and a sound wave waveform section in the sound source object is obtained for each section divided by the step width. This is reproduced by expanding / contracting it to the time length Δt. At this time, since the time u ₂ obtained in the previous repeat unit corresponds to the time u ₁ in the next repeat unit, the substitution is performed.

  According to the present invention, it is possible to appropriately suppress the calculation amount by using the fact that the sections are continuous when the speech simulation is repeated at the time interval.

In the sound simulation method according to another aspect of the present invention, in a virtual space in which a sound source object that emits sound and an observation object that observes sound move, the observation object observes from time t ₁ to time t _2. Executed in a speech simulation apparatus including a history storage unit, a time calculation unit, and an audio output unit for simulating a sound waveform to be stored and storing a movement history of the sound source object in the virtual space. And is configured as follows.

That is, in the time calculation step, the moving time calculation unit, and the sound velocity c in the virtual space, a position b ₁ at time t ₁ of the observation object, and the position b ₂ at time t _2, the of the stored the sound source object From the history, time u ₁ and time u ₂ are calculated. At this time, with respect to the position s _{1 at} the time u ₁ of the sound source object and the position s _{2 at} the time u ₂ , the condition “the distance between the position b ₁ and the position s ₁ is c (t ₁ -u ₁ ) equally, and the distance between the position b ₂ and the position s ₂ calculates the time u ₁ and the time u ₂ satisfying "equal to c (t ₂ -u _2).

On the other hand, in the audio output process, the audio output unit generates an audio waveform of the time length (u ₂ -u ₁ ) generated by the sound source object between time u ₁ and time u ₂ calculated in the virtual space, The speech waveform expanded and contracted to the time length (t ₂ -t ₁ ) and expanded and contracted is output as a speech waveform observed from time t ₁ to time t ₂ by the observation object.

A program according to another aspect of the present invention is configured to cause a computer to function as the above-described sound simulation apparatus and to cause the computer to execute the above-described sound simulation method.
The program of the present invention can be recorded on a computer-readable information storage medium such as a compact disk, flexible disk, hard disk, magneto-optical disk, digital video disk, magnetic tape, and semiconductor memory.
The above program can be distributed and sold via a computer communication network independently of the computer on which the program is executed. The information storage medium can be distributed and sold independently from the computer.

  According to the present invention, it is possible to provide an audio simulation apparatus, an audio simulation method, and a program that realizes these on a computer, which can easily simulate the Doppler effect.

  Embodiments of the present invention will be described below, but the embodiments described below are for explanation and do not limit the scope of the present invention. Therefore, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention.

  FIG. 1 is a schematic diagram showing a schematic configuration of a typical information processing apparatus that performs the function of the speech simulation apparatus of the present invention by executing a program. Hereinafter, a description will be given with reference to FIG.

  The information processing apparatus 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an interface 104, a controller 105, an external memory 106, and an image processing unit 107. A DVD-ROM (Digital Versatile Disc ROM) drive 108, a NIC (Network Interface Card) 109, an audio processing unit 110, and a microphone 111.

  A DVD-ROM storing a game program and data is loaded into the DVD-ROM drive 108 and the information processing apparatus 100 is turned on to execute the program, thereby realizing the sound simulation apparatus of the present embodiment. Is done.

  The CPU 101 controls the overall operation of the information processing apparatus 100 and is connected to each component to exchange control signals and data. Further, the CPU 101 uses arithmetic operations such as addition / subtraction / multiplication / division, logical sum, logical product, etc. using an ALU (Arithmetic Logic Unit) (not shown) for a storage area called a register (not shown) that can be accessed at high speed. , Logic operations such as logical negation, bit operations such as bit sum, bit product, bit inversion, bit shift, and bit rotation can be performed. In addition, the CPU 101 itself is configured so that saturation operations such as addition / subtraction / multiplication / division for multimedia processing, vector operations such as trigonometric functions, etc. can be performed at a high speed, and those provided with a coprocessor. There is.

  The ROM 102 records an IPL (Initial Program Loader) that is executed immediately after the power is turned on, and when this is executed, the program recorded on the DVD-ROM is read out to the RAM 103 and execution by the CPU 101 is started. The The ROM 102 stores an operating system program and various data necessary for operation control of the entire information processing apparatus 100.

  The RAM 103 is for temporarily storing data and programs, and holds programs and data read from the DVD-ROM and other data necessary for game progress and chat communication. Further, the CPU 101 provides a variable area in the RAM 103 and performs an operation by directly operating the ALU on the value stored in the variable, or temporarily stores the value stored in the RAM 103 in the register. Perform operations such as performing operations on registers and writing back the operation results to memory.

  The controller 105 connected via the interface 104 receives an operation input performed when the user executes the game.

  The external memory 106 detachably connected via the interface 104 stores data indicating game play status (past results, etc.), data indicating game progress, and log of chat communication in the case of a network match ( Data) is stored in a rewritable manner. The user can record these data in the external memory 106 as appropriate by inputting an instruction via the controller 105.

  A DVD-ROM mounted on the DVD-ROM drive 108 stores a program for realizing the game and image data and audio data associated with the game. Under the control of the CPU 101, the DVD-ROM drive 108 performs a reading process on the DVD-ROM loaded therein, reads out necessary programs and data, and these are temporarily stored in the RAM 103 or the like.

  The image processing unit 107 processes the data read from the DVD-ROM by an image arithmetic processor (not shown) included in the CPU 101 or the image processing unit 107, and then processes the processed data on a frame memory ( (Not shown). The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to a monitor (not shown) connected to the image processing unit 107. Thereby, various image displays are possible.

  The image calculation processor can execute a two-dimensional image overlay calculation, a transmission calculation such as α blending, and various saturation calculations at high speed.

  Also, polygon information arranged in the virtual three-dimensional space and added with various texture information is rendered by the Z buffer method, and the polygon arranged in the virtual three-dimensional space from the predetermined viewpoint position is determined in the direction of the predetermined line of sight It is also possible to perform high-speed execution of operations for obtaining rendered images.

  Further, the CPU 101 and the image arithmetic processor operate in a coordinated manner, so that a character string can be drawn as a two-dimensional image in a frame memory or drawn on the surface of each polygon according to font information defining the character shape. is there.

  The NIC 109 is used to connect the information processing apparatus 100 to a computer communication network (not shown) such as the Internet, and is based on the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network). To connect to the Internet using an analog modem, ISDN (Integrated Services Digital Network) modem, ADSL (Asymmetric Digital Subscriber Line) modem, cable television line A cable modem or the like and an interface (not shown) that mediates between these and the CPU 101 are configured.

  The audio processing unit 110 converts audio data read from the DVD-ROM into an analog audio signal and outputs the analog audio signal from a speaker (not shown) connected thereto. Further, under the control of the CPU 101, sound effects and music data to be generated during the progress of the game are generated, and sound corresponding to this is output from the speaker.

  When the audio data recorded on the DVD-ROM is MIDI data, the audio processing unit 110 refers to the sound source data included in the audio data and converts the MIDI data into PCM data. If the compressed audio data is in ADPCM format or Ogg Vorbis format, it is expanded and converted to PCM data. The PCM data can be output by performing D / A (Digital / Analog) conversion at a timing corresponding to the sampling frequency and outputting it to a speaker.

  Furthermore, a microphone 111 can be connected to the information processing apparatus 100 via the interface 104. In this case, the analog signal from the microphone 111 is subjected to A / D conversion at an appropriate sampling frequency so that processing such as mixing in the sound processing unit 110 can be performed as a PCM format digital signal.

  In addition, the information processing apparatus 100 uses a large-capacity external storage device such as a hard disk so as to perform the same function as the ROM 102, the RAM 103, the external memory 106, the DVD-ROM mounted on the DVD-ROM drive 108, and the like. You may comprise.

  The information processing apparatus 100 described above corresponds to a so-called “consumer video game apparatus”, but the present invention can be realized as long as it performs image processing to display a virtual space. Therefore, the present invention can be realized on various computers such as a mobile phone, a portable game device, a karaoke apparatus, and a general business computer.

  For example, a general computer, like the information processing apparatus 100, includes an image processing unit that includes a CPU, RAM, ROM, DVD-ROM drive, and NIC and has simpler functions than the information processing apparatus 100. In addition to having a hard disk as an external storage device, a flexible disk, a magneto-optical disk, a magnetic tape, and the like can be used. Further, not the controller 105 but a keyboard or a mouse is used as an input device.

(Doppler effect model)
In the following, a basic simulation method of the Doppler effect in the case where the sound source and the observer freely move in the virtual space and sometimes the moving speed exceeds the sound speed will be described.

The coordinates of the sound source S at time t are
s (t) = (s _x (t), s _y (t), s _z (t))
And the coordinates of the observer B at time t are
b (t) = (b _x (t), b _y (t), b _z (t))
And s (t) and b (t) are position vectors. In addition, the speed of sound is c.

  FIG. 2 is an explanatory diagram showing a positional relationship when the sound source S and the observer B move. Hereinafter, a description will be given with reference to FIG.

In this figure, the sound source S (201) moves from position s ₁ = s (u ₁ ) to position s ₂ = s (u ₂ ) from time u ₁ to time u ₂ . It is assumed that the sound wave waveform 221 generated during this time arrives at the observer B (202) as the sound wave waveform 222 from time t ₁ to time t ₂ (in this figure, for the sake of easy understanding, attenuation by distance) Is omitted).

The observer B (202) moves from the position b ₁ = b (t ₁ ) to the position b ₂ = b (t ₂ ) from time t ₁ to time t ₂ .

Then, the sound wave waveform 221 propagates at the sound velocity c to become a sound wave waveform 222. Therefore, if the calculation for obtaining the vector length is | · | and the arrival time of the sound wave is taken into consideration,
c (t ₁ -u ₁ ) = | s (u ₁ ) -b (t ₁ ) |
And the end
c (t ₂ -u ₂ ) = | s (u ₂ ) -b (t ₂ ) |
Is established.

Generalization is performed based on this model. That is, it is assumed that the sound wave emitted from the sound source S at time u has reached the observer B at time t.
c (tu) = | s (u)-b (t) |
The relationship is established.

When the operation for calculating the inner product of the vector a and the vector b is expressed as a · b, this relationship is
c ² (tu) ² = (s (u)-b (t)) ・ (s (u)-b (t));
Where t ≧ u
Can also be written.

In general, given a time t, the condition
c (tu) = | s (u)-b (t) |
The function to find the time u that satisfies
u = g (t)
Suppose that

In addition, an attenuation coefficient when sound waves are transmitted from the position vector p to the position vector q is k (p, q). Typically,
k (p, q) = 1 / ((pq) ・ (pq));
k (p, q) = 1 / | pq |
For example, various calculation methods can be employed.

  Here, the displacement (deviation from the reference point) of the sound wave generated by the sound source S at time t is defined as a (t). That is, a (t) is calculated by gradually increasing the time t in a predetermined sampling period unit, and the calculated a (t) is a sound wave waveform at that time, and the value a is sent to the speaker at the time t. If (t) is output, the sound wave emitted by the sound source S is reproduced.

Under such circumstances, the displacement of the sound wave that the observer B listens at time t is
a (g (t)) × k (s (g (t)), b (t))
Can be written. The first half of this equation is due to the Doppler effect, and the second half is due to sound wave attenuation.

  Therefore, a (g (t)) × k (s (g (t)), b (t)) is calculated by gradually increasing the time t in units of a predetermined sampling period, and the calculated a (g ( t)) × k (s (g (t)), b (t)) is the sound wave waveform at that time, and the value a (g (t)) × k (s (g If (t)) and b (t)) are output, the sound wave heard by the observer B is reproduced.

That is, the sound wave waveform observed by the observer B from time t ₁ to time t ₂ is the sound wave generated by the sound source S from time u ₁ = g (t ₁ ) to time u ₂ = g (t ₂ ). This is due to the waveform. The amplification factor (attenuation factor) for the original waveform at time t ₁ is k (s (g (t ₁ )), b (t ₁ )), and the amplification factor for the original waveform at time t ₂ is k (s (g (t ₂ )), b (t ₂ )).

  The problem here is how to find the function u = g (t).

  Particularly in a virtual environment, the position b (t) of the observer B and the position s (t) of the sound source S at time t are updated based on various instruction operations by the user, based on random numbers, or The computer is often updated based on a predetermined algorithm. Therefore, it is difficult to analytically obtain the function u = g (t).

  Therefore, a practical solution of this function u = g (t) is obtained by using a temporally discrete approximation. First, let ΔT be the step size of the discrete approximation time. For example, when the simulation is performed by a computer or the like, it is typical to employ the interval of the vertical synchronization signal as ΔT.

Then, the position vector of the sound source S and the position vector of the observer B at time t = iΔT are stored in the i-th element of the arrays S and B, respectively.
S [i] = s (iΔT);
B [i] = b (iΔT)

  That is, in this description, the history of the positions of the sound source S and the observer B is stored in the arrays S and B.

  As a method for limiting the length of the array, the position vector at time iΔT may be stored in the i mod L-th element using a ring buffer. Here, L is the length of the sequence.

  Hereinafter, in order to facilitate understanding, it is assumed that the position history is stored indefinitely. However, when the above-described ring buffer or the like is employed, only valid subscripts need be processed.

To obtain the value of the function u = g (t) = g (iΔT) at time t = iΔT, first, an integer k _i where k _i <i,
The magnitude relationship between c (iΔT-k _i ΔT) and | s (k _i ΔT)-b (iΔT) |
The magnitude relationship between c (iΔT-(k _i +1) ΔT) and | s ((k _i +1) ΔT)-b (iΔT) |
However, look for k _i that seems to be reversed. That is,
c (iΔT-k _i ΔT) ≤ | s (k _i ΔT)-b (iΔT) | and c (iΔT-(k _i +1) ΔT) ≥ | s ((k _i +1) ΔT)-b ( iΔT) |
c (iΔT-k _i ΔT) ≧ | s (k _i ΔT)-b (iΔT) | and c (iΔT-(k _i +1) ΔT) ≦ | s ((k _i +1) ΔT)-b ( iΔT) |
Find k _i that becomes. This is referred to as a “reverse condition”.

here,
c (iΔT-k _i ΔT) = c (ik _i ) ΔT;
s (k _i ΔT) −b (iΔT) = S [k _i ] −B [i];
c (iΔT-(k _i +1) ΔT) = c (ik _i -1) ΔT;
s ((k _i +1) ΔT)-b (iΔT) = S [k _i +1]-B [i]
Therefore, by scanning the array in the direction in which the subscript i (current time) decreases, such a k _i should be found.

Also, once k _i is found, in the step of time t = (i + 1) ΔT, if the vicinity of k _i found last time is searched, the scanning time is shortened and k _{i + 1} is found. Should be able to. If it cannot be found, it is assumed that the sound wave has not reached.

  When a ring buffer is used, the number of scanning steps is set to L at the maximum, and mod L may be used when referring to by a subscript.

If such a k _i is found,
c (iΔT-u) = | s (u)-b (iΔT) |
U = g (t) = g (iΔT) that satisfies the above can be obtained by interpolation.

When adopting the simplest linear interpolation,
g (iΔT) = k _i ΔT + ΔT × (c (ik _i ) ΔT-| S [k _i ]-B [i] |) / (| S [k _i +1]-B [i] |-c (ik _i -1) ΔT);
And Other interpolation methods include, for example:
g (iΔT) = k _i ΔT + ΔT × (c ² (ik _i ) ² ΔT ^2- | S [k _i ]-B [i] | ² ) / (| S [k _i +1]-B [i ] | ² -c ² (ik _i -1) ² ΔT ² )
For example, various interpolation techniques can be employed.

  In addition, when one of the equal signs of magnitude relation is materialized, the value at that time is used as it is, and when both are materialized simultaneously, two values are averaged.

Now, it was sought in this way
u = g (iΔT)
Is also stored in the i-th element G [i] of the array G.
G [i] = g (iΔT)

  Array G can also be a ring buffer. Also, only the latest two elements of the array G are important.

Further, the position of the sound source S at time u = g (iΔT) = G [i] is obtained by calculating the ratio between the vector S [k _i ] and the vector S [k _i +1] by linear interpolation.
(G [i]-k _i ΔT): ((k _i +1) ΔT-G [i])
It is a point to divide internally. The position vector of this point is denoted as S ′ [i], and this is also stored in the array in the same manner as described above.

Now, in the case of an actual Doppler effect, the displacement observed by the observer B between times (i−1) ΔT to iΔT is
a (G [i-1]) × k _i (S ′ [i-1], B [i-1]) to a (G [i]) × k _i (S ′ [i], B [i] )
Should be.

  In other words, the displacement of the original sound source a (u) at time u = G [i-1] to G [i] is multiplied by an appropriate attenuation coefficient so that it fits exactly at time t = (i-1) ΔT to iΔT. If the waveform is expanded and contracted in the time direction at the same time, it is most suitable for the actual Doppler effect.

  However, in simulation, reproduction cannot be performed retroactively. Therefore, the displacement of the original sound source a (u) at time u = G [i-1] to G [i] is multiplied by an appropriate attenuation coefficient so that it fits exactly at time t = iΔT to (i + 1) ΔT. The Doppler effect is simulated by expanding and contracting the waveform in the time direction.

Also, to simplify the calculation of the attenuation coefficient,
k _i (S '[i-1], B [i-1]);
k _i (S '[i], B [i])
It is possible to adopt various modes such as using only one of these, using an average of these, or gradually changing from the former to the latter.

  So far, both the sound source S and the observer B have considered the Doppler effect when moving at less than the speed of sound, but special processing is required when operating at supersonic speed.

In other words, when moving below the speed of sound,
u = g (t)
Is a monotonically increasing function,
G [i-1] <G [i]
Is established.

However, if there is a supersonic movement,
G [i-1] ≧ G [i]
It can happen.

  In this case, since it is considered that the observer B is in a situation where the shock wave is straddled, it is not prepared to reverse the sound reproduction, but is prepared separately in advance between time t = iΔT to (i + 1) ΔT. What is necessary is just to play the impact sound source.

  At this time, the same attenuation coefficient as described above may be considered as the volume of the impact sound, or the impact sound may be reproduced at a constant volume.

If it is less than the speed of sound, scan the array S to look for k _i , look for the neighborhood of k _i previously found,
k _i <k _{i + 1}
Therefore, it is only necessary to scan from k _i in the direction in which the value increases, but in the case where there is a supersonic speed, it is necessary to scan the vicinity of k _i in both directions.

In the case of performing the processing as described above, it is sufficient for the arrays S ′ and G to hold only the latest two pieces of information. For the array B, it is sufficient that only the latest one or two pieces of information can be held. Therefore, the size of the array may be changed as appropriate. Also, instead of “array”, two variables may be used. For example, when the current time is t = iΔT, the latest values to be stored in the elements B [i] and S '[i] of the array are stored in the variables b ₂ and s ₂ , respectively. The values to be stored in the element B [i-1] are stored in the variables b ₁ and s ₁ , respectively.

  Below, based on said view, the detail of embodiment of this invention is further demonstrated.

(Details of the embodiment)
FIG. 3 is an explanatory diagram showing a schematic configuration of the speech simulation apparatus according to the present embodiment. Hereinafter, a description will be given with reference to FIG.

  The voice simulation apparatus 301 includes a history storage unit 302, a time calculation unit 303, a voice output unit 304, and a repetition control unit 305.

  The voice simulation apparatus 301 performs a simulation discretely with a time interval ΔT. As the length of ΔT, for example, a vertical synchronization interrupt cycle that is a unit of various repeated calculations in a game device or the like can be employed.

The history storage unit 302 is realized by the RAM 103, and the following variable areas are secured.
(A) Counter i representing the current time. This counter is incremented by 1 every time ΔT elapses.
(B) An array S for storing a history of position vectors of a sound source object that is a sound source S (201). This array is configured as a ring buffer of length L, and the position vector s (iΔT) of the sound source object at time t = iΔT is stored in the i mod L-th element S [i mod L] of the array S The In order to facilitate understanding, hereinafter, S [i mod L] is simply written as S [i].
(C) A variable b ₁ = b ((i−1) ΔT) for storing a position vector at the time t = (i−1) ΔT of the observation object that is the observer B (202), and a current time t = iΔT A variable b ₂ = b (iΔT) that stores the position vector at.
(D) A variable s ₁ = g ((i−1) ΔT) for storing a position vector of the sound source S (201) corresponding to the variable b ₁ and a position vector of the sound source S (201) corresponding to the variable b ₂ Variable to store s ₂ = g (iΔT).
(E) Times u ₁ and u ₂ . The sound wave waveform observed in the latest time interval t = (i−1) ΔT to iΔT is attributed to the sound wave waveform generated by the sound source object from time u ₁ to time u ₂ . It should be noted that information indicating “unknown” can be substituted for these time variables u ₁ and u ₂ . This is used when the corresponding time is not found.
(F) An amplification attenuation factor m ₁ corresponding to times t ₁ and u ₁ and an amplification attenuation factor m ₂ corresponding to times t ₂ and u ₂ .
(G) Temporary counter variable k. Used in repeated processing.
(H) Various other information necessary for physical simulation of sound source objects and observation objects. For example, when these are realized as an airplane, various conditions such as position, speed, posture, operation amount such as a ladder, altitude, atmospheric pressure, and wind force are stored.

  FIG. 4 is a flowchart showing a flow of control of the voice simulation process executed by the voice simulation apparatus. Hereinafter, a description will be given with reference to FIG.

  When this process is started, the CPU 101 initializes the history storage unit 302 to an appropriate value (step S401).

Next, the CPU 101 stores the current position of the observation object in the variable b ₁ (step S402).

Furthermore, the current position of the sound source object is stored in all the array elements S [0], S [1],..., S [L-1] as the initial value for the dummy, and the current position of the sound source object is stored in the variable s ₁ . Store the value meaning “not found” in the variable u ₁ and store the value 1 in the variable m ₁ . (Step S404).

  Then, it waits for the time ΔT (step S405). Various other processes can be executed during this waiting time. For example, calculate the physical simulation of each object, calculate the position of the sound source object or observation object at the next time t = (i + 1) ΔT, or execute another process in a coroutine manner. Execute.

When the standby is completed, the current time counter i is updated as i ← i + 1 (step S406). Here, “←” means “assignment”. That is, the current time t ₂ = iΔT, and the immediately preceding time t ₁ = (i−1) ΔT.

Further, CPU 101 stores the current position of the sound source object in the array element S [i] (step S407), and stores the current position of the observation object to a variable b ₂ (step S408).

  Then, the CPU 101 repeats the following processing for each of the variables k = 0, 1, 2, 3,..., L−1 (step S409).

That is,
d (i, h) = c × h-| S [ih] -b ₂ |
The reverse condition
d (i, k) × d (i, k-1) ≦ 0
Whether or not is satisfied is checked (step S410).

This reversal condition is
The magnitude relationship between c × k and | S [ik]-b ₂ |
c × (k-1) and | S [i- (k-1)]-b ₂ |
It is established when is reversed.

  If not established (step S410; No), the processing from step S409 is repeated (step S411). Note that the repetition for k does not necessarily have to be in the order of 0, 1,.

That is, the value of k found last time may be set in the variable k ′, and the vicinity thereof may be searched first, and if it is not found, the remaining value may be searched. For example, using an appropriate constant W (W <L),
k'-W, k'-W + 1, k'-W + 2, ..., k'-W + L-1
It is a method of searching in the order of. As described above, mod L is performed on the subscript of the array S. Therefore, the array S can be appropriately accessed by changing k as described above.

If established (step S410; Yes), the vector is between the vector (k, d (i, k)) and the vector (k-1, d (i, k-1)), and Time u _{2 at} which the value becomes (u ₂ , 0) is obtained by interpolation (step S412).

Similarly, a vector between a vector (S [ik], d (i, k)) and a vector (S [i- (k-1)], d (i, k-1)), A position vector s ₂ whose value is (s ₂ , 0) is obtained by interpolation (step S413).

For example, when performing linear interpolation of the above model, the following calculation and substitution are performed.
u ₂ ← [d (i, k) (k-1) -d (i, k-1) k] / [d (i, k) -d (i, k-1)];
s ₂ ← [d (i, k) S [k-1]-d (i, k-1) S [k]] / [d (i, k) -d (i, k-1)]
Thus, the CPU 101 operates as the time calculation unit 303 in cooperation with the RAM 103 and the like.

When u ₂ is obtained as described above, whether or not u ₁ and u ₂ are valid is compared with the magnitude (step S414).

If u ₁ <u ₂ (step S414; <), the amplification attenuation factor m ₂ is
m ₂ ← k (s ₂ , b ₂ )
(Step S415).

Next, a sound wave waveform generated by the sound source object from time u ₁ to time u ₂ is acquired (step S416).

  For example, when the sound wave waveform is prepared by a PCM sound source, the waveform of the corresponding time section of the PCM sound source may be acquired. When the sound wave waveform is prepared by a MIDI sound source, it is reproduced in the time interval. What is necessary is just to acquire performance information.

Then, the time length u ₂ -u ₁ of the acquired sound wave waveform is expanded or contracted to ΔT (step S417). This is a process corresponding to frequency conversion in the prior art of the Doppler effect. When a sound wave waveform is considered to be a two-dimensional figure with the horizontal axis as time elapsed and the vertical axis as displacement, the expansion / contraction process corresponds to a process of expanding / contracting the two-dimensional figure in the horizontal axis direction. In the case of a PCM sound source, what is equivalent to such graphic expansion / contraction processing may be directly performed, and when the audio processing unit 107 has a function of changing the pitch corresponding thereto, this may be used. In the case of a MIDI sound source, the pitch at which the performance information is reproduced may be changed in accordance with the time length ratio.

Further, in the expanded time length ΔT sound wave waveform, while changing the amplification attenuation factor smoothly so that the amplification attenuation factor for the beginning of the interval is m ₁ and the amplification attenuation factor for the end of the interval is m ₂ , Expansion and contraction in the waveform displacement direction (step S418). This processing can also be realized by causing the CPU 101 to change the waveform or cause the audio processing unit 107 to perform processing in the same manner as the pitch change described above.

Then, the observation object as the observed wave waveform from time t ₁ to time t _2, the output of the sound wave waveforms stretch in the time direction and the displacement direction (step S419). Specifically, an instruction may be issued to the audio processing unit 107 so as to add the expanded and contracted sound wave waveform of the time length ΔT to the reproduction audio buffer prepared by the audio processing unit 107.

  As described above, the CPU 101 functions as the audio output unit 304 in cooperation with the RAM 103 and the audio processing unit 107.

And
b ₁ ← b ₂ ;
s ₁ ← s ₂ ;
u ₁ ← u ₂ ;
m ₁ ← m ₂
Thus, each variable is updated to prepare for the next repetition unit (step S420), and the process returns to step S405.

  Therefore, the CPU 101 functions as the repetition control unit 305 in cooperation with the RAM 103.

On the other hand, if the repetition of steps S409 to S411 ends without u ₂ being found, “undiscovered” is substituted for variable u ₂ (step S421), and the observation object is observed from time t ₁ to time t _2. As the sound wave waveform, “silence” of time length ΔT (speech waveform whose displacement is always 0) is output (step S422), and the process proceeds to step S420. In step S414, if at least one of u ₁ and u ₂ is “not found” (step S414; not found), the process proceeds to step S422.

In addition, in step S414, when u ₁ ≧ u ₂ (step S414; ≧), the impact sound having the time length ΔT is used as the sound wave waveform observed by the observation object prepared in advance from time t ₁ to time t _2. The waveform is output (step S423), and the process proceeds to step S420.

  The impact sound waveform represents “sound” when the shock wave passes. For example, a method of using a speech waveform such as white noise or sampling and using explosion sounds can be considered.

In the above embodiment, in step S418, the speech waveform is expanded and contracted in the displacement direction so as to change the amplification factor smoothly from m ₂ to m _1, but the average of these two amplification factors may be adopted, or the amplification factor There may be a technique such as adopting only m ₂ . Further, depending on the application, it is possible to use a voice waveform expanded and contracted in the time direction as it is without considering attenuation due to distance.

In the case of real-time simulation, since the current time is equivalent to t _2, "time t ₁ from to time t ₂ to the observation object is observed waveform" becomes a be output at time t ₂ later, [Delta] T Although the above time delay occurs, when a short time such as the vertical synchronization interrupt period is adopted as ΔT, this delay is acceptable to the user without a sense of incongruity.

  In the case of off-line (not real-time) simulation, if the speech waveforms output in steps S419, S422, and S423 are simply connected, the speech waveform observed by the observation object can be obtained as it is in the simulation time zone. Will be.

  As described above, according to the present embodiment, the Doppler effect can be easily and realistically simulated even when the sound source and the observer freely move in the virtual space or when the sound speed is exceeded. It becomes like this.

  As described above, according to the present invention, it is possible to provide an audio simulation apparatus, an audio simulation method, and a program that realizes these on a computer, which can easily simulate the Doppler effect.

It is a schematic diagram which shows the outline | summary structure of the typical information processing apparatus which performs the function of the audio | voice simulation apparatus of this invention by running a program. It is explanatory drawing which shows the positional relationship at the time of the sound source S and the observer B moving. It is explanatory drawing which shows schematic structure of the audio | voice simulation apparatus which concerns on this embodiment. It is a flowchart which shows the flow of control of the audio | voice simulation process performed with this audio | voice simulation apparatus.

Explanation of symbols

100 Information processing apparatus 101 CPU
102 ROM
103 RAM
104 Interface 105 Controller 106 External Memory 107 Image Processing Unit 108 DVD-ROM Drive 109 NIC
DESCRIPTION OF SYMBOLS 110 Audio | voice processing part 111 Microphone 201 The movement path | route of the sound source S 202 The movement path | route of the observer B 221 The sound wave waveform which the sound source S emits 222 The sound wave waveform which the observer B observes 301 Voice simulation apparatus 302 History storage part 303 Time calculation part 304 Voice output Part 305 Repeat control part

Claims (5)

A sound simulation device that simulates a sound waveform observed from time t ₁ to time t ₂ in a virtual space in which a sound source object that emits sound and an observation object that observes sound move. The observation object moves in the virtual space based on a user instruction,
A history storage unit for storing a movement history of the sound source object in the virtual space;
And sonic c in the virtual space, a position b ₁ at time t ₁ of the observation object, and the position b ₂ at time t _2, the a movement history of said stored the sound source object from the time u ₁ and the time u a time calculating unit for calculating a _2, the distance between the position s ₁ at time u ₁ of the sound source object, the position s ₂ at time u _2, with respect to a condition "position b ₁ and the position s ₁ Time calculation unit that calculates time u ₁ and time u ₂ satisfying `` equal to c (t <sub>1</sub> -u <sub>1</sub> ) and the distance between position b <sub>2</sub> and position s <sub>2</sub> is equal to c (t <sub>2</sub> -u <sub>2</sub> ) '' ,
The sound waveform of the time length (u <sub>2</sub> -u <sub>1</sub> ) emitted by the sound source object between the calculated time u <sub>1</sub> and time u <sub>2</sub> in the virtual space is expanded or contracted to the time length (t <sub>2</sub> -t <sub>1</sub> ). And an audio output unit for outputting the audio waveform expanded and contracted as the time length as an audio waveform observed from time t <sub>1</sub> to time t <sub>2 on the</sub> observation object,
When u <sub>1</sub> ≧ u <sub>2</sub> , the sound output unit observes a predetermined shock sound waveform for a time length (t <sub>2</sub> -t <sub>1</sub> ) from the time t <sub>1</sub> to the time t <sub>2 on the</sub> observation object. A voice simulation apparatus characterized by outputting as a voice waveform. The speech simulation apparatus according to claim 1,
The speech waveform emitted by the sound source object is represented by pulse code modulation data,
T <sub>2</sub> = t <sub>1</sub> + ΔT for a given time step ΔT,
When a voice waveform observed from time t <sub>1</sub> to time t <sub>2</sub> is output from the sound output unit by the voice output unit, time t <sub>1</sub> and time t <sub>2</sub> are increased by ΔT, respectively, and time t <sub>1</sub> The speech simulation apparatus further comprising: a repetition control unit that sets u <sub>2</sub> and causes the time calculation unit to perform the calculation of time u <sub>2</sub> again. The speech simulation apparatus according to claim 1 or 2,
Based on the distance between the position b <sub>1</sub> and the position s <sub>1</sub> and the distance between the position b <sub>2</sub> and the position s <sub>2</sub> , the sound output unit expands and contracts the amplitude of the sound waveform whose time length is expanded and contracted, A speech simulation apparatus, wherein the speech waveform whose amplitude is expanded and contracted is output as a speech waveform observed from time t <sub>1</sub> to time t <sub>2</sub> by the observation object. In a virtual space in which a sound source object that emits sound and an observation object that observes sound move, the sound waveform observed from time t <sub>1</sub> to time t <sub>2 on the</sub> observation object is simulated, A sound simulation method executed by a sound simulation apparatus including a history storage unit that stores a movement history of the sound source object, a time calculation unit, and a sound output unit, wherein the observation object is based on a user instruction Move through space,
The time calculation unit is based on the sound speed c in the virtual space, the position b <sub>1</sub> of the observation object at time t <sub>1,</sub> the position b <sub>2 at</sub> time t <sub>2,</sub> and the stored movement history of the sound source object. a time calculating step of calculating the time u <sub>1</sub> and the time u <sub>2,</sub> and the position s <sub>1</sub> at time u <sub>1</sub> of the sound source object, the position s <sub>2</sub> at time u <sub>2,</sub> with respect to a condition "position b <sub>1</sub> position The time u <sub>1</sub> and the time u <sub>2</sub> satisfying “the distance from s <sub>1</sub> is equal to c (t <sub>1</sub> -u <sub>1</sub> ) and the distance between the position b <sub>2</sub> and the position s <sub>2</sub> is equal to c (t <sub>2</sub> -u <sub>2</sub> )” Time calculation process to calculate,
The audio output unit, the sound source object emitted time length between the time u <sub>1</sub> that is the calculated in the virtual space to time u <sub>2</sub> speech waveform (u <sub>2</sub> -u <sub>1),</sub> the time length (t <sub>2</sub> -t <sub>1</sub> ), and a speech output step of outputting the speech waveform expanded and contracted as the time length as a speech waveform observed from time t <sub>1</sub> to time t <sub>2 on the</sub> observation object,
In the audio output step, when u <sub>1</sub> ≧ u <sub>2</sub> , a predetermined impact audio waveform is observed from the time t <sub>1</sub> to the time t <sub>2 on the</sub> observation object for a time length (t <sub>2</sub> -t <sub>1</sub> ). A voice simulation method characterized by outputting as a voice waveform. A program that causes a computer to simulate a sound waveform observed from time t <sub>1</sub> to time t <sub>2</sub> in a virtual space in which a sound source object that emits sound and an observation object that observes sound move. The observation object moves in the virtual space based on a user instruction, and the program
A history storage unit for storing a movement history of the sound source object in the virtual space;
And sonic c in the virtual space, a position b <sub>1</sub> at time t <sub>1</sub> of the observation object, and the position b <sub>2</sub> at time t <sub>2,</sub> the a movement history of said stored the sound source object from the time u <sub>1</sub> and the time u a time calculating unit for calculating a <sub>2,</sub> the distance between the position s <sub>1</sub> at time u <sub>1</sub> of the sound source object, the position s <sub>2</sub> at time u <sub>2,</sub> with respect to a condition "position b <sub>1</sub> and the position s <sub>1</sub> Time calculation unit that calculates time u <sub>1</sub> and time u <sub>2</sub> satisfying `` equal to c (t ₁ -u ₁ ) and the distance between position b ₂ and position s ₂ is equal to c (t ₂ -u ₂ ) '' ,
The sound waveform of the time length (u ₂ -u ₁ ) emitted by the sound source object between the calculated time u ₁ and time u ₂ in the virtual space is expanded or contracted to the time length (t ₂ -t ₁ ). Then, the voice waveform whose time length is expanded and contracted is made to function as a voice output unit that outputs the voice waveform observed from the time t ₁ to the time t _{2 on the} observation object,
When u ₁ ≧ u ₂ , the sound output unit observes a predetermined shock sound waveform for a time length (t ₂ -t ₁ ) from the time t ₁ to the time t _{2 on the} observation object. A program that functions to output as a voice waveform.

Images