JPH0984199A - Stereoscopic acoustic processor using linear prediction coefficient - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the tap coefficient of a filter, to miniaturize the filter, to reduce cost and to speed up an arithmetic processing by adding a desired acoustic characteristic to an original signal by using the linear synthetic filter in which the linear prediction coefficient obtained by the linear prediction analysis of the impulse response showing an acoustic characteristic is made a filter coefficient. SOLUTION: An acoustic adder 28 for performing a sound image localization by the acoustic filter using a linear prediction coefficient is composed of acoustic characteristic addition filters 35 and 37 and filters 36 and 38, etc., eliminating the acoustic characteristics of the headphones of left and right channels. The acoustic characteristic addition filters 35 and 37 are composed of IIR filters 54 and 55 to which frequency characteristics are added by using the linear prediction coefficients, delay parts 56 and 57 imparting pitch and the time difference, etc., till the characteristics are attained to and left and right ears by the inut means, and amplifiers 58 and 59 individually controlling gain at an output stage. The filters 36 and 38 are composed of the FIR filters using the linear prediction coefficients. A filter coefficient selection means 39 performs the selection settings of the filter coefficients within each filter parameter, pitch/delay time and gains, etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound processing technique, and more particularly to a stereophonic sound processing device for providing a stereoscopic sound effect to a listener in a sound field reproduced through headphones or the like.

[0002]

2. Description of the Related Art Generally, in order to accurately reproduce or localize a sound image, acoustic characteristics of an original sound field from a sound source to a listener and sound of a reproduced sound field from an acoustic output device such as a speaker or headphones to a listener. It is necessary to obtain the characteristics and. In the actual playback sound field, by adding the former acoustic characteristic to the sound source and removing the latter acoustic characteristic from the sound source, the sound image of the original sound field is accurately reproduced to the listener even when using a speaker or headphones. The position of the original sound image can be accurately localized.

FIG. 1 shows the case of listening to a sound image from a conventional 2-channel stereo device. FIG. 2 shows a basic circuit block configuration for realizing an acoustic space equivalent to that of FIG. 1 with headphones. In FIG. 1, the transfer characteristics of each acoustic space path from the left and right speakers (L, R) 1 and 2 to the left and right ears (1, r) of the listener 3 are represented by Ll,
It is represented as Lr, Rr, and Rl. 2, in addition to the transfer characteristics 11 to 14 of the acoustic space paths shown in FIG. 1, the transfer characteristics (Hl ^-1 , Hr ^-1 ) 15,1
6 is added.

As shown in FIG. 2, by adding the transfer characteristics 11 to 16 to the original signals (L signal and R signal), the outputs from the headphones 5 and 6 accurately reproduce the signals from the speakers 1 and 2. Therefore, the listener can have an illusion that he or she is listening to the signals from the speakers 1 and 2.

FIG. 3 shows a circuit configuration example of a conventional FIR filter (non-recursive filter) for realizing the above-mentioned transfer characteristics. In general, a filter simulating the transfer characteristics 11 to 14 of each acoustic space path and the inverse characteristics 15 and 16 of the transfer characteristics from the headphones to the ears shown in FIG. 2 includes an impulse of each acoustic space path shown in the following formula (1). An FIR filter (non-recursive filter) having a response as a coefficient is used.

[Equation 1]

The filter coefficients (a0, a1, a2, ..., An) representing the transfer characteristics 11 to 14 of the acoustic space path are the filters based on the impulse response obtained by acoustic measurement or acoustic simulation for each path. The coefficient is used. In order to add a desired acoustic characteristic to the original signal, the impulse response indicating the acoustic characteristic of each path is convoluted through these filters.

The inverse characteristics (Hl ^-1 ,
Hr ⁻¹ ) 15 and 16 filter coefficients (a0, a1, a
2,. . . , An) is obtained in the frequency domain. First, the frequency characteristic of the headphones is measured to obtain the inverse characteristic, and then the impulse response obtained by returning the result to the time domain is used as the coefficient of the filter.

Next, FIG. 4 shows an example of a general system configuration in the case of moving a sound image in accordance with a computer graphics (CG) image. In FIG. 4, the CG display device 2 is operated by the user / software.
The controller 26 of 4 is a CG accelerator 2 for displaying an image.
5 and provides the controller 29 of the stereophonic device 27 with position information of the sound image synchronized with the image.

The acoustic characteristic adder 28 controls each channel so that the sound image is localized at an image display position within the display screen of the display 21 or at a virtual position outside the display screen under the control of the controller 29 based on the position information. Speaker 22
And 23 (or headphones) audio output signal levels.

FIG. 5 shows the basic structure of the acoustic characteristic adder 28 shown in FIG. The acoustic characteristic adder 28 is the FI of FIG.
Acoustic characteristic adding filters 35 and 37 that give transfer characteristics of the acoustic spatial paths Sl and Sr from the sound source S to the left and right ears using the R filter, and acoustic characteristic removing filters 36 and 38 of the headphones L and R channels, and further, the above It is composed of a filter coefficient selecting section 39 which selectively gives the filter coefficients of the acoustic characteristic adding filters 35 and 37 based on the position information.

FIGS. 6 to 8 are for explaining a conventional sound image localization technique used in the acoustic characteristic adder 28 of FIG. FIG. 6 generally shows the relationship between the sound source and the listener. The transfer characteristics Sl and Sr of the acoustic space between the sound source 30 and the listener 31 are the same as those in FIG. 1 described above.

FIG. 7A shows the acoustic characteristic adder 2 of FIG.
Sound source for localizing one sound source in S8 (S)
30 (S → l) 35, (S → r) 37, and inverse characteristics (h ⁻¹ ) 36, 38 of the transfer characteristics of the headphones 33, 34 of the acoustic space path between the listener 30 and the listener 31.
This is an example. FIG. 7B shows the configuration of the acoustic characteristic addition filters 35 and 37 when the sound image 30 is further localized at a plurality of sound image positions P to Q.

8 (a) and 8 (b) are shown in FIG. 7 (b).
3 shows a more specific circuit block configuration example of the acoustic characteristic adding filters 35 and 37 of FIG. (A) of FIG.
Is the acoustic characteristic addition filter 3 for the left ear of the listener 31.
5 shows the configuration of FIG. 5 and shows the transfer characteristics of each acoustic space between the plurality of sound image positions P to Q shown in FIG. 7B and the listener 31 (P → l), ..., (Q → l), a plurality of amplifiers g _Pl , which individually control their respective output gains,
, G _Ql , and an adder for adding and outputting the outputs of the respective amplifiers.

8B is the same as FIG. 8A except that the configuration of the acoustic characteristic addition filter 37 for the right ear of the listener 31 is shown. The gains of the respective amplifiers of the acoustic characteristic adding filters 35 and 37 are controlled by the position information indicating any one of the sound image positions P to Q, whereby the sound image 30 becomes one of the instructed sound image positions P to Q.
Localize.

FIG. 9 shows an example of a surround type sound image localization. FIG. 9 shows an example of a surround system in which five speakers (L, C, R, SR, SL) are arranged around the listener 31. In this example, the sound image can be localized around the listener 31 by relatively controlling the output levels from the five sound sources.

For example, the sound image can be localized during the relative change between the output levels of the speaker L and the speaker SL shown in FIG. Therefore, it is understood that the above-described conventional technique can be applied as it is to the case of such sound image localization.

[0017]

However, when the impulse response is conventionally measured in a normal room for the coefficient of the FIR filter of FIG. 3, each transfer characteristic L
The number of taps of the FIR filter representing l, Lr, Rr, and Rl is 44.
Thousands of taps or more are required when 1 KHz is used. In addition, the inverse characteristic HL ⁻¹ of the transfer characteristic of the headphones,
Even in the case of HR ^-1 , several hundred taps or more are required.

Therefore, when the FIR filter shown in FIG. 3 is used, the number of taps and the amount of calculation thereof become enormous, and when a circuit is actually constructed, a plurality of general-purpose DSPs and a dedicated convolution calculation processor are required, which is low. There were problems in terms of price reduction and miniaturization. Further, when the sound image is localized, a plurality of channel filters for each sound image position must be processed in parallel, which makes it more difficult to solve the above problems.

Further, in general, the amount of image processing such as CG in real time is very large, and when the image processing apparatus has a small capacity or a large number of images are displayed at the same time, for example, a frame drop occurs due to insufficient processing capacity. There may be a case where continuous images cannot be displayed like a video. In such a case, since the motion of the sound image is controlled in synchronization with the motion of the image, there is a problem that the motion of the sound image becomes discontinuous. Furthermore, when the viewing environment such as the viewing position of the user is different from the previously planned environment, there is a problem that the apparent movement of the image does not match the movement of the sound image.

Therefore, in view of the above various problems, an object of the present invention is to perform linear prediction analysis of an impulse response representing an acoustic characteristic to be added to an original signal in order to add an acoustic characteristic, and use the linear prediction coefficient. By constructing a synthesizing filter, the tap coefficients of the filter can be significantly reduced, and hardware miniaturization, cost reduction, and high-speed arithmetic processing can be achieved.

Another object of the present invention is to divide the acoustic characteristics from a plurality of positions for localizing a sound image to a listener into a characteristic common to each position and a characteristic peculiar thereto and arrange filters in series to add them. By controlling the position of the sound image, the amount of calculation processing is reduced.

Further, according to the present invention, when the sound image is moved, one sound image is localized at a plurality of positions in the reproduced sound field, and the sound image level between the positions is controlled by controlling the level difference of the sound output between the positions. It is to realize the movement of the sound image in accordance with the interpolated position by interpolating the position of the image which is discontinuously moved and is interpolated.

[0023]

According to the present invention, there is provided a stereophonic processing apparatus for localizing a sound image by using a virtual sound source, wherein a desired acoustic characteristic to be added to an original signal is represented by an impulse response representing the acoustic characteristic. There is provided a stereophonic processing apparatus which is formed by a linear synthesis filter having a linear prediction coefficient obtained by a linear prediction analysis of a response as a filter coefficient and adds desired acoustic characteristics to the original signal through the linear synthesis filter.

The linear synthesis filter has a IIR filter configuration using the linear prediction coefficient and adds a frequency characteristic of a desired acoustic characteristic to the original signal, and an IIR also using the linear prediction coefficient. It includes a pitch synthesizing filter having a filter structure and adding a time characteristic of a desired acoustic characteristic to the original signal. The pitch synthesis filter is composed of a pitch synthesis section for direct sound with a large attenuation rate, a pitch synthesis section for subsequent reflected sound with a small attenuation rate, and a delay section for giving a delay time thereof.

Further, the inverse characteristic of the acoustic characteristic of the acoustic output device such as headphones or speaker is formed by a linear prediction filter using a linear prediction coefficient obtained from a linear prediction analysis of an impressive response representing the acoustic characteristic as a filter coefficient. Then, the acoustic characteristics of the acoustic device are removed through the linear prediction filter. The linear prediction filter is composed of an FIR filter using the linear prediction coefficient.

Further, according to the present invention, the stereophonic acoustic processing device for locating a sound image by using a virtual sound source is: an impulse response representing each acoustic characteristic of one or a plurality of acoustic paths to the left ear to be added to an original signal. Acoustic prediction device obtained by the linear prediction analysis of the first acoustic property addition filter including a linear synthesis filter using the linear prediction coefficient as a filter coefficient; and an audio output device to the left ear connected in series to the first acoustic property addition filter. The first prediction sound composed of a linear prediction filter having a linear prediction coefficient obtained by a linear prediction analysis of an impulse response representing the sound characteristics of the sound output device as an inverse characteristic for removing the sound characteristics of the sound output device. Characteristic removal filter;

A linear synthesis filter having a linear prediction coefficient obtained by a linear prediction analysis of an impulse response representing each acoustic characteristic of one or more acoustic paths to the right ear added to the original signal as its filter coefficient. A second acoustic characteristic addition filter; a linear prediction coefficient obtained by a linear prediction analysis of an impulse response that is connected in series to the second acoustic characteristic addition filter and represents the acoustic characteristics of an acoustic output device to the right ear, A second acoustic characteristic removing filter comprising a linear prediction filter having a filter coefficient that gives an inverse characteristic for eliminating the acoustic characteristic of the output device;

Further, there is provided a stereophonic processing apparatus comprising a selection setting section for selectively setting a predetermined parameter corresponding to the first acoustic characteristic addition filter and the second acoustic characteristic addition filter according to position information of a sound image. To be done.

The first and second acoustic characteristic addition filters include a common section for adding a common characteristic to the acoustic characteristics of each acoustic path, and a unique section for adding a unique characteristic to each acoustic characteristic of each acoustic path. , And the overall acoustic characteristics are added by connecting the common part and the unique part in series.

Further, it has a storage medium for storing the calculation result of the common part for a predetermined sound source and a read instructing part for instructing to read out the stored operation result, and the read instructing part reads out according to the instruction. The calculated result is directly given to the proper part. Further, in the storage medium, in addition to the calculation result of the common unit for the predetermined sound source, the calculation result of the corresponding first or second acoustic characteristic removal filter may be stored together.

Further, each of the first acoustic characteristic addition filter and the second acoustic characteristic addition filter further includes a delay section for giving a delay time between the two ears, and the first acoustic characteristic addition filter of the first or second acoustic characteristic addition filter. By setting either one of the delay times between the ears among the delay sections as a reference (delay time zero), the reference delay section can be omitted.

The first acoustic characteristic addition filter and the second acoustic characteristic addition filter
The acoustic characteristic addition filter further includes an amplification unit capable of variably setting the output signal levels thereof, and the selection setting unit sets the first sound by the gain setting of the amplification unit according to the position information of the sound image. The localization position of the sound image can be moved by relatively varying the output signal levels of the characteristic addition filter and the second acoustic characteristic addition filter.

The first and second filters for adding acoustic characteristics may be arranged symmetrically with respect to the front of the listener.
In that case, the parameters of the delay unit and the amplification unit are shared between the corresponding positions on the left and right.

Further, according to the present invention, the stereophonic processing apparatus has a position information interpolating unit for interpolating position information in the middle of the position information of the past and future sound images, and the interpolation from the position information interpolating unit. The position information is given as position information to the selection setting unit. Similarly, it has a position information prediction unit that predictively interpolates the position information of the future from the position information of the past and present sound images, and the future position information from the position information prediction unit is given as position information to the selection setting unit. To be

The position information predicting unit further includes a regularity determining unit that determines whether or not there is regularity in the moving direction of the past and present sound image position information, and the position information predicting unit includes the regularity determining unit. If it is determined that there is regularity, the future position information is given. It should be noted that instead of the position information of the sound image, image position information from an image display device in which an image emitting the sound image is displayed can be used. The selection setting unit may move the listening environment according to the given position information of the listener in order to further provide / maintain a good listening environment of the listener.

According to the present invention, the linear synthesis filter having the linear prediction coefficient obtained by the linear prediction analysis of the impulse response representing the desired acoustic characteristic added to the original signal is constructed. Next, the linear prediction coefficient is corrected so that the time domain envelope (time characteristic) and spectrum (frequency characteristic) of this linear synthesis filter become equal to or close to the original impulse response. Acoustic characteristics are added to the original sound by using the corrected linear synthesis filter. Since the time domain envelope and spectrum are equal to or close to the original impulse response, this linear synthesis filter can be used to add acoustic characteristics equal to or close to the desired acoustic characteristics.

In this case, by constructing the linear synthesis filter by the IIR filter (recursive filter) pitch configuration filter and the short-term synthesis filter, a linear synthesis filter having a significantly smaller number of taps than the conventional configuration can be constructed. . Here, the time domain envelope is controlled by the pitch synthesis filter, and the spectrum is mainly controlled by the short term synthesis filter.

Further, according to the present invention, the acoustic characteristic added to the input signal is divided into a characteristic common to each position for sound image localization and a characteristic unique to the filter. When adding acoustic characteristics, these filters are connected in series and used.
As a result, it is possible to reduce the total calculation processing amount. In this case, the greater the number of unique characteristics, the greater the effect of the reduction.

Further, by storing the result of processing the common characteristic portion in a storage medium such as a hard disk in advance, the above-mentioned storage is used for applications such as a game in which a sound to be used is determined. Simply by reading the signal directly from the medium, it can be input to a filter that adds unique acoustic properties to each location that requires real-time processing. Therefore, not only the amount of calculation is reduced, but also the storage capacity is smaller than in the case where all information is simply stored in the storage medium.

Further, the output signal obtained by inputting to the acoustic characteristic removing filter may be stored in the storage medium together with the output signal of the filter for adding the characteristic common to each position. In this case, it is not necessary to perform the processing of the acoustic characteristic removal filter in real time. Thus, by using the storage medium, the sound image can be moved with a small amount of processing.

Further, according to the present invention, it is possible to continuously move the sound image by interpolating the position of the image in which the motion is discontinuous and moving the sound image in accordance with the interpolated position.
Further, by inputting the viewing environment of the user to the image controller and the sound image controller and controlling the motions of the image and the sound image using the information, the apparent motion of the image and the motion of the sound image can be matched.

[0042]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 10 shows a principle configuration for obtaining a linear synthesis filter for adding acoustic characteristics according to the present invention. In the present invention, the filter representing the transfer characteristics Ll, Lr, Rl, Rr shown in FIG. Therefore, the impulse response of each acoustic space path that represents each of the above transfer characteristics is measured in an anechoic room in which reflected sounds and residual sounds are excluded, and linear prediction analysis processing 41 is performed based on the measured impulse response to determine the linear shape of the impulse response. Calculate the prediction coefficient.

Further correction processing 42 is applied to the linear prediction coefficient.
And the resulting coefficients are IIR according to the invention.
The linear prediction coefficient of the linear synthesis filter 40 of the filter configuration is set. Therefore, the original signal that has passed through the linear synthesis filter 40 is added with the frequency characteristic that is one of the acoustic characteristics of the acoustic space path.

FIG. 11 shows an example of the configuration of a linear synthesis filter for adding acoustic characteristics according to the present invention. In FIG. 11, the linear synthesis filter 40 includes a short-term synthesis filter 44 and a pitch synthesis filter 43, which are represented by the following equations (2) and (3), respectively.

[0045]

[Equation 2]

The former short-term synthesis filter 44 is constructed as an IIR filter having a linear prediction coefficient obtained from a linear prediction analysis of impulse responses representing each transfer characteristic, and gives a listener a sense of direction. The latter pitch synthesizing filter 43 gives an initial reflected sound and a residual sound to the original sound.

FIG. 12 shows how to obtain the linear prediction coefficients (b1, b2, ..., Bm) of the short-term synthesis filter 44 and the pitch coefficients L and bL of the pitch synthesis filter 43. First, the autocorrelation coefficient processing 45 of the impulse response measured in the anechoic room is used to obtain the autocorrelation coefficient, and then the linear prediction analysis processing 46 is performed. The linear prediction coefficients (b1, b2, ..., Bm) obtained as a result are used to configure the short-term synthesis filter 44 (IIR filter) of FIG. With the IIR filter configuration using the linear prediction coefficient, it is possible to add the frequency characteristic, which is one of the transmission characteristics, with the number of taps significantly smaller than the number of samples of the original impulse response. For example, 256 taps can be significantly reduced to about 10 taps.

Delays and gains representing the time difference and the level difference in which the signals reach each ear through each path, which is another transmission characteristic, are added by the delay (Z- ^d ) and the gain (g) in FIG. . In FIG. 12, the linear prediction coefficients (b1, b2, ..., Bm) obtained by the linear prediction analysis process 46 are expressed by the following formula (4) showing the inverse characteristic of the short-term synthesis filter 44. The short-term prediction filter 47 (FIR filter) ) Is used as a coefficient.

[0049]

(Equation 3)

As can be seen from equations (2) and (4), by passing through the short-term prediction filter 47, the short-term synthesis filter 4 can be obtained.
The frequency component equivalent to the acoustic characteristic added in 4 can be removed conversely. As a result, the delay (Z ^−L ) and the gain (bL) can be obtained from the time component remaining in the pitch extraction processing 48 of the next stage. The obtained value is given as the delay and coefficient of the pitch synthesis filter 43. From the above, it can be seen that the transfer characteristic representing the desired acoustic characteristic having both the frequency characteristic and the time characteristic can be realized by the circuit configuration of FIG. 11.

FIG. 13 shows an example of one block configuration of the pitch synthesis filter 43. Separate pitch synthesis filters are used for so-called direct sound and reflected sound. The impulse response obtained by measuring the sound field is
Usually, a part with a large attenuation rate (direct sound) starts, and then a part with a small attenuation rate (reflected sound) follows. Therefore, the pitch synthesizing filter 43 includes the former pitch synthesizing filter unit 49 relating to the direct sound, the latter pitch synthesizing filter unit 51 shown in FIG.
Can be configured separately. Note that the direct sound portion may be configured by an FIR filter, or the direct sound portion and the reflected sound portion may be overlapped.

FIG. 14 shows an example of correction processing of the linear prediction coefficient obtained as described above. FIG.
Time domain envelope and spectrum evaluation processing 49
In the above, the once obtained short-term synthesis filter 54 and the pitch synthesis filter 53 are connected in series and compared with the impulse response of the desired acoustic characteristic, whereby the time domain envelope and spectrum of the impulse response of the linear synthesis filter are the original. The filter coefficient correction process is performed so as to be equal to or closer to the impulse response.

FIG. 15 shows an example of a filter configuration showing the inverse characteristics Hl ^-1 , Hr ^-1 of the transfer characteristics of the headphones according to the present invention. The filter 53 of FIG. 15 has the same configuration as the short-term prediction filter 47 shown in FIG.
The linear prediction analysis is performed by obtaining the autocorrelation coefficient of the impulse response of the headphones, and the obtained linear prediction coefficient (c
1, c2 ,. . . , Cm) to show the opposite characteristic from FIR
Form a linear predictive filter. As a result, 1/10 of the impulse response of the conventional inverse characteristic shown in FIG.
The frequency characteristic of the headphones can be removed by the following filter with a small number of taps. It is not necessary to consider the time difference and level difference between the two ears by assuming that the characteristics between the ears are mutually symmetrical.

FIG. 16 shows an example of frequency characteristics of the sound addition filter according to the present invention in comparison with the conventional one. In FIG. 16, the solid line is the conventional 25 shown in FIG.
The case of the frequency characteristic of the acoustic characteristic addition filter having 6 taps is shown, and the dotted line shows the case of the frequency characteristic of the acoustic characteristic addition filter having 10 taps (using only the short-term synthesis filter) according to the present invention. According to the present invention, it can be seen that the spectrum approximation is performed with a significantly smaller number of taps than the conventional one.

Next, FIG. 17 shows a basic block configuration for performing sound image localization by the acoustic filter using the linear prediction coefficient according to the present invention. Figure 17
The acoustic characteristic adding filters 35 and 37 correspond to the acoustic adding device 28 of FIGS. 4 and 5 described above, and the acoustic characteristic adding filters 35 and 37 add the frequency characteristic using the linear prediction coefficient according to the present invention.
R filters 54 and 55, delay units 56 and 5 that give pitches and time differences before reaching the left and right ears at their input stages
7, and an amplifier 5 that controls the gain individually at the output stage
It is composed of 8,59. Further, the filters 36 and 38 for removing the acoustic characteristics of the headphones of the left and right channels are configured by FIR filters using the linear prediction coefficient according to the present invention.

Here, of the acoustic characteristic adding filters 35 and 37, the short-term synthesis filter 44 described in FIG. 11 is used for the IIR filters 53 and 54, and the delay circuits 55 and 56 are also the delay circuits of FIG. (Z- ^d ) is used.
In addition, a filter 36 for removing the acoustic characteristics of the headphones,
The FIR type linear prediction filter 53 described with reference to FIG. 15 is used for 38. Therefore, each of the above filters will not be described further here. The filter coefficient selecting means 39 selects and sets a filter coefficient, a pitch / delay time, a gain, etc., among the above filter parameters.

FIG. 18 shows an example in which the sound image localization described in FIG. 9 is realized by using the sound adding device 28 according to the present invention in FIG. Five speakers (L, C,
Five virtual sound sources corresponding to R, SR, and SL) are acoustic characteristic addition filters (Cl to SRl and Cr to SRr) 54 to
57 is placed in a similar position, and the headphones 3
The acoustic characteristics of 3, 34 are removed by the removal filters 36, 38. This environment becomes the same as that shown in FIG. 9 for the listener. Therefore, the virtual sound sources (L, C, R, SR, SL) are changed by changing the gains of the amplifiers 58 and 59 in the level adjusting unit 39 as described in FIG. The sound volume from can be varied, thereby localizing the sound image to surround the listener.

FIG. 19 shows another configuration example of the sound adding device according to the present invention, which is basically the above-mentioned FIG.
7 has the same configuration as that of No. 7, but is different in that a position information interpolation / prediction unit 60 and a regularity determination unit 61 are newly provided. 20 to 24 are explanatory diagrams regarding the functions of the position information interpolation / prediction unit 60 and the regularity determination unit 61 of FIG. 19.

20 to 22 relate to interpolation of position information. As shown in FIG. 20, the image controller 62 (see FIG.
Corresponding to the CG display device 24 of FIG.
Corresponding to the stereophonic sound device 27), the future position information regarding the image processing is postponed prior to the image processing having a long processing time. In FIG. 20, the position information interpolation / prediction unit 60 included in the sound image controller 63 uses the future image position, the current position, and the past position as shown in FIG. 21 to display the display 21 (see FIG. 4). Interpolates the upper sound image position.

Here, a method of interpolating the value of the x-axis in the Cartesian coordinate system (x, y, z) of the image position will be described with reference to FIG. Note that the y-axis and z-axis values can be similarly determined. In FIG. 22, t0 is the current time, t−1 ,. . . , T−m is past time, and t + 1 is future time. Here, using the Taylor series expansion, times t + 1 ,. . . , T−m, it is assumed that x (t) is expressed by the following equation. x (t) = a0 + a1 * t + a2 * t ² +. . . + An * t ⁿ (n ≦ m + 1) (5.1)

The values x (t + 1) ,. . . , X (tm), the coefficients a0 ,. . . , An, the x-axis value x (t ′) at time t ′ (t0 <t ′ <t + 1) can be obtained. here,

[0062]

[Equation 4]

As in the above case, the future position can be predicted by interpolating the x-axis value. For example, using the prediction coefficients b1, ..., Bn, the prediction value x ′ (t + 1) at time t + 1 is calculated by the following equation. x ′ (t + 1) = b1 * x (t) + b2 * x (t−1) +. . . + Bn * x (t-n + 1) (5.4) The prediction coefficients b1, ..., Bn in the above equation are the current and past values x (t), ..., x (t-m). It is obtained by performing a linear prediction analysis from the autocorrelation coefficient of. Alternatively, it can be obtained by trial and error using an algorithm such as the maximum gradient method.

23 and 24 show a method of predicting a future position by determining whether or not the motion of an image has regularity. For example, when the regularity determiner 64 of FIG. 23 corresponding to the regularity determination unit 61 of FIG. 19 obtains the prediction coefficients b1, ..., Bn in the above equation (5.4) using linear prediction analysis, If a stable coefficient of the prediction system is obtained, it is determined that there is a rule in the motion of the image.
Alternatively, in the same equation (5.4), the prediction coefficients b1, ...
.., bn is determined by trial and error using a predetermined adaptive algorithm, and it is determined that the motion of the image has regularity when the coefficient values converge to a certain value. Then, only in the case of making such a judgment, the coefficient obtained from the equation (5.4) is adopted as the future position information.

Although the above has described the interpolation / prediction of the sound image position on the display according to the position information of the image given by the user or software, the position information may be the position information of the listener. 25 and 26 show an example in which the sound image is optimally localized according to the position information of the listener.

FIG. 25 shows an example in which, in the system of FIG. 4, the listener 31 moves away from the appropriate viewing environment range surrounded by the diagonal lines, and the loci of the sound image position and the image position do not match for the listener 31. Shows. Even in such a case, by constantly monitoring the position of the listener 31 with a position sensor or the like according to the present invention, the viewing environment range is automatically moved to the listener 31 side as shown in FIG. It is possible to match the sound image with the image according to the viewing environment of. This is the reason why the sound image controller 63 is controlled according to the viewing environment in FIG. The method described so far is used as it is for moving the sound image localization position. That is,
The L and R channel signals are controlled to move the viewing environment range to the user's position.

FIG. 27 shows an embodiment of streamlining the arithmetic processing according to the present invention. FIG. 27 is a diagram illustrating a calculation unit (C → l) 64 and (C → r) 65 common to each of the acoustic characteristic addition filters 35 and 37 shown in FIG. And an operation unit (P → l) to (Q → r) 66 unique to each filter
This is intended to further improve the efficiency / speed of the arithmetic processing as compared with the conventional example described in FIG. 8 by avoiding redundant arithmetic processing between them.

The common arithmetic units 64 and 65 are connected in series with their own arithmetic units 66 to 69. Also,
Amplifiers g _{Pl to} g _Qr for controlling the level difference between both ears and the position of the sound image are connected to each of the unique arithmetic units 66 to 69. Here, the acoustic characteristic from the virtual sound source (C) located in the middle of two or a plurality of real sound sources (P to Q) to the listener is used as the common characteristic of the acoustic characteristics.

FIG. 28A shows a processing system for obtaining the linear prediction coefficient of the common characteristic by using the impulse response representing the acoustic characteristic from the virtual sound source C to the listener. In this example, the acoustic characteristic of C → l is shown, but the acoustic characteristic of C → r is also the same. Note that, for further commonality, it is possible to assume that the virtual sound source C is in front of the listener and that the acoustic characteristics C → l and C → r are equal to each other. Generally, a Hamming window or the like is used for the windowing process 70, and the Levinson-Durbin method is used for the linear prediction analysis.

FIG. 28 (b) shows a processing system for obtaining a linear prediction coefficient representing the characteristic characteristic of the acoustic characteristics from the real sound sources P to Q to the listener. A filter (C → l) ^-17 for removing the common characteristic from the impulse response representing each acoustic characteristic
2 or (C → r) ⁻¹ 73, and the output thereof is subjected to linear prediction analysis to obtain a linear prediction coefficient representing a characteristic peculiar to each acoustic characteristic. A linear prediction coefficient having a common characteristic is set in the filters 72 and 73 by the same method as described with reference to FIG. As a result, the filter characteristics (P → l) to (Q → l) and (P → r) to the unique filter characteristics are removed in advance with the common characteristic portion removed from the unique impulse responses.
The linear prediction coefficient of (Q → r) is obtained.

29 and 30 show a common part according to the present invention.
Acoustic characteristics in which the characteristic part is separated and they are connected in series
An example of the additional filters 35 and 37 is shown.
You. The common parts 64 and 65 in FIG. 29 are the same as the short parts described in FIG.
A linear combination consisting of a phase synthesis filter and a pitch synthesis filter.
It is composed of the composition filter itself, and the unique parts 66 to 69
Is a binaural in addition to a short-term synthesis filter that represents individual frequency characteristics.
Delay device Z for controlling the time difference between^-DP, Z^-DQAnd the level difference
And an amplifier g for controlling the position of the sound image _Pl~ G_Qlso
Be composed.

FIG. 30 shows an example of the acoustic characteristic addition filter between the two sound sources L and R and the listener. Here, the pitch synthesis filter is not used in the common units 64 and 65 in consideration of the consistency with the description of FIGS. 31 to 33 described below. 31 to 33 show an example of frequency characteristics of the acoustic characteristic addition filter shown in FIG.

The two sound sources L and R in FIG.
The sound sources S1 and S2 shown in (a) and (b) of FIG. 3 are respectively arranged with an opening of 30 degrees when viewed from the listener. FIG. 31B is a block diagram of the acoustic characteristic addition filter of FIG. 30 and shows the measurement system of FIGS. 32 and 33.

The dotted line in FIG. 32 is the common part (C → l) in FIG.
33, and the dotted line in FIG. 33 shows the frequency characteristic when the common part and the unique part (s1 → l) are connected in series. Here, the number of taps of the conventional filter shown by the solid line is 256, and the number of taps of the short-term synthesis filter according to the present invention shown by the dotted line is 10 in total for C → 1 and 4 for s1 → l.
It is as a tap. Note that the pitch synthesis filter is not used as described above. Therefore, it can be seen that the greater the number of unique parts, the greater the effect of reducing the calculation amount.

FIG. 34 shows that the common characteristics of the common sections 64 and 65 shown in FIG. 30 are stored in the storage medium 74 such as a hard disk as the original signal already added as sound data. In FIG. 35, the storage medium 7 is replaced with the common characteristic calculation process.
4 shows that the signal to which the common characteristic has already been added is read out from No. 4 and is given to the proper portions 66 to 69. In the example of FIG. 35, the listener can read the signal for which the common characteristic has been calculated from the storage medium 74 by operating the sound image control device 75 when necessary. The read signal is localized at a desired sound image position by performing a calculation for adding a characteristic and adjusting an output gain thereof. According to the present invention, real-time arithmetic processing of common characteristics is unnecessary. The storage medium 7
In addition to the common characteristics, the signal stored in 4 can include up to the inverse characteristic calculation processing of headphones whose processing content is fixed.

FIG. 36 shows processing that is symmetrical to the listener.
It shows an example of performing. In FIG. 36, two virtual
Using sound sources A and B, level g between them_Al, G_Ar,
g _Bl, G_BrThe sound image S is localized by the difference of
You. Here, the left side is based on the listener's center line (dashed line)
Perform target processing on the right. That is, the virtual sound on the left side of the center line
Sources A and B and their locations relative to the centerline
The virtual sound sources (A) and (B) on the right side are
A substantially similar acoustic space shall be formed.

As shown in FIG. 37, the listener's surroundings are divided into n equal parts, virtual sound sources A and B are placed at each boundary, and the sound corresponding to the transmission path from each virtual sound source to both ears 1 and r. The characteristics are left and right as shown in FIG. As a result, only the coefficients 0, ..., N / 2−1 need to be actually held. The sound image position for the listener is represented by a counterclockwise angle θ from the front as shown in FIG. 39. Next, from the angle θ, which section of the section equally divided into n in FIG. 37 contains the sound image is calculated from the following equation (6).

Section number = (round down to the nearest whole number) θ / (2π / n) (6) In addition, when the levels g _Al , g _Ar , g _Bl , and g _Br of the virtual sound source are to be obtained, there are left and right objects. The angle θ is converted from the condition as shown in Expression (7). θ = θ (0 ≦ θ <π) (7) or 2π−θ (π ≦ θ ≦ 2π) In this way, the left and right are shared with the left and right coefficients such as delay and gain representing the acoustic characteristics. can do. When θ obtained in FIG. 39 is π ≦ θ ≦ 2π, lch
And rch output signals are exchanged and output to headphones. Thereby, the sound image calculated as being on the left side of the listener can be localized on the right side.

FIG. 40 shows an example of the acoustic characteristic addition filter for processing the above-mentioned symmetrical system. The characteristic of this acoustic characteristic addition filter is that the transmission path A →
By performing the delay process of r B → r on the basis of the delay of A → l and B → l respectively, the delay process of A → l and B → l can be omitted. Therefore, the delay process indicating the interaural time difference can be halved.

[0080]

As described above, according to the present invention, a desired sound image localization is performed by a plurality of virtual sound sources. Therefore, even when the number or position of sound images changes, the acoustic characteristics from the virtual sound source to the listener are changed. Therefore, it is not necessary to change the configuration of the linear synthesis filter. In addition, desired acoustic characteristics can be added to the original signal with a filter having a small number of taps.

Further, according to the present invention, the acoustic characteristic added to the input signal is divided into the characteristic common to each position for sound image localization and the characteristic unique to the filter. Therefore, the calculation of the common characteristic of each filter is performed once. Therefore, it is possible to reduce the total calculation processing amount. In this case, the greater the number of unique characteristics, the greater the effect of the reduction.

Further, by storing the result of processing the common characteristic part in a storage medium such as a hard disk in advance, it is possible to read a signal directly from the storage medium and to make it unique to each position requiring real-time processing. It can be input to a filter that adds acoustic properties. Therefore, not only the amount of calculation is reduced, but also the storage capacity is smaller than in the case where all information is simply stored in the storage medium.

The output signal obtained by inputting to the acoustic characteristic removing filter may be stored in the storage medium together with the output signal of the filter that adds a common characteristic to each position. In this case, it is not necessary to perform the processing of the acoustic characteristic removal filter in real time. Thus, by using the storage medium, the sound image can be moved with a small amount of processing.

Further, according to the present invention, it is possible to continuously move the sound image by interpolating the position of the image in which the motion is discontinuous and moving the sound image in accordance with the interpolated position.
Further, by inputting the viewing environment of the user to the image controller and the sound image controller and controlling the motions of the image and the sound image using the information, the apparent motion of the image and the motion of the sound image can be matched.

[Brief description of drawings]

FIG. 1 is a diagram showing an example of listening to a stereoscopic sound image from a 2-channel stereo device.

FIG. 2 is a diagram showing a configuration example of an acoustic space equivalent to that of FIG. 1 using headphones.

FIG. 3 is a diagram showing an example of a conventional FIR filter.

FIG. 4 is a diagram showing a configuration example of a CG display device and a stereophonic device.

5 is a diagram showing a basic configuration example of the acoustic characteristic adder of FIG.

FIG. 6 is an explanatory diagram (1) of a conventional sound image localization technique.

FIG. 7 is an explanatory diagram (2) of a conventional sound image localization technique.

FIG. 8 is an explanatory diagram (3) of a conventional sound image localization technique.

FIG. 9 is a diagram showing an example of a surround-type sound image localization.

FIG. 10 is a principle configuration diagram for obtaining a linear synthesis filter that adds acoustic characteristics according to the present invention.

FIG. 11 is a diagram showing a basic configuration of a linear synthesis filter for adding acoustic characteristics according to the present invention.

FIG. 12 is a diagram showing an example of how to obtain a linear prediction coefficient and a pitch coefficient.

FIG. 13 is a diagram showing a configuration example of a pitch synthesizing filter.

FIG. 14 is a diagram showing an example of correction processing of a linear prediction coefficient.

FIG. 15 is a diagram showing an example of an FIR filter that realizes an inverse characteristic of a transfer characteristic by using a linear prediction coefficient.

FIG. 16 is a diagram showing an example of frequency characteristics of the acoustic characteristic addition filter according to the present invention.

FIG. 17 is a diagram showing an example of the basic configuration of a sound adding device according to the present invention.

18 is a diagram showing an example of a surround type sound image localization by the sound adding device of FIG.

FIG. 19 is a diagram showing another configuration example of the sound adding device according to the present invention.

FIG. 20 is an explanatory diagram (1) of interpolation of position information.

FIG. 21 is an explanatory diagram (2) of interpolation of position information.

FIG. 22 is an explanatory diagram (3) of interpolation of position information.

FIG. 23 is an explanatory diagram (1) of prediction of position information.

FIG. 24 is an explanatory diagram (2) of prediction of position information.

FIG. 25 is an explanatory diagram (1) for localizing a sound image according to the position information of the listener.

FIG. 26 is an explanatory diagram (2) in which a sound image is localized according to the position information of the listener.

FIG. 27 is a diagram showing an arithmetic processing configuration according to the present invention.

FIG. 28 is a diagram showing an example of how to obtain a common characteristic and a unique characteristic.

FIG. 29 is a diagram showing an embodiment (1) of the acoustic characteristic addition filter in which the common part and the unique part are separated.

FIG. 30 is a diagram showing an embodiment (2) of the acoustic characteristic addition filter in which the common part and the specific part are separated.

FIG. 31 is a diagram showing a measurement system for frequency characteristics of FIG. 32.

FIG. 32 is a frequency characteristic diagram of a common portion C → l.

FIG. 33 is a frequency characteristic diagram when the common portion C → l and the unique portion s1 → l are connected in series.

FIG. 34 is a diagram showing an example of a common characteristic storage.

FIG. 35 is a diagram showing an example of FIG. 34.

FIG. 36 is a diagram showing an example of left-right symmetrical processing.

FIG. 37 is a diagram showing an example of the position of a virtual sound source.

FIG. 38 is a diagram showing an example of left-right symmetrical acoustic characteristics in FIG. 37.

FIG. 39 is an explanatory diagram of an angle θ that represents a sound image position.

FIG. 40 is a diagram showing an example of left-right symmetrical acoustic characteristic addition filters.

[Explanation of symbols]

 40 ... Linear synthesis filter 43 ... Pitch synthesis filter 44 ... Short-term synthesis filter 35, 37 ... Acoustic characteristic addition filter 36, 38 ... Acoustic characteristic removal filter 39 ... Selection setting unit 60 ... Position information interpolation / prediction unit 61 ... Regularity judgment Part 64, 65 ... Common part 66-69 ... Unique part 74 ... Hard disk

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI technical display location H04S 7/00 G10K 15/00 B

Claims (20)

[Claims] 1. A stereophonic processing apparatus for localizing a sound image using a virtual sound source, wherein a desired acoustic characteristic to be added to an original signal is linearly predicted by linear predictive analysis of an impulse response representing the acoustic characteristic. A stereophonic processing apparatus formed by a linear synthesis filter having a coefficient as a filter coefficient, and adding desired acoustic characteristics to the original signal through the linear synthesis filter. 2. The stereoscopic synthesis filter according to claim 1, wherein the linear synthesis filter includes a short-term synthesis filter having an IIR filter configuration using the linear prediction coefficient and adding a frequency characteristic of a desired acoustic characteristic to the original signal. Sound processing device. 3. The linear synthesis filter according to claim 2, further comprising a pitch synthesis filter having an IIR filter configuration using the linear prediction coefficient and adding a time characteristic of a desired acoustic characteristic to the original signal. 3D sound processing device. 4. The pitch synthesis filter is composed of a pitch synthesis section for direct sound with a large attenuation rate, a pitch synthesis section for reflected sound with a small attenuation rate that follows, and a delay section for giving a delay time thereof. The stereophonic sound processing apparatus according to claim 3. 5. An inverse characteristic of an acoustic characteristic of an acoustic output device is formed by a linear prediction filter having a linear prediction coefficient obtained from a linear prediction analysis of an impulse response representing the acoustic characteristic as a filter coefficient. The stereophonic processing apparatus according to claim 1, wherein the acoustic characteristics of the acoustic device are removed through a linear prediction filter. 6. The stereophonic processing apparatus according to claim 5, wherein the linear prediction filter has an FIR filter configuration using the linear prediction coefficient. 7. A stereophonic processing apparatus for localizing a sound image by using a virtual sound source, which is a linear prediction of an impulse response representing each acoustic characteristic of one or a plurality of acoustic paths to the left ear added to an original signal. A first acoustic characteristic addition filter composed of a linear synthesis filter having a linear prediction coefficient obtained by analysis as its filter coefficient, which is connected in series to the first acoustic characteristic addition filter, and acoustic characteristics of an acoustic output device to the left ear. A first acoustic characteristic removing filter comprising a linear predictive filter having a linear predictive coefficient obtained by a linear predictive analysis of an impulse response representing the above as a filter coefficient for giving an inverse characteristic for removing an acoustic characteristic of the acoustic output device. , Obtained by linear predictive analysis of the in-pulse response representing each acoustic characteristic of one or more acoustic paths to the right ear added to the original signal A second acoustic characteristic addition filter composed of a linear synthesis filter having a linear prediction coefficient as a filter coefficient, and an in-pass representing the acoustic characteristic of the acoustic output device to the right ear, which is connected in series to the second acoustic characteristic addition filter. A second acoustic characteristic removal filter comprising a linear prediction filter obtained by a linear response linear prediction analysis and having a filter coefficient that gives an inverse characteristic for eliminating the acoustic characteristic of the acoustic output device, and A stereophonic processing apparatus comprising: a selection setting unit that selectively sets a predetermined parameter corresponding to the first acoustic characteristic addition filter and the second acoustic characteristic addition filter according to position information of a sound image. 8. The first and second acoustic characteristic addition filters include a common unit that adds a common characteristic to the acoustic characteristics of each acoustic path, and a unique unit that adds a unique characteristic to each acoustic characteristic of each acoustic path. The three-dimensional acoustic processing apparatus according to claim 7, wherein the three-dimensional acoustic processing device is configured by being divided into parts, and the overall acoustic characteristics are added by connecting the common part and the specific part in series. 9. A storage medium for storing a calculation result of the common unit for a predetermined sound source, and a read instruction unit for instructing reading of the stored operation result, wherein the read instruction unit is instructed by the instruction. 9. The stereophonic processing apparatus according to claim 8, wherein the read calculation result is directly given to the proper part. 10. The storage medium stores, in addition to the calculation result of the common unit for the predetermined sound source, the calculation result of the corresponding first or second acoustic characteristic removal filter. 3D sound processing device. 11. The stereophonic sound processing apparatus according to claim 7, wherein the first acoustic characteristic addition filter and the second acoustic characteristic addition filter further include a delay unit that gives a delay time between both ears. 12. The delay unit serving as a reference by using, as a reference (delay time zero), one of the delay times between both ears of the delay sections of the first or second acoustic characteristic addition filter. The stereophonic sound processing apparatus according to claim 11, wherein the can be omitted. 13. The first acoustic characteristic addition filter and the second acoustic characteristic addition filter further include an amplification section capable of variably setting the output signal levels thereof.
The three-dimensional sound processing device according to the above. 14. The selection / setting unit relatively sets output signal levels of the first acoustic characteristic addition filter and the second acoustic characteristic addition filter by gain setting of the amplification unit according to position information of the sound image. The stereophonic processing apparatus according to claim 13, wherein the localization position of the sound image is moved by changing the position of the sound image. 15. The first and second acoustic characteristic addition filters are configured symmetrically with respect to the front of a listener, and parameters of the delay unit and the amplification unit are shared between corresponding positions on the left and right. The stereophonic sound processing apparatus according to claim 11 or 13. 16. A position information interpolating unit for interpolating intermediate position information from position information of past and future sound images, and the interpolated position information from the position information interpolating unit is the position to the selection setting unit. The stereophonic sound processing apparatus according to claim 7, which is given as information. 17. A position information predicting unit for predictively interpolating the position information of the future from the position information of the past and present sound images, and the future position information from the position information predicting unit is sent to the selection setting unit. The stereophonic sound processing apparatus according to claim 7, which is given as position information. 18. The position information predicting unit further includes a regularity determining unit that determines whether or not there is regularity regarding a moving direction of the past and present sound image position information, and the positional information predicting unit determines the regularity determining unit. 18. The stereophonic sound processing apparatus according to claim 17, wherein the future position information is given when the unit determines that there is regularity. 19. The stereophonic sound processing according to claim 16, wherein instead of the position information of the sound image, image position information from an image display device that displays an image emitting the sound image is used. apparatus. 20. The stereophonic processing apparatus according to claim 7, wherein the selection setting unit moves the listening environment in accordance with the given position information of the listener so as to further provide / maintain a favorable listening environment of the listener. .