JP4951985B2 - Audio signal processing apparatus, audio signal processing system, program - Google Patents

Description

  The present invention relates to an audio signal processing apparatus that performs signal processing on an audio signal. The present invention also relates to a program executed by an information processing apparatus that intends to provide the function of such an audio signal processing apparatus.

For example, it is known and widely used that sound reproduction is performed by a so-called multi-channel channel structure such as 5.1ch surround or 7.1ch surround.
On the other hand, as represented by, for example, two-channel stereo using Lch and Rch, a reproduction system that has been widely established before the multi-channel system is still widely used. For this reason, it is unavoidable that a multi-channel audio source must be played back by a playback system having a smaller number of channels, such as the above-described 2-channel stereo.

However, a multi-channel audio source is originally composed of an audio signal corresponding to each individual channel formed so as to obtain an appropriate acoustic effect when viewing the multi-channel as a whole. . In the case of 5.1 ch surround, 6 according to each of L (left) ch, C (center) ch, R (right) ch, LS (left surround) ch, RS (right surround) ch, SW (subwoofer) ch. It consists of two audio signals. For this reason, when a multi-channel source is played back by L and R stereo channels, for example, if the multi-channel Lch and Rch audio are simply played back and output, the sound source to be played back by the remaining Cch, LSch, and RSch. This causes the inconvenience that this element is completely lost and a sound that cannot be heard is produced.
Therefore, an encoding technique is known in which audio signals of respective channels forming a multi-channel are appropriately distributed and converted into an audio source having a 2-channel stereo channel configuration using, for example, Lch and Rch. For example, when a two-channel stereo sound source encoded in this way is reproduced, the reproduced sound field is based on a general two-channel stereo in which the sound image is localized only in the left-right direction, but all channels are reproduced. Since the sound component is included, there is no sound that is lost and cannot be heard.

The following encoding techniques are known as described above. In the description of the encoding technique here, the multi-channel to be encoded is assumed to be four channels of Lch, Cch, Rch, and S (surround) ch, and converted into Lch and Rch two-channel stereo by encoding. Take the case where it is done as an example.
Here, the audio signals for each of the Lch, Cch, Rch, and Sch channels forming the above-described multichannel are Sl, Sc, Sr, and Ss, respectively, and the Lch and Rch signals of the two channels after encoding are S1, S2, respectively. And As an encoding process, for example, the signals S1, Sc, Sr, and Ss are used to execute the operations shown in the following equations (1) and (2) to obtain these signals S1 and S2. Is done.
S1 = L + 0.7C + 0.7S ... Formula (1)
S2 = R + 0.7C-0.7S (2)
In this way, the signal S1 is obtained by adding the Cch and Sch signals multiplied by the predetermined coefficient (0.7) to the Lch signal. The signal S2 is obtained by adding Cch multiplied by a predetermined coefficient (0.7) to the Rch signal and subtracting the Sch signal. Then, if the audio source based on the signals S1 and S2 obtained in this way is reproduced by a 2-channel stereo playback system, the sound image localization of the 2-channel stereo based on the normal Lch and Rch is obtained. The sound will be reproduced without being lost.

  As a technique corresponding to the above-described encoding technique, there is a decoding technique for converting an encoded audio source such as a two-channel stereo into an original multi-channel audio source. Such a decoding technique will be described with reference to FIG.

In FIG. 22, signals S1 and S2 obtained by the above-described encoding process are input as decoding source signals.
The signals S1 and S2 are directly input to the direction enhancement circuits 501 and 504, respectively. At the same time, the signals S 1 and S 2 are added by the adder 511 and input to the directionality enhancement circuit 502 as the signal S 3. Further, the signals S1 and S2 are subtracted by the adder 512 and input to the directionality emphasis circuit 503 as the signal S4. That is, the part which receives the signals S1 and S2 and generates the signals S3 and S4 adopts a configuration as a matrix circuit.
Based on the operation of this matrix circuit, the signals S3 and S4 are expressed by the following equations (3) and (4), respectively.
S3 = 1.4C + L + R (Formula 3)
S4 = 1.4S + LR (Formula 4)
Note that the signals S1 and S2 shown in FIG. 22 are represented by the equations (1) and (2) shown above, respectively.

As characteristics of the signals S1, S2, S3, and S4 obtained as described above, first, the signal S1
The decoded Lch signal component is 3 dB higher than the signal components of the other channels. In the signal S2, the decoded Rch signal component is 3 dB higher than the signal components of other channels. The signal S3 has a decoded Cch signal component 3 dB higher than the signal components of other channels, and the signal S4 has a decoded Cch signal component 3 dB higher than the signal components of other channels. In other words, the signals S1, S2, S3, and S4 have the property that only the signal component of one specific channel is higher than the signal components of other channels among the signal components of each channel included in itself. , Respectively, have obtained properness as signals of the Lch, Cch, Rch, and Sch channels.
However, in the state of the signals S1, S2, S3, and S4 at the stage where they are still generated by the matrix circuit, the sound image is not sufficiently separated. Therefore, direction enhancement circuits 501, 502, 503, and 504 are provided, and signals S 1, S 2, S 3, and S 4 are passed through these circuits, respectively, and reproduction is performed for each actual Lch, Cch, Rch, and Sch channel. The signal for use is obtained. The direction enhancement circuit is configured to change its output level in accordance with the level difference between the signals S1, S2, S3, and S4. For example, if the level of the Lch signal S1 becomes higher than that of the signals S2, S3, and S4 of other channels, the level of the signal S1 is dynamically increased in response to this, and the Lch audio is transmitted to the other channels. Make it stand out from your voice. By such an operation, the separation of the sound image between the sound of the four channels becomes better.
Note that the encoding and decoding techniques described above are employed in, for example, Dolby Pro Logic.

JP 2003-274493 A

However, it can be said that the above-described encoding and decoding techniques are not perfect in the following points, and there is still room for improvement.
For example, in the decoding process, as described with reference to FIG. 22, direction enhancement processing is performed on the audio signals (S 1, S 2, S 3, S 4) for each multichannel restored by the matrix circuit. However, this process is to enhance the channel sound at a higher level than the other channel sounds. This has the effect of making the sound image separation between channels more clear, but the level of the output sound for each channel fluctuates dynamically, and it is easy to feel unnaturally changing volume. Have a problem. Further, when the audio signals of all the channels are at substantially the same level, the processing for increasing the level difference is not performed. For example, the sound volume separation between the channels can secure about 3 dB. The sound image separation is not good. In addition, depending on the content of the audio, the sound output from one speaker may be attracted to the other speaker side between speakers that are adjacent to each other, so that the localization may be inadvertently changed. . That is, the technology corresponding to FIG. 22 leaves room for higher quality regarding the sound when the encoded audio signal is decoded and reproduced by multi-channel.

In view of the above-described problems, the present invention is configured as an audio signal processing apparatus as follows.
In other words, the audio signal processing device of the present invention uses the spatial transfer function obtained based on the position of the sound source as the corresponding decode channel for each of the audio signal components corresponding to the decode channel having a predetermined channel configuration. The audio signal corresponding to one specific channel in the decode channel is input by inputting the audio signal of the encode channel, which gives the transfer characteristics expressed, and is generated by distributing these audio signal components according to the channel configuration of the encode channel. Audio signal generation means for generating signal components is provided for each of the decode channels.
Each of the audio signal generation means corrects the transfer characteristic given to the audio signal component of the decoding channel corresponding to the audio signal generation means corresponding to each of the audio signals of the input encode channels. And proximity detection means for detecting a predetermined approximation between signals corrected by the correction means, and for each encode channel output from the signal correction means based on the detection result of the proximity detection means. Separation means that separates and outputs signal components that are said to be close to each other from the signal, and channel sound that is output as the audio signal of the corresponding decode channel by adding the signal components separated by this separation means And signal output means.

The audio signal processing system is configured as follows.
In other words, the audio signal processing system of the present invention converts a set of audio signals of the original channel that forms a predetermined channel configuration into a set of audio signals of the encode channel that forms a predetermined channel configuration other than the original channel, and outputs it. The encoding device includes a decoding device that inputs a set of audio signals of an encoding channel having a predetermined channel configuration and converts the set into a set of audio signals of a decoding channel having a predetermined channel configuration.
In the encoding apparatus, one original channel corresponding to each encoding channel is provided, and the input audio signal is represented by a spatial transfer function set based on the position of the sound source as the corresponding original channel. A transfer characteristic applying unit that applies a transfer characteristic to an input audio signal, and a signal that is provided for each encode channel and that is processed by each of the transfer characteristic applying units is input and added, and this addition is performed. And adding means for outputting the output as an audio signal of the corresponding encoding channel.
In addition, the decoding apparatus includes an audio signal separation unit that separates an audio signal component corresponding to one specific channel in the decode channel corresponding to each decode channel. Correction means for correcting the transfer characteristic given to the audio signal component of the corresponding decode channel for the input audio signal for each encode channel, and a predetermined signal for the signal for each encode channel after correction by the correction means Based on the detection result of the proximity detection means and the signal for each encode channel output from the signal correction means, signal components that are approximated to each other are detected. separating means for separating and outputting the signal component separated by the separating means San, it was decided and a channel sound signal output means for outputting as a voice signal of the corresponding decoding channels.
The channel configuration here refers to the configuration content determined by the number of audio channels for forming one acoustic system and the positional relationship between sound sources corresponding to the audio channels.

According to each of the above configurations, an encoded audio source is obtained by converting a predetermined channel configuration into another channel configuration. At that time, the audio signal for each channel after encoding is given a transfer characteristic corresponding to the appropriate spatial transfer function according to the position of the sound source of each channel in the channel configuration before encoding. The sound source encoded in this way can be reproduced by a reproduction system according to the channel configuration of the encode channel, thereby realizing a sound image localization equivalent to the case where the sound source before encoding is reproduced.
Then, the audio signal processing device (decoding device) of the present invention inputs the audio signal having the above-described encoding channel configuration and converts it into an audio source consisting of an audio signal group having the same channel configuration as before encoding or another channel configuration. To do. For this purpose, the audio signal component as each decode channel is separated from the input audio signal of each channel after encoding and output.
The configuration for separating the audio signal component corresponding to one decode channel is that the transfer characteristic for encoding is given from the audio signal components included in each audio signal for each encode channel. A predetermined element (for example, phase, level, propagation time difference, etc.) of the audio signal component of the decoded channel that has changed is corrected. Then, signal components that are approximated by these elements are separated from the audio signals of the encoding channel. The signal component separated in this way is output as an audio signal of the corresponding decode channel. Such signal separation processing is executed for each decode channel. In this case, the output as the audio signal of each decode channel is composed of only the audio signal component to be reproduced and output by the decode channel, and may be regarded as not including the audio signal component of other channels.

  Therefore, the audio signal processing apparatus according to the present invention does not perform processing such as directionality enhancement when reproducing the decoded audio source by the reproduction system according to the configuration of the decode channel, and performs proper sound image localization. This can be reproduced, and this leads to, for example, an improvement in the quality of reproduced sound.

Hereinafter, the best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described.
FIG. 1 and FIG. 2 show the input / output channel configuration for each of the encoding apparatus and decoding apparatus of the present embodiment.
First, as shown in FIG. 1, the encoding apparatus 1 according to the present embodiment has L (left) ch, C (center) ch, R (right) ch, which is one of channel configurations called multi-channels. Input an audio source with a set of 5 channels of LS (left surround) ch and RSch (right surround) audio signals, and convert them to an audio source of an audio signal set with a channel configuration corresponding to 2-channel stereo of Lch and Rch. And output. The channel configuration of Lch, Cch, Rch, LSch, and RSch described above can be considered as a channel configuration of 5.1ch surround in which the subwoofer channel is omitted.
Further, as shown in FIG. 2, the decoding apparatus 2 of the present embodiment inputs a set of audio signals of an audio source having a channel configuration corresponding to 2-channel stereo. The audio signal input in this way is that of the audio source encoded by the encoding device 1. Then, as a result of performing decoding processing on these input audio signals, audio signals having a set of five channels similar to those before being encoded by the encoding apparatus 1 are output.

FIG. 3 shows a model for the channel configuration of the audio source to be encoded by the encoding apparatus 1 shown in FIG.
In this figure, speakers SP-L, SP-C, SP-R, SP-LS, and SP-RS are shown as sound sources corresponding to each of Lch, Cch, Rch, LSch, and RSch. A model is shown in which a listener (listener) M listens to sounds that are output and reach the left and right ears respectively.
Incidentally, in such a channel configuration, as shown in the figure, the speaker SP-L is arranged at the front left with respect to the position of the listener M, the speaker SP-C is arranged at the center front, and the speaker SP-R at the right front. Are usually arranged, the speaker SP-LS is arranged on the left rear side, and the speaker SP-RS is arranged on the right rear side. As for the arrangement position of such multi-channel speakers, a predetermined arrangement angle, height, and the like that are ideal by ITU-R and the like are recommended.

The path of the sound that reaches the right and left ears of the listener M from each speaker under the channel configuration shown in FIG. 3 is represented by the following transfer function (spatial transfer function).
Hll: Transfer function of the path from the speaker SP-L to the left ear Hlr: Transfer function of the path from the speaker SP-L to the right ear Hcl: Transfer function of the path from the speaker SP-C to the left ear Hcr: Transfer function of path from speaker SP-C to right ear Hrl: Transfer function of path from speaker SP-R to left ear Hrr: Transfer function of path from speaker SP-R to right ear Hlsl: Speaker SP -Transfer function of path from LS to left ear Hlsr: Transfer function of path from speaker SP-LS to right ear Hrsl: Transfer function of path from speaker SP-RS to left ear Hrsr: Speaker SP-RS Transfer function from the path to the right ear

When the target position of the sound emitted from the speaker (sound source) is the listener's left ear and right ear, the space for the path from which the sound reaches the left ear and right ear from the sound source. In particular, the transfer function is handled as a head-related transfer function.

FIG. 4 shows an example of the internal configuration of the encoding apparatus 1 shown in FIG.
As an input of the encoding apparatus 1, as in FIG. 1, an audio signal for each channel (original channel) forming the channel structure of Lch, Cch, Rch, LSch, and RSch is input.
First, regarding the input audio signal corresponding to Lch, the input audio signal as the Lch (original channel (L)) of the original channel is branched and input to the filters 11a and 11b. The filter 11a executes a process for giving a transfer characteristic represented by the transfer function Hll to the input audio signal of the original channel (L). For this purpose, for example, an impulse response obtained by converting the transfer function Hll on the time axis is obtained, and a filtering process for convolving the impulse response with the input audio signal of the original channel (Lch) may be executed. Further, the filter 11b executes a process for giving a transfer characteristic represented by the transfer function Hlr to the input audio signal of the original channel (L) by the same filtering process as described above.

Similarly, the input audio signals of the original channels (C), (R), (LS), and (RS) remaining are subjected to processing for giving transfer characteristics corresponding to the corresponding transfer functions. To be done.
That is, for the input audio signal of the original channel (C), the filter 12a gives the transfer characteristic represented by the transfer function Hcl, and the filter 12b gives the transfer characteristic represented by the transfer function Hcr.
For the input audio signal of the original channel (R), the filter 13a gives the transfer characteristic represented by the transfer function Hrl, and the filter 13b gives the transfer characteristic represented by the transfer function Hrr.
The input audio signal of the original channel (LS) is given a transfer characteristic represented by the transfer function Hlsl by the filter 14a and given a transfer characteristic represented by the transfer function Hlsr by the filter 14b.
For the input audio signal of the original channel (RS), the transfer characteristic represented by the transfer function Hrsl is given by the filter 15a, and the transfer characteristic given by the transfer function Hrsr is given by the filter 15b.

Here, each of the filters 11a, 11b to 15a, 15b can be constituted by a predetermined order FIR (Finite Impulse Response) type digital filter shown in FIG. As the FIR filter, for example, a number of delay devices 21 (1) to 21 (N), multipliers 22 (1) to 22 (n), and an adder 23 (1) corresponding to the order to be configured (Nth order). To 23 (M) are connected as shown in the figure. The delay units 21 (1) to 21 (N) each delay the timing signal of one sample, and the multipliers 22 (1) to 22 (n) are set with coefficients corresponding to the impulse responses to be convoluted. The With such a configuration, the digital audio signal input from the input terminal 20 is output from the output terminal 24 with the impulse response convoluted. That is, it is converted into an audio signal having a transfer characteristic corresponding to the impulse response and output.
Further, the impulse response convoluted by these filters 11a, 11b to 15a, 15b, or the transfer function based on the impulse response may be obtained by actually measuring after creating a predetermined environment, It can be obtained by calculation after assuming the environment. In addition, as the position of the sound source (speaker) of the original channel that is actually or virtually set at this time, the position according to the recommendation of the ITU-R described above can be adopted. Further, positions other than those recommended by the ITU-R may be set.

Returning to FIG.
Of the signals output with appropriate transfer characteristics given by the filters 11a, 11b to 15a, 15b, the signals output from the filters 11a, 12a, 13a, 14a, 15a are added by the adder 16a and encoded. Is output as an L channel signal in stereo channels (2 channels).
The signals output from the filters 11b, 12b, 13b, 14b, and 15b are added by the adder 16b and output as an R channel signal in the stereo channel after encoding.

Here, the L channel signal after the encoding is a path of the path reaching the left ear of the listener M in FIG. 3 with respect to each audio signal of the original channel (L) (C) (R) (LS) (RS). It is the result of adding (synthesizing) those given transfer characteristics. Further, the encoded R channel signal has a transfer characteristic of a path reaching the right ear of the listener M with respect to each audio signal of the same original channel (L) (C) (R) (LS) (RS). It is the result of adding (combining) the given ones.
For example, it is assumed that the 2-channel audio source encoded in this way is reproduced and output by an audio reproducing apparatus compatible with normal 2-channel stereo, and the reproduced sound is heard through headphones.
At this time, the sounds heard by the left and right ears of the listener wearing the headphones are the left and right ears of the listener M from the speakers SP-L, SP-C, SP-R, SP-LS, and SP-RS in FIG. And has the transfer characteristics of the route to reach each. Therefore, the sound actually perceived by the listener wearing the headphones is not localized in the head like a normal two-channel stereo. For example, as shown in FIG. The localization when the sound of the original channel is emitted at the position where the SP-L, SP-C, SP-R, SP-LS, and SP-RS are virtually present is perceived. .
Here, in order to make the correspondence with FIG. 3 easy to understand, the case where the audio source encoded by the encoding apparatus 1 of the present embodiment is reproduced by headphones is described. When sound is reproduced and output from two speakers corresponding to the channels R and R, for example, the localization of the virtual sound source similar to that shown in FIG. 3 can be performed. In this case, in addition to the transfer function corresponding to each speaker of the original channel shown in FIG. 3, the transfer function of the path of the sound reaching the both ears of the listener from the two speakers corresponding to the L and R channels. In consideration of the above, the transfer function of the impulse response to be convoluted in the filters 11a, 11b to 15a, 15b in FIG.
For example, when an audio source encoded in a two-channel stereo channel configuration by the encoding technique described as a conventional example is reproduced by a playback device corresponding to a normal two-channel stereo, sound image localization as a normal two-channel stereo is obtained. On the other hand, with the encoding apparatus 1 of the present embodiment, virtual sound image localization using the original channel before encoding can be obtained as described above. As a result, for example, content information including an encoded audio source increases its added value.

Next, an internal configuration example of the decoding device 2 according to the present embodiment will be described with reference to FIG.
As shown in this figure, for example, Lch and Rch audio signals in 2-channel stereo after being encoded by the encoding device 1 are input to the decoding device 2. Here, the L and R channels corresponding to the channel structure after encoding input to the decoding device 2 are also referred to as an encode channel (L) and an encode channel (R).

The encoder 2 shown in this figure includes fast Fourier transform units (FFT units) 31a and 31b, channel signal separation blocks 32-L, 32-C, 32-R, 32-C, 32-LS, 32-RS, and inverse high speed. It comprises a Fourier transform unit (IFFT unit) 33-L, 33-C, 33-R, 33-LS, 33-RS.
Of the input audio signal of the encode channel (L) and the input signal of the encode channel (R), the input audio signal of the encode channel (L) is input to the fast Fourier transform unit 31a. The Fourier transform unit 31a converts the input audio signal into a frequency domain signal Sgl by executing a fast Fourier transform process. The signal Sgl branches and is input to the correction processing unit 41a provided in the channel signal separation blocks 32-L, 32-C, 32-R, 32-LS, and 32-RS.
The input audio signal of one encode channel (R) is input to the fast Fourier transform unit 31b. Also in the Fourier transform unit 31b, the input audio signal is subjected to fast Fourier transform processing to be converted into a frequency domain signal Sgr, and channel signal separation blocks 32-L, 32-C, 32-R, 32-LS, 32 -Input each to the correction processing unit 41b provided in the RS.

  The channel signal separation blocks 32-L, 32-C, 32-R, 32-LS, and 32-RS are Lch, Cch, Rch, and the channel configuration after decoding, as will be understood from the following description. Five channels are provided corresponding to five channels (decode channels) of LSch and RSch, and each includes correction processing units 41a and 41b and a separation processing unit 42 as illustrated. .

7 and 8 show configuration examples showing the details of the inside of the separation processing unit 42 as the configuration of the channel signal separation block 32. 7 and 8, the channel signal separation block 32-L among the five channel signal separation blocks is taken as an example.
FIG. 7 shows the internal configuration based on the concept of the signal processing operation executed in the channel signal separation block 32-L.
The signal Sgl obtained by converting the input audio signal of the encode channel (L) into the frequency domain by the fast Fourier transform unit 31a is input to the correction processing unit 41a in the channel signal separation block 32-L. In the correction processing unit 41a, a filtering process using a filter characteristic that is opposite to the filter characteristic of the impulse response convolution process according to the transfer function Hll is executed.
The inverse characteristic of the filter characteristic of the impulse response convolution process according to this transfer function Hll is represented by the inverse of the transfer function Hll here as [1 / Hll].
For example, when the frequency response characteristic of an audio signal component that has a transfer characteristic corresponding to the transfer function Hll is as shown in FIG. 9A, the frequency response characteristic of the inverse characteristic [1 / Hll] is As shown in FIG. 9B, the characteristics of FIG. 9A are reversed.
The transfer function Hll is a transfer function of a path from the speaker SP-L of the original channel (L) shown in FIG. 3 to the left ear of the listener M, and the filter 11a in the encoding apparatus 1 shown in FIG. This corresponds to the filter characteristic set to. That is, in the correction filter 41a of FIG. 7, the inverse filter shown as [1 / Hll] is applied, and thereby the signal components of the filters 11a to 15a that are included in the signal Sgl. Among them, the transfer characteristic by the transfer function Hll given to the output signal component of the filter 11a is cancelled. For this reason, the output signal component of the filter 11a included in the signal Sgl becomes as close as possible to the audio signal of the previous original channel (L) input to the filter 11a, that is, the signal of the audio source before encoding. It will be corrected to a characteristic that can be regarded as equivalent. It should be noted that the signal components corrected by the correction filter 41a so as to have characteristics equivalent to those of the audio signal of the original channel are only those corresponding to the original channel (L), and other original channels. The signal component corresponding to is left uncorrected.

Further, the signal Sgr obtained by converting the input audio signal of one encoding channel (r) into the frequency domain by the fast Fourier transform unit 31a is also transferred by the correction processing unit 41a in the channel signal separation block 32-L. A filtering process is performed with an inverse characteristic [1 / Hlr] with respect to the filter characteristic of the impulse response convolution process according to Hlr (the filter characteristic of the filter 11b in FIG. 4). As a result, the output of the correction processing unit 41a is corrected so that only the audio signal component of the original channel (L) among the signal components included in the signal Sgr has the same characteristics as those before the input of the filter 11b. Will be.
The correction processing units 41a and 41b can also be configured by forming, for example, an FIR type filter as shown in FIG. 5 and setting, for example, a coefficient corresponding to the inverse filter characteristic in the multiplier.

  As described above, depending on the correction processing unit 41a of the channel signal separation block 32-L, only the signal component of the original channel (L) included in the signal Sgl corresponding to the encode channel (L) is encoded. The correction processing unit 41b has a signal Sgr corresponding to the encoding channel (R), and only the signal component of the original channel (L) included in the signal Sgr is equal to that before encoding. Correct for characteristics. That is, for the signal Sgl and the signal Sgr, signals Sgla and Sgra obtained by correcting only the signal component of the original channel (L) in the same manner as before encoding are obtained. As for the relationship between these signals Sgla and Sgra, the phase and level of the signal components of the original channel (L) corrected in common coincide with each other. In other words, the correction processing by the correction processing units 41a and 41b is performed by applying the transfer characteristics (Hll and Hlr) to the signal component of the original channel (L) during encoding, and the original channel between the signals Sgl and Sgr. It can also be considered that the phase difference and level difference of the signal component (L) are corrected. In addition, in the signals Sgla and Sgra, signal components other than the original channel (L) do not coincide with each other because the transfer characteristics given by the filters 12a, 12b to 15a and 15b remain at the time of encoding. It is a state.

The signals Sgla and Sgra having such properties are input to the level / phase comparison processing block 51 in the separation processing unit 12. Further, it is input to multipliers 53 and 54, which will be described later.
The level / phase comparison processing block 51 performs level comparison and phase comparison on the input signals Sgla and Sgra, and, as a comparison result, an approximation rate of the level and phase in the frequency domain for the signals Sgla and Sgra. A signal indicating a value is output to the sound source separation function calculation block 52.

  The sound source separation function calculation block 52 performs a calculation on a predetermined sound source separation function based on the approximate value as the detection signal input from the level / phase comparison processing block 51, thereby calculating the coefficients of the multipliers 53 and 54. The obtained coefficient is set for the multipliers 53 and 54. The multipliers 53 and 54 multiply the input signals Sgla and Sgra by set coefficients, respectively, and output the result. A more specific example of how to obtain this coefficient will be described later. By setting the coefficients in this way, the multiplier 53 outputs a component of the signal Sgla whose level and phase approximate to a certain level or more from the other signal Sgra. Similarly, the multiplier 54 outputs a component in the signal Sgra whose level and phase approximate to a certain level or more from the signal Sgla. As a result, the outputs of the multipliers 53 and 54 are signal components that can be regarded as the same level and the same phase in the signal component included in the signal Sgla and the signal component included in the signal Sgra. Become. As described above, the signal having the same level between the signal Sgla and the signal Sgra is the signal component of the original channel (L) corrected by the correction filters 41a and 41b. Therefore, the multiplier 53, It can be said that the output 54 is obtained by separating and extracting the corrected signal component of the original channel (L) from each of the signals Sgla and Sgra. The outputs of the multipliers 53 and 54 are added by an adder 55 and output. The output of the adder 55 becomes an output signal of the channel signal separation processing block 32-L, and this output signal is a signal component equivalent to the audio signal of the original channel (L) before encoding. become. That is, in the channel signal separation processing block 32-L, the audio signal of the encoding channel (L) (R) converted into the frequency domain is input, and the signal having the same component as the audio signal of the original channel (L) before encoding. Are extracted and output.

FIG. 8 shows a channel signal separation block 32-L that is actually configured based on the processing concept described with reference to FIG. In this figure, the same parts as those in FIG.
In FIG. 8, a more practical internal configuration example of the separation processing unit 42 is shown, and this point will be described. The separation processing unit 42 shown in FIG. 8 includes a level comparison unit 61, a coefficient generation unit 62, a phase comparison unit 63, a coefficient generation unit 64, multipliers 65, 66, 67, 68, and an adder 55. Is done.
When the signals Sgla and Sgra are input to the separation processing unit 42, first, the signals Sgla and Sgra are input to the level comparison unit 61. The level comparison unit 61 obtains the level of the input signals Sgla and Sgra for each frequency sample, for example, and based on the obtained both levels, for example, the level ratio of the signal Sgra to the signal Sgla (or the signal Sgla to the signal Sgla) m is calculated and output to the coefficient generator 62. Incidentally, the level ratio m takes a range of 0 ≦ m ≦ 1, and if m = 1, it indicates that the mutual levels are completely the same. Further, as the value of the level ratio m is smaller, the mutual level difference becomes larger and the approximation becomes lower.
The coefficient generator 62 obtains a coefficient r to be set for the multipliers 65 and 66 based on the input level ratio m. The range of the coefficient r is 0 ≦ r ≦ 1. And in order to determine this coefficient r, the calculation using a predetermined sound source separation function is performed. As the sound source separation function, a predetermined relationship is set such that the coefficient r approaches 1 as the level ratio m approaches 1. Examples of sound source separation functions used by the coefficient generator 62 are shown in FIGS. 10 (a), 10 (b), and 10 (c).

  FIGS. 10A, 10B, and 10C show the sound source separation function by the relationship between the level ratio m and the coefficient r, where the horizontal axis is the level ratio m and the vertical axis is the coefficient r. Yes. The sound source separation functions shown in these figures are common in that, for example, the coefficient r = 1 is set when the level ratio m = 1, but how to set the coefficient r when the level ratio m is smaller than 1. Is different. Further, functions other than those shown in FIGS. 10A, 10B, and 10C are also conceivable. In this case, the value of the coefficient r less than 1 can be set even when the level ratio m = 1. Sex is also included.

Returning to FIG.
For example, the coefficient r obtained by the coefficient generator 62 as described above is set for each of the multipliers 65 and 66. The multipliers 65 and 66 multiply the input signals Sgla and Sgra by a set coefficient r and output the result. The signals output from the multipliers 65 and 66 in this way are understood based on the description of the level / phase comparison processing block 51 and the sound source separation function calculation block 52 in the separation processing unit 42 in FIG. As described above, the spectral components whose levels are approximated by a certain level or more from the signal Sgla are separately extracted, and the levels of the signal Sgla and the signal Sgla are approximated by a certain level or more. The spectral components are separated and extracted. This means that the outputs of the multipliers 65 and 66 are signal components that are the same as the audio signal of the original channel (L) corrected by the correction filters 41a and 41b in terms of level. Become.

However, the outputs of the multipliers 65 and 66 are signals that are separated and extracted from the signals Sgla and Sgra based only on the level comparison result. Therefore, for example, there is a possibility that the audio signal components of the original channels other than the original channel (L), which happen to have the same level as the audio signal of the original channel (L) in a certain time series, are included accordingly.
Therefore, the outputs of the multipliers 65 and 66 are further input to the phase comparison unit 63, where phase comparison is performed. As a result of the comparison, the phase difference p of the output signal of the multiplier 66 (or the multiplier 65 for the output signal of the multiplier 66) with respect to the output signal of the multiplier 65 is obtained and output to the coefficient generator 64. The The phase difference p takes a range of, for example, 0 ≦ p ≦ π. If p = 0, it indicates that the phase difference is completely the same. In addition, as the value of the phase difference p increases and the phase difference increases, the signal approximation with respect to the phase decreases.
The coefficient generator 64 obtains a coefficient rp to be set for the multipliers 67 and 68 based on the input value of the phase difference p. The range of the coefficient rp is 0 ≦ rp ≦ 1. And in order to determine this coefficient rp, the calculation using a predetermined sound source separation function is performed. As the sound source separation function, the coefficient rp has a relationship of approaching 1 as the phase difference p approaches 0. Examples of sound source separation functions used by the coefficient generator 62 are shown in FIGS. 11 (a), 11 (b), and 11 (c).

Figure 11 (a) (b) ( c) , the sound source separation function according to the phase difference, in which are shown by the relation between the phase difference p and the coefficient rp, the horizontal axis in the phase difference p, the vertical axis The coefficient is rp . The sound source separation functions shown in these figures are also common in that, for example, the coefficient rp = 1 is set when the phase difference p = 0, but how to set the coefficient rp when the phase difference p is smaller than 1. Is different. Also in this case, sound source separation functions other than those shown in FIGS. 11A, 11B, and 11C are conceivable. For example, such a function includes a coefficient less than 1 even when the phase difference p = 0. The possibility to set the value of rp is included.

For example, the coefficient rp obtained by the coefficient generator 64 as described above is set for each of the multipliers 67 and 68, as shown in FIG. Multipliers 67 and 68 receive the output signals of multipliers 65 and 66, respectively, multiply the set coefficient rp, and output the result.
The signals output from the multipliers 67 and 68 in this way are spectrums whose phase difference is within a certain range (there is a phase approximation of a certain level or more) from the output signals of the multipliers 65 and 66. The components are separated and extracted. Therefore, the signals output from the multipliers 67 and 68 have the same phase from the signal components that are the same as the audio signal of the original channel (L) corrected by the correction filters 41a and 41b with respect to the level. This means that the signal components are separated. That is, it corresponds to the signal output from the multipliers 53 and 54 in FIG. 7, and is the same as the audio signal of the original channel (L) corrected by the correction filters 41a and 41b in both level and phase. Therefore, the signal is equivalent to the audio signal of the original channel (L) before encoding.
Then, the outputs of the multipliers 67 and 68 thus obtained are added by the adder 55 in the same manner as in FIG. 7, and the added signal is used as the output of the channel signal separation processing block 32-L. .

Comparing FIG. 7 and FIG. 8, in the configuration of FIG. 8, the functions as the level / phase comparison processing block 51 and the sound source separation function calculation block 52 shown in FIG. The signal component of the same phase is separated and extracted by performing only phase comparison with a part (level corresponding separation processing system: level comparison unit 61, coefficient generation unit 62, multipliers 65 and 66) that separates and extracts signal components of the same level. It can be seen that the configuration (phase correspondence separation processing system: phase comparison unit 63, coefficient generation unit 64, multipliers 67, 68) to be performed is provided so as to be divided into the former stage and the latter stage.
As another configuration of the separation processing unit 42 in FIG. 8, a phase-corresponding separation processing system (phase comparison unit 63, coefficient generation unit 64, multipliers 67 and 68) is placed in the previous stage, and a level-corresponding separation processing system is placed in the subsequent stage. It is also possible to employ a configuration in which (level comparison unit 61, coefficient generation unit 62, multipliers 65 and 66) are provided.
In addition, as the separation processing unit 42, for example, when the quality of reproduced sound required for the decoding device is not so high quality, either one of the level correspondence separation processing system and the phase correspondence separation processing system is used. It is also conceivable to have a configuration including only the above. Even if only one of the level-corresponding separation processing system and the phase-corresponding separation processing system is performed, the signal component that is the same as that before encoding of the original channel (L) is determined based on either the level or the phase. Since it can be extracted, for example, compared with the encoded output by the conventional matrix circuit and the directionality enhancement circuit, the quality of the decoded output sound can be kept reasonably good.

Returning to FIG.
For example, with the configuration shown in FIGS. 7 and 8, the channel signal separation block 32-L separates and outputs a signal having the same frequency component as that of the original channel (L) before encoding.
The remaining four channel signal separation blocks 32-C, 32-R, 32-LS, and 32-RS also adopt the configuration shown in FIG. 7 or FIG. 8 in terms of block configuration. In addition, the correction processing units 41a and 41b of the channel signal separation block 32-C are respectively subjected to inverse filters based on the inverse characteristics [1 / Hcl] [1 / Hcr] of the transfer functions Hcl and Hcr. As a result, the channel signal separation block 32-C separates and outputs a signal having the same frequency component as that of the original channel (C) before encoding.
Further, the inverse filter characteristics of the correction processing units 41a and 41b of the channel signal separation block 32-R set the inverse characteristics [1 / Hrl] and [1 / Hrr] of the transfer functions Hrl and Hrr, respectively. As a result, the output of the channel signal separation block 32-R becomes a signal having a frequency component that is the same as that of the original channel (R) before encoding.
Further, the inverse filter characteristics of the correction processing units 41a and 41b of the channel signal separation block 32-LS set inverse characteristics [1 / Hlsl] [1 / Hlsr] of the transfer functions Hlsl and Hlsr, respectively. Thereby, the output of the channel signal separation block 32-LS becomes a signal having a frequency component which is the same as that of the original channel (LS) before encoding.
The inverse filter characteristics of the correction processing units 41a and 41b of the channel signal separation block 32-RS set inverse characteristics [1 / Hrsl] [1 / Hrsr] of the transfer functions Hrsl and Hrsr, respectively. Thereby, the output of the channel signal separation block 32-LS becomes a signal having a frequency component which is the same as that of the original channel (RS) before encoding.

  The signals output from the channel signal separation blocks 32-L, 32-C, 32-R, 32-LS, and 32-RS are respectively converted into IFFT units 33-L, 33-C, and 33-R. By 33-LS and 33-RS, a frequency domain signal is converted into a time domain audio signal and output. The audio signals output in this way are audio signals that are the same as the original channels (L), (C), (R), (LS), and (LR) before encoding. That is, an output decoded by the decoding device 2 is obtained.

The audio signal obtained by the decoding apparatus 2 of the present embodiment having the above configuration is based on the phase and level of the signal from the audio signal (the audio signal of the encoding channel (L) (R)) as the encoded audio source. It is assumed that the audio signal component of the original channel is separated and extracted according to the closeness detection result. This is because, for example, the decoded audio signal does not contain audio signals of other channels at a fixed ratio in the decoded audio signal as in the case of the decoding output by the encoding / decoding technology described as a conventional example. This means that the audio signal of each channel can be considered substantially the same as the audio signal of each original channel before encoding.
As a result, when the audio signal that is the output of the decoding device 2 of the present embodiment is reproduced and output by a speaker or the like that is appropriately arranged according to each channel, the audio signal of the original channel is reproduced and output. It is possible to obtain an acoustic effect having a quality almost equal to that of the case. In other words, it is possible to listen to the reproduced sound of good channel separation without causing a change in volume or localization as in the conventional case.

By the way, the correction processing units 41a and 41b included in the channel signal separation blocks 32-L, 32-C, 32-R, 32-C, 32-LS, and 32-RS in the decoding device 2 are as described above. In addition, the characteristic is inverse to the transfer function of the impulse response given to each of the filters 11a, 11b to 15a, 15b of the encoding apparatus 1 shown in FIG. The impulse response corresponding to the inverse characteristic tends to be difficult to converge when the impulse response corresponding to the transfer function used for encoding is a complex and long response.
For example, FIG. 12A shows an example of an impulse response waveform assumed to be measured assuming a reverberant environment. As is well known, as an impulse response, as time progresses, first, a direct sound part that responds to a direct sound, and an indirect sound part that responds to a reflected sound (indirect sound) after the subsequent direct sound arrives, There is. In FIG. 12A, the response portion having the time width indicated by the section A is a direct sound portion, and the subsequent response portion is, for example, a reflected sound portion.
In general, when the response times of the direct sound portion and the reflected sound portion are compared, the reflected sound portion is considerably longer. In addition, the reflected sound part also has a large change in response time according to the measurement environment, conditions, and the like. For example, if the response time of the reflected sound part is long, a filter having the opposite characteristic becomes difficult to converge.

Therefore, in this embodiment, the cause that the inverse filter is difficult to converge is mainly due to the length of the impulse response of the transfer function that is the source of the inverse characteristics, and the length of the impulse response is mainly reflected. Focusing on the fact that it depends on the length of the sound part, the inverse characteristic is set as follows.
That is, as shown in FIG. 12B, an impulse response obtained by extracting only the response corresponding to the direct sound portion shown as the section A from the entire impulse response waveform of FIG. For example, assuming that the impulse response waveform in FIG. 12A corresponds to the transfer function Hll, the correction processing unit 41a of the channel signal separation block 32-L has the following configuration as shown in FIG. The inverse filter characteristic [1 / Hll] is obtained by an impulse response corresponding to the original transfer function Hll with the reflected sound portion omitted, and set in the correction processing unit 41a. Similarly, the remaining correction processing units 41a and 41b are configured to set the inverse filter characteristics obtained from the impulse response, although the reflected sound portion is omitted from the corresponding encoding transfer function.

  When the inverse filter characteristic is set in this way, the correction for the reflected sound component is not performed in the inverse filter process at the time of decoding, so that the proper separation for the signal component corresponding to the reflected sound part is not performed. It will not be possible. However, as is well known, the direct sound is more dominant in the impulse response, and therefore, the quality of the decoded audio is not particularly problematic.

  In addition, for the decode output channel where sound reproducibility is more important than other channels, for example, located on the front side of the listener, reverse filter characteristics including the reflected sound part are set, For the decode channel, an inverse filter characteristic from which the reflected sound portion is removed may be set so that the channel is properly used for each channel.

  In addition to using the direct sound part of the impulse response, transfer characteristics measured in an environment with no reverberation such as an anechoic room, or based on transfer characteristics obtained by calculation assuming an environment with no reverberation. It is also possible to consider a method of setting reverse characteristics. Since the transfer characteristic of an environment having no reverberation does not have a response of the reverberation part, for example, the reflection sound part is omitted without extracting the sound part directly from the impulse response as described with reference to FIG. An impulse response equivalent to that obtained can be obtained. However, in an environment with reverberation, the direct sound part of the impulse response also contains a reverberation component. Therefore, as in the former example, the direct sound part of the impulse response obtained in an environment with reverberation was used. However, the reproduced sound field will be richer.

FIG. 13 shows another example of the decoding device 2 of the present embodiment. In this figure, the same parts as those in FIG.
In this figure, each configuration of the channel signal separation blocks 32-L, 32-C, 32-R, 32-C, 32-LS, and 32-RS is different from that in FIG. That is, for the channel signal separation blocks 32-L, 32-R, 32-C, 32-LS, and 32-RS, the correction processing units 41a and 41b are omitted, and instead, one correction processing unit 41A is provided. . The correction processing unit 41A in this case is provided only on the signal Sgl side, and the signal Sgr is input to the separation processing unit 42 as it is.
The channel signal separation block 32-C is not provided with the correction processing unit 41A, and the signals Sgl and Sgr are input to the separation processing unit 42 as they are. As described above, the reason why the correction processing unit 41A is not provided only in the channel signal separation block 32-C is that the speaker SP-C, which is the sound source of the corresponding original channel, is understood from the following explanation. As shown in FIG. 5, if the speaker is positioned on the midplane of the listener, there will be a difference in propagation time and a level difference between the speakers SP-C and the left and right ears of the listener. This is because it can be treated as not.

For example, one of the important elements when the listener M listens to the sound heard from one sound source and perceives the localization of the sound source reaches (propagates) the left ear and right ear of the listener M from each speaker. The sound time difference (propagation time difference) can be mentioned first. Such a propagation time difference appears as a rise time difference of the impulse response, for example, as shown in FIGS. In this figure, the relationship between the impulse response of the path (transfer function Hll) for the sound of the speaker SP-L to reach the listener's left ear and the impulse response of the path (transfer function Hlr) for reaching the right ear is shown as an example. ing. For example, in this way, the rise time of the impulse response of the transfer function Hlr shown in FIG. 14B is delayed by the time Td with respect to the rise time of the impulse response of the transfer function Hll shown in FIG. ing. This time Td is, for example, the distance until the speaker M reaches the left ear of the listener M and the time it reaches the right ear because the speaker SP-L considered as a point sound source is biased to the left front of the listener M. This is caused by the difference in distance and the propagation time differing accordingly.
At the time of encoding, convolution processing of impulse responses according to the transfer functions Hll and Hlr is performed by the filters 11a and 11b, so that the original channels (L) and (R) included in each audio signal (L) (R) Between the signal components of (L), a rise time difference (Td) of the impulse response is generated as shown in FIGS. 14 (a) and 14 (b).

Therefore, by delaying the audio signal Sgl itself by the time difference Td, the audio signal component of the original channel (L) included in the audio signal Sgl and the audio signal component of the same original channel (L) included in the audio signal Sgr When the time difference Td is canceled and viewed as an impulse response, the rise times are made to coincide.
In this way, the correction processing unit 41A is provided to perform a filter process for delaying the audio signal Sgl by the time difference Td.
Since the signal is delayed by the correction processing unit 41A in this manner, as described above, the rise time between the audio signal components of the original channel (L) included in the audio signals Sgl and Sgr is the same. It is adjusted to become. That is, the time lag of the audio signal component of one specific original channel included in the audio signals Sgl and Sgr is corrected.

In the present embodiment, the correction processing unit 41A also corrects the level difference between the audio signal components of one specific common original channel included in the audio signals Sgl and Sgr. Yes.
For example, regarding the relationship between the speaker SP-L and the listener M in FIG. 3, the speaker SP-L is biased to the left front of the listener M, so that the listener M reaches the left ear and the right ear. Due to the difference in distance and the direction in which the sound reaches, the sound that reaches the left and right ears from the speaker SP-L has a level difference in addition to the propagation time difference.
For example, FIGS. 14C and 14D show the frequency characteristics of the impulse response according to the transfer functions Hll and Hlr, respectively. 14A and 14B, the basic frequency distribution characteristics are similar to each other, but the level difference is relatively remarkable as shown by the level difference Lv between the two. It has become. Such a level difference is also caused between the signal components of the original channel (L) to which the characteristics of the transfer functions Hll and Hlr included in the signals Sgl and Sgr are given, and the delay time. Together with the (propagation time difference), it becomes a factor that determines the localization of the sound source.
In the correction processing unit 41A of the channel signal separation block 32-L, the signal Sgl is delayed by the delay time Td as described above, and the level reduction process by the level difference Lv is also executed. .
Since the signal is delayed by the correction processing unit 41A in this way, the level between the audio signal components of the original channel (L) included in the audio signals Sgl and Sgr becomes the same as described above. To be adjusted. That is, the level difference of the audio signal component of one specific original channel included in the audio signals Sgl and Sgr is corrected.

  The separation processing unit 42 executes the same configuration and processing as previously shown in FIG. 8, and finally separates and outputs the signal of the original channel (L). However, in this case, the phase comparison / comparison unit 63 detects a time difference between signals. Accordingly, the coefficient generation unit 62 performs a sound source separation function calculation so that the coefficient rp is obtained according to the detected time difference.

  The correction processing unit 41A provided in the example of FIG. 13 may be configured to be capable of signal delay and level change, for example, as shown in FIGS. 7 and 8 in the previous embodiment. The correction processing units 41a and 41b provided can be configured more simply. Accordingly, the separation performance of the output audio signal after decoding is better in the configuration shown in FIGS. 7 and 8, but the configuration of the channel signal separation block 32 can be as much as possible even in the example of FIG. This is intended to extract only the audio signal component of the necessary channel by removing the signal component of the other channels, and therefore, for example, even when compared with a technique combining a conventional matrix circuit and a direction enhancement circuit. That is, sufficiently good reproducibility is maintained.

FIG. 15 shows a configuration example of a recording system to which the encoding apparatus of the present embodiment is applied.
The recording system shown in this figure includes an encoding unit 100 and a media recording unit 101.
The encoding unit 100 is a unit part having the same configuration as that of the encoding apparatus 1 of the present embodiment in the recording system. For example, an audio signal having a multi-channel configuration of Lch, Cch, Rch, LSch, and RSch produced as audio source content is input to the encoding unit 100. For example, the signal processing configuration shown in FIG. It is converted into a channel stereo audio signal and output.

The Lch and Rch audio signals obtained by encoding in this way are input to the media recording unit 101. The media recording unit 101 records the input Lch and Rch audio signals on a predetermined storage medium (media) 102. In this way, the encoded audio signal is stored in the medium 102 as content information, for example.
Such a recording system is used, for example, by a content creator, and provides a medium 102 storing audio information as a package medium. Further, the content as a 2-channel stereo audio signal obtained by the encoding unit 100 using Lch and Rch may be distributed via a network.

FIG. 16 shows a configuration example of a playback system to which the decoding device 2 of the present embodiment is applied.
The playback system shown in this figure includes a media playback unit 201 and a decode unit 200. The media playback unit 201 loads the media 102 and executes playback processing corresponding to the format of the media, thereby outputting audio signals of Lch and Rch that are audio sources after encoding.

  For example, the Lch and Rch audio signals reproduced by the media reproducing unit 201 can be first reproduced by headphones. As described above, for the listener wearing the headphones 6 at this time, for example, as shown in FIG. 3, it is as if the five speakers SP-L, SP-C, and SP-R installed around the speaker. , SP-LS and SP-RS can perceive a sound field where sound can be heard.

  Also, the Lch and Rch audio signals reproduced by the media reproducing unit 201 are input to the decoding unit 200. The decoding unit 200 has the same configuration as the decoding device 2 of the present embodiment having the configuration shown in FIGS. 6 to 8 or FIG. 13, for example, and performs the decoding process as described above to encode the decoding unit 200. The audio signal of the previous original channel is converted. The audio signals of the original channels (L), (C), (R), (LS), and (RS) thus obtained are amplified, for example, and actually installed speakers SP-L, SP-C, SP- R, SP-LS, and SP-RS are driven. When the sound output from the speaker thus driven is heard at an appropriate listening position, an ideal sound image localization as an audio source as the original channel is reproduced. Further, as described above, higher quality reproducibility can be obtained as compared with the case where the audio signal decoded by the conventional encoding / decoding technique is output from the speaker.

Also, when considering the recording system and the playback system as described above, in order to obtain the best decoding result on the playback system side, the same transmission as that used for the impulse response convolution process when encoding on the recording system side is performed. Based on the function (transfer characteristic), it is necessary to execute the inverse filter processing by the correction processing units 41a and 41b in the channel signal separation block 32 or the delay and level correction processing by the correction processing unit 41A.
For this purpose, first, only one transfer characteristic group to be used for encoding on the recording system side is determined. In the reproduction system, an inverse filter characteristic or a delay time is determined according to the determined transfer characteristic group. The correction processing unit 41 incorporating the level correction amount and the like is configured.

However, in the above case, the audio source content of the encoding source is determined because the position of the speaker assumed as the original channel and the acoustic environment such as the surrounding environment are determined as one according to the content of the audio source. Inconvenience such as loss of freedom in creating
Therefore, when creating the content, an acoustic environment can be arbitrarily created or selected from a plurality of predefined acoustic environments so that variations of the acoustic environment are given. When an audio source encoded by the recording system is recorded on the medium 102, an identification signal indicating the acoustic environment set for the original sound source before encoding is used according to a predetermined format or the like, or used for encoding determined according to the acoustic environment setting. An identification signal indicating a transfer function group is recorded together. On the playback system side, when the medium 102 is played back, the identification signal is also read out and output to the decode unit 200, for example. Based on the input identification signal, the decode unit 200 changes the parameter setting for a required signal processing unit such as the correction processing unit 41 in the channel signal processing block 32. A configuration example for this is shown in FIG.

17 shows a channel signal separation block 32-L having the same configuration as that of FIG. 8 and a parameter setting unit 400. The parameter setting by the parameter unit 400 is performed not only for the channel signal processing block 32-L but also for the remaining channel signal processing blocks. Here, for convenience of illustration and description, the channel is set. Only the relationship between the signal processing block 32-L and the parameter setting unit 400 is shown.
The parameter setting unit 400 reads the identification signal input to the decode unit 200. Then, based on the read identification signal, for example, the inverse filter characteristics to be set in the correction processing units 41a and 41b are determined as parameters.
In this case, the parameter setting unit 400 also determines the sound source separation functions of the coefficient generation units 62 and 64. For example, depending on the difference in the acoustic environment set at the time of encoding, it is necessary to change the sound source separation function used when generating the coefficients by the coefficient generating units 62 and 63, or it is more optimal to change the decoding result. This is because it may be preferable that the above is obtained.
If the configuration of the channel signal separation block 32 is as shown in FIG. 13, the delay time and the correction level amount of the correction processing unit 41A are used as parameters instead of the inverse filter characteristics of the correction processing units 41a and 41b. Determine as.

Here, as a method of determining (acquiring) each parameter by the parameter setting unit 400, the following can be considered.
First, when parameters to be set are stored in the structure of the identification signal (identification information), parameter information may be acquired from the read identification signal.
In addition, when the identification signal specifies an encoding type according to the acoustic environment at the time of encoding, for example, the parameter setting unit 400 prepares table information describing parameters according to the encoding type. It is conceivable that the parameter associated with the encoding type identified by the content of the identification signal is retrieved from the table information and acquired. Alternatively, a configuration may be considered in which an operation based on a predetermined arithmetic expression or function is executed according to the encoding type identified by the identification information, and the operation result is output as a parameter.
Further, the actual configuration of the parameter setting unit 400 may be realized by a computer having a CPU or the like executing a program for parameter setting.

  The inverse filter characteristic and the sound source separation function as parameters determined by the parameter setting unit 400 as described above are set for the correction processing units 41a and 41b and the coefficient generation units 62 and 64, respectively. For example, the inverse filter characteristics can be set by changing the coefficient of the multiplier in the digital filter forming the correction processing units 41a and 41b.

The parameter setting for the channel separation processing block 32-L by the parameter setting unit 400 as described above is the same for the remaining channel separation processing blocks 32-C, 32-R, 32-C, 32-LS, and 32-RS. Is done.
Correction processing units 41a and 41b in the channel separation processing blocks 32-L, 32-C, 32-R, 32-C, 32-LS, and 32-RS in which parameters are set in accordance with the identification signal in this way. When the coefficient generators 62 and 64 execute processing, for example, signal separation processing is performed using parameters that are optimized according to the encoding conditions. As a result, for example, decoding output is performed. As for the signal, the best signal that is very close to the sound signal of the original channel before encoding is obtained.

As a supplement, FIG. 18 shows another example of a reproduction system that reproduces and outputs an audio source encoded by the encoding apparatus 2 of the present embodiment.
The playback system shown in this figure includes a media playback unit 201 and a speaker drive unit 202. The media playback unit 201 plays back and outputs Lch and Rch audio signals, which are audio sources after encoding, from the media 102 in the same manner as shown in FIG.
Also in this case, the Lch and Rch audio signals reproduced by the media reproducing unit 201 can be reproduced and output as sound by the headphones.
The Lch and Rch audio signals reproduced by the media reproducing unit 201 are also input to the speaker driving unit 202.

  The speaker driving unit 202 performs amplification after performing necessary signal processing on the input Lch and Rch audio signals, and drives the two speakers SP-L and SP-R corresponding to the L and R channels. That is, in this playback system, the audio signals of Lch and Rch, which are audio sources after encoding, are not decoded by the configuration of the decoding device 2 of the present embodiment and are reproduced and output by the speaker system having a 5-channel configuration. Playback and output are performed by the same two-channel speaker system.

  As described above, the audio signals of the L and R channels, which are the audio sources encoded by the encoding apparatus 2 of the present embodiment, are reproduced before being encoded by a reproduction system compatible with normal L and R stereo. Sound image localization equivalent to that of listening with a speaker system corresponding to the channel configuration is obtained. However, in order to listen to the sound image localization that is faithful to the acoustic environment assumed at the time of encoding, playback using headphones is suitable. This is because the sound output from the headphone driver part directly reaches the listener's ears, so there is almost no crosstalk between the sound of the left and right channels. However, when the sound is reproduced by the speaker, for example, the sound output from the left channel speaker can be heard by reaching not only the left ear of the listener but also the right ear, and similarly, the right channel speaker. The sound output from can be heard not only by the listener's right ear but also by the left. That is, crosstalk occurs between the left and right channel speakers and the left and right ears of the listener. This is the main factor that hinders reproduction by proper sound image localization.

Therefore, the speaker drive unit 202 of the reproduction system shown in FIG. 18 has a signal processing function for canceling the above-described crosstalk as described below.
First, in FIG. 19, two speakers SP-L and SP-R corresponding to each of the L (left) and R (right) channels are arranged, and serve as the median plane of the speakers SP-L and SP-R. A model is shown in which a listener M is positioned at a position and sounds arriving from speakers SP-L and SP-R are listened to.
In this model, the transfer function of the path from the speaker SP-L to the left ear is Hsll, the transfer function of the path from the speaker SP-L to the right ear is Hslr, and the path from the speaker SP-R to the left ear. Hsrl and the transfer function of the path from the speaker SP-R to the right ear are indicated as Hsrr.
Among the paths according to the transfer function described above, the paths corresponding to the crosstalk are the path from the speaker SP-L to the right ear and the path from the speaker SP-R to the left ear. Except for these two routes from the model shown in FIG. 13, only the sound from the route from the speaker SP-L to the left ear and the route from the speaker SP-R to the right ear reaches the listener M. Will be the same as In other words, just like listening to the playback sound from headphones, you listen without crosstalk.
Therefore, the speaker drive unit 202 in FIG. 18 performs signal processing for removing transfer characteristics corresponding to the transfer functions Hslr and Hsrl of the path corresponding to the crosstalk from the input L and R channel audio signals. This means that it should be executed. As a result, crosstalk between the actual speakers SP-L and SP-R and the left and right ears of the listener is eliminated, and for the listener, for example, during encoding, which is equivalent to listening to playback sound through headphones. Sound image localization that is very faithful to the assumed acoustic environment can be perceived.

Next, the configuration for canceling the crosstalk in the speaker drive unit 202 will be described.
Here, it is assumed that the speakers SP-L and SP-R shown in FIG. 19 are arranged symmetrically with respect to the median plane of the listener M. Transfer functions Hsll and Hsrr corresponding to a path from the speaker SP-L to the left ear of the listener M and a path from the speaker SP-R to the right ear of the listener M, which are not talks,
Hsll = Hsrr = S
And Further, transfer functions Hsll and Hsrr corresponding to a path from the speaker SP-L to the right ear of the listener M and a path from the speaker SP-R to the left ear of the listener M, which are crosstalk,
Hslr = Hsrl = A
And Then, a transfer function C expressed by the following equation is defined.
C = -A / S

Using the transfer function C obtained as described above, a signal processing system for crosstalk cancellation in the speaker drive unit 202 can be configured as shown in FIG. 20, for example.
The signal processing system for crosstalk cancellation shown in FIG. 20 includes adders 211 and 213 and filters 212, 214, 215, and 216 as shown in the figure.
Of the input Lch and Rch audio signals, the Lch audio signal is input to the adder 211 and branched to be input to the filter 212. The filter 212 gives a transfer characteristic of the transfer function C to the Lch audio signal and outputs it to the adder 213.
In addition, the Rch audio signal is input to the adder 213 and branched to be input to the filter 214. The filter 214 gives a transfer characteristic of the transfer function C to the Rch audio signal and outputs it to the adder 211.

Depending on the adder 211, the Lch audio signal and the Rch audio signal to which the transfer function C is given are added, synthesized, and output. The signal output from the adder 211 is obtained by removing in advance components corresponding to transfer characteristics that reach the right ear of the listener M from the speaker SP-L in FIG. 19 from the original Lch audio signal. It becomes.
The adder 213 adds and synthesizes the Rch audio signal and the Lch audio signal to which the transfer characteristic C is given, and outputs the resultant signal. The signal output from the adder 211 is obtained by removing in advance components corresponding to transfer characteristics that reach the left ear of the listener M from the speaker SP-R from the original Rch audio signal.

The output of the adder 211 passes through the filter 215 and is output as an Lch playback audio signal, and the output of the adder 213 passes through the filter 216 and is output as an Rch playback audio signal. The filters 215 and 216 are provided to correct the frequency characteristics so as to be flattened by the filter characteristics F, for example.
When the speakers SP-L and SP-R are driven by the Lch playback audio signal and the Rch playback audio signal output in this way, a listener that actually listens to the sound emitted from the speakers SP-L and SP-L. As M, a state equivalent to listening to only the sound from the path from the speaker SP-L to the left ear of the listener M and the sound from the speaker SP-R to the right ear of the listener is obtained. Will be. That is, the crosstalk is canceled and the sound image localization corresponding to the acoustic environment assumed at the time of encoding can be perceived as in the case of listening with headphones.

FIG. 21 shows another configuration example of the signal processing system for canceling the crosstalk in the speaker drive unit 202.
In the configuration shown in this figure, an Lch signal is input to the filter 221 and the filter 222. The filter 221 executes a filtering process based on the filter characteristic F1, and the filter 221 executes a filtering process based on the filter characteristic F2.
Further, the Rch signal is subjected to filtering processing by the filter 223 having the filter characteristic F3 and filtering processing by the filter 224 having the filter characteristic F4.
An output obtained by adding the filter 221 and the filter 223 by the adder 211 becomes an Lch reproduction audio signal, and an output obtained by adding the filter 222 and the filter 224 by the adder 213 becomes an Rch reproduction audio signal.
The filter characteristics F1, F2, F3, and F4 of the filters 221, 222, 223, and 224 are expressed as follows in relation to the transfer function of FIG.
F1 = Hsrr / (Hsll × Hsrr−Hslr × Hsrl)
F2 = −Hslr / (Hsll × Hsrr−Hslr × Hsrl)
F3 = −Hsrl / (Hsll × Hsrr−Hslr × Hsrl)
F4 = Hsll / (Hsll × Hsrr−Hslr × Hsrl)

  Even in the configuration of FIG. 21, the signals output from the adders 211 and 213 have the same composition as the signals output from the adders 211 and 213 of FIG. Therefore, even when the Lch and Rch playback audio signals output through the processing of the configuration of FIG. 21 are driven by the speakers SP-L and SP-R, sound image localization equivalent to the case of listening with headphones is perceived. It will be possible.

By the way, in the description so far, the encoding apparatus 1 corresponds to a set of channel configurations of Lch, Cch, Rch, LSch, and RSch as original channels, and is set to a set of 2-channel configurations of Lch and Rch as encode channels. It is supposed to respond. However, this channel configuration is merely an example, and may be changed on the original channel side and the encode channel side. In addition, for example, the same content may have different channel configurations before and after encoding, and in this respect, the number of constituent channels may be the same before and after encoding. This is because even if the number of constituent channels is the same, the channel configuration is different if there is a difference in the sound source position between the channels, for example.
Further, the decoding device 2 as an embodiment inputs the audio source encoded by the encoding device 1 and decodes it as a decoding channel into the same channel configuration as the original channel. The channel configuration is not necessarily the same as the channel configuration of the original channel to which the encoding apparatus 1 corresponds, and may be another channel configuration. Such a decoding apparatus can be realized by setting a characteristic to be given to the correction processing unit 41 in consideration of a transfer function according to a model of a channel configuration after decoding.

  Furthermore, in the description so far, the encoding apparatus 1 and the decoding apparatus 2 of the present embodiment are individually provided in the recording system and the reproduction system, respectively, but the encoding apparatus 1 of the present embodiment. It is also possible to construct a recording / reproducing apparatus and a recording / reproducing system having both the configuration of the decoding device 2 and the decoding device 2.

In addition, the configuration as the encoding device 1 and the decoding device 2 according to the present embodiment described so far can be physically configured as, for example, an audio device having a sound recording and reproducing function. Further, the configuration of the signal processing system can be configured as a program. When the functions of the encoding device and decoding device of the present embodiment are configured by a program, signal processing as encoding and decoding is realized by a CPU or the like executing according to the program. And such a program can be written and memorize | stored at the time of manufacture etc. with respect to ROM etc. with which the apparatus which implement | achieves the function as an audio | voice reproducing apparatus is provided. Further, the program can be stored in, for example, a removable storage medium (magnetic disk, optical disk, semiconductor memory, etc.) and read out from the storage medium and executed by various devices such as a personal computer. Alternatively, the program stored in the storage medium can be installed in the device, and then the device can be configured to execute the installed program. In addition, it can be stored in a storage device such as a server on the network and executed after various devices are temporarily acquired via the network, or the device is installed via the network and the installed program is It may be configured to be executable.
Further, the invention of the present application is not limited to the example as the embodiment described so far, and can be appropriately changed. For example, in this embodiment, since the spatial transfer function of sound is assumed to have a path from the sound source to the listener's ear, it may be considered synonymous with the head-related transfer function. It is also possible that the position corresponding to the other position is not the listener's ear. In this case, a spatial transfer function in the original sense is used as a transfer characteristic representing a route from the sound source to the position of the target to be reached.

It is a figure which shows the example of an input-output channel structure which the encoding apparatus as embodiment of this invention corresponds. It is a figure which shows the example of a channel structure of the input / output which the decoding apparatus as embodiment of this invention corresponds. It is a figure which shows the model in the case of making into a sound source the channel structure of the audio | voice source encoded by the encoding apparatus of this Embodiment. It is a figure which shows the structural example of the encoding apparatus of embodiment. It is a figure which shows the structural example of the filter in the encoding apparatus of embodiment. It is a figure which shows the structural example of the decoding apparatus of embodiment. It is a figure which shows notionally the structural example of the channel signal separation block in the decoding apparatus of embodiment. It is a figure which shows the structural example of the channel signal separation block in the decoding apparatus of embodiment. It is a figure which compares and shows the transfer function Hll and the reverse characteristic with respect to this transfer function Hll by a frequency characteristic. It is a figure which shows the example of a function for a coefficient generation part to set the coefficient of a multiplier according to a level ratio. It is a figure which shows the example of a function for a coefficient generation part to set the coefficient of a multiplier according to a phase difference. It is a figure which shows the impulse response waveform in a sympathetic environment, and the response waveform which took out only the sound part directly from this impulse response waveform. It is a figure which shows the other structural example about the decoding apparatus of embodiment. It is a figure which shows the example of the propagation time difference and level difference which are produced according to the transfer function given to the sound of the same sound source. It is a figure which shows the structural example of a recording system provided with the encoding apparatus of embodiment. It is a figure which shows the structural example of a reproduction system provided with the decoding apparatus of embodiment. It is a figure which shows the structural example for changing and setting the parameter in a channel signal separation block according to an identification signal. It is a figure which shows the structural example of the reproduction | regeneration system which reproduces | regenerates the audio | voice source encoded by the encoding apparatus of embodiment. It is a figure which shows the model of an acoustic in case a sound source is 2 channels. It is a figure which shows the structural example for the crosstalk cancellation with which the speaker drive unit of FIG. 18 is equipped. It is a figure which shows the structural example for the crosstalk cancellation with which the speaker drive unit of FIG. 18 is equipped. It is a figure which shows the structural example of the encoding technique as the past.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Encoding apparatus, 2 Decoding apparatus, 6 Headphone, 11a-15a * 11b-15b filter, 16a * 16b 31a * 31b Fast Fourier-transform part, 32-L * 32-C * 32-R * 32-LS * 32-LR Channel signal separation block, inverse Fourier transform unit 33-L / 33-C / 33-R / 33-LS / 33-LR, 41a / 41b / 41A correction processing unit, 42 separation processing unit, 51 level / phase comparison processing block , 52 Sound source separation function calculation block, 53/54/65/66/67/68 Coefficient unit, 55 Adder, 61 Level comparison unit, 62/64 Coefficient generation unit, 63 Phase comparison unit, 100 Encoding unit, 101 Media recording Unit, 102 media, 200 decode unit, 201 media playback unit, 4 0 parameter setting unit

Claims (11)

For each of the audio signal components corresponding to the decode channel constituting a predetermined channel configuration, transfer characteristics represented by a spatial transfer function obtained based on the position of the sound source as the corresponding decode channel are given, and these audio signals are given. Audio signal generation means for generating an audio signal component corresponding to one specific channel in the decode channel by inputting an audio signal of the encode channel generated by distributing the signal component according to the channel configuration of the encode channel, Corresponding to each decoding channel,
Each of the audio signal generation means includes
For each input audio signal of the encoding channel, correction means for correcting the transfer characteristics given to the audio signal component of the decoding channel corresponding to the audio signal generation means,
Proximity detection means for detecting a predetermined approximation between the signals corrected by the correction means;
Separation means for separating and outputting signal components that are approximated to each other from signals for each encoding channel output from the signal correction means based on the detection result of the proximity detection means;
Channel audio signal output means for adding the signal components separated by the separation means and outputting as a corresponding decode channel audio signal;
An audio signal processing device. The correction means is
For each input audio signal of the encoding channel, the audio signal generating means is configured to execute a filtering process that gives an inverse characteristic based on the transfer characteristic given to the audio signal of the corresponding decoding channel. ,
The audio signal processing apparatus according to claim 1. The correction means is
The audio signal generating means executes a filtering process that gives the inverse characteristic of the direct sound part in the impulse response of the transfer characteristic given to the audio signal of the corresponding decode channel.
The audio signal processing apparatus according to claim 2. The correction means is
The audio signal generation means performs filtering processing that gives the inverse characteristic of the transfer characteristic due to the anechoic environment, which is given to the audio signal of the corresponding decoding channel.
The audio signal processing apparatus according to claim 2. The proximity detection means is
Detecting the proximity of the phase of the signal for each encoding channel after correction by the correction means;
The audio signal processing apparatus according to claim 2. The proximity detection means is
Detecting the level approximation of the signal for each encoding channel after correction by the correction means;
The audio signal processing apparatus according to claim 2. The correction means is
A process for correcting a propagation time difference related to an audio signal component of a corresponding decode channel caused by an assigned transfer characteristic between audio signals for each input encode channel is executed.
The proximity detection means is
Detecting the propagation time difference for the signal for each encoding channel after correction by the correction means as approximation,
The audio signal processing apparatus according to claim 1. The correction means is
Furthermore, a process for correcting a level difference related to the audio signal component of the corresponding decode channel caused by the imparted transfer characteristic between the audio signals of each input encode channel is executed.
The proximity detection means is
Furthermore, a process for correcting the level difference for the signal for each encoding channel after correction by the correction means is performed.
8. An audio signal processing apparatus according to claim 7, An encoding apparatus that converts a set of audio signals of an original channel having a predetermined channel configuration into a set of audio signals of an encode channel having a predetermined channel configuration other than the original channel, and outputs the set;
A decoding device that inputs a set of audio signals of an encoding channel that constitutes a predetermined channel configuration and converts the set to an audio signal set of a decoding channel that constitutes a predetermined channel configuration;
The encoding device is
Each original channel has a corresponding one for each encode channel, and the input audio signal has a transfer characteristic represented by a spatial transfer function set based on the position of the sound source as the corresponding original channel. Transfer characteristic imparting means for imparting to the audio signal;
An adder that is provided corresponding to each encode channel, inputs and adds signals processed by each of the transfer characteristic assigning means, and outputs the added output as an audio signal of the corresponding encode channel; With
The decoding device is
Audio signal separation means for separating an audio signal component corresponding to one specific channel in the decode channel is provided for each decode channel,
Each of the audio signal separation means is
Correction means for correcting the transfer characteristic given to the audio signal component of the corresponding decode channel for the input audio signal for each encode channel;
Proximity detection means for detecting a predetermined approximation for a signal for each encoding channel after correction by the correction means;
Separation means for separating and outputting signal components that are approximated to each other from signals for each encoding channel output from the signal correction means based on the detection result of the proximity detection means;
Channel audio signal output means for adding the signal components separated by the separation means and outputting as a corresponding decode channel audio signal;
An audio signal processing system. For each of the audio signal components corresponding to the decode channel constituting a predetermined channel configuration, transfer characteristics represented by a spatial transfer function obtained based on the position of the sound source as the corresponding decode channel are given, and these audio signals are given. An audio signal generation procedure for generating an audio signal component corresponding to one specific channel in the decode channel by inputting an audio signal of the encode channel generated by distributing the signal component according to the channel configuration of the encode channel, It is executed corresponding to each decoding channel,
As an audio signal generation procedure corresponding to each decoding channel,
For each input audio signal of the encoding channel, a correction procedure for correcting the transfer characteristics given to the audio signal component of the decoding channel corresponding to the audio signal generation procedure;
An approximation detection procedure for detecting a predetermined approximation between the signals corrected by the correction procedure;
Based on the detection result of the proximity detection procedure, a separation procedure for separating and outputting signal components that are approximated to each other from the signals for each encoding channel obtained by the signal correction procedure;
A channel audio signal output procedure for adding the signal components separated by the separation procedure and outputting as an audio signal of the corresponding decode channel;
For causing an information processing apparatus to execute the program. An encoding process for converting an audio signal set of an original channel forming a predetermined channel configuration into an audio signal set of an encoding channel forming a predetermined channel configuration other than the original channel, and outputting the set;
An information processing apparatus is configured to execute a decoding process of inputting a set of audio signals of an encode channel that forms a predetermined channel configuration and converting the set of audio signals of a decode channel that forms a predetermined channel configuration,
The above encoding process
A transfer characteristic represented by a spatial transfer function set based on the position of the sound source as the corresponding original channel, which is a procedure to be executed corresponding to each encode channel for each original channel. , A transfer characteristic applying procedure to be applied to the input audio signal,
An addition procedure that is provided corresponding to each encode channel, inputs the signals processed by each of the transfer characteristic imparting procedures, adds them, and outputs the added output as an audio signal of the corresponding encode channel; Is executed by the information processing device,
The decoding process is
An audio signal separation procedure for separating an audio signal component corresponding to one specific channel in the decode channel is to be executed for each decode channel,
Each of the above audio signal separation procedures includes:
A correction procedure for correcting the transfer characteristic given to the audio signal component of the corresponding decode channel for the input audio signal for each encode channel;
A proximity detection procedure for detecting a predetermined proximity of the signal for each encoding channel after correction by the correction procedure;
Based on the detection result of the proximity detection procedure, a separation procedure for separating and outputting signal components that are approximated to each other from the signals for each encoding channel obtained by the signal correction procedure;
Adding the signal components separated by the separation procedure, and causing the information processing apparatus to execute a channel audio signal output procedure for outputting the audio signal of the corresponding decode channel;
A program characterized by that.

Images