JP2011239036A - Audio signal converter, method, program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an audio signal converter which can convert an audio signal for a multi-channel system without generating any noise caused by a discontinuous point.SOLUTION: An audio signal converter (illustrated by an audio signal processing part 113) comprises: a converter to perform discrete Fourier transformation to input audio signals of two channels; a correlation signal extraction part to extract correlation signals about the audio signals of the two channels after the discrete Fourier transformation by the converter, while ignoring a DC component; an inverse conversion part to perform discrete Fourier inverse transformation to the correlation signals extracted by the correlation signal extraction part, or to the correlation signals and non-correlation signals, or to audio signals generated from the correlation signals or from the correlation signals and non-correlation signals; and a noise removal part 122 to remove discontinuous points of waveform from the audio signals after the discrete Fourier inverse transformation in the inverse conversion part.

Description

  The present invention relates to an audio signal conversion apparatus, method, program, and recording medium for converting an audio signal for a multi-channel playback system.

  Conventionally proposed sound reproduction methods include a stereo (2ch) method, a 5.1ch surround method (ITU-R BS.775-1), and the like, which are widely used for consumer use. The 2ch system is a system for generating different audio data from the left speaker 11L and the right speaker 11R as schematically illustrated in FIG. The 5.1ch surround system is, as schematically illustrated in FIG. 2, a left front speaker 21L, a right front speaker 21R, a center speaker 22C, a left rear speaker 23L, a right rear speaker 23R disposed between them, This is a method of inputting and outputting different audio data to a subwoofer dedicated to a low sound range (generally 20 Hz to 100 Hz) not shown.

  In addition to the 2ch system and 5.1ch surround system, various sound reproduction systems such as 7.1ch, 9.1ch, and 22.2ch have been proposed. In any of the methods described above, each speaker is arranged on a circumference or a spherical surface centered on the listener (listener), and ideally a listening position (listening position) that is equidistant from each speaker, so-called sweet. It is preferable to listen at a spot. For example, it is preferable to listen to the sweet spot 12 in the 2ch system and the sweet spot 24 in the 5.1ch surround system. When listening at the sweet spot, the synthesized sound image based on the balance of sound pressure is localized where the producer intended. Conversely, when listening at a position other than the sweet spot, the sound image / quality is generally deteriorated. Hereinafter, these methods are collectively referred to as a multi-channel reproduction method.

On the other hand, apart from the multi-channel playback method, there is also a sound source object-oriented playback method. This method is a method in which all sounds are sounds emitted by any sound source object, and each sound source object (hereinafter referred to as “virtual sound source”) includes its own position information and audio signal. It is out. Taking music content as an example, each virtual sound source includes the sound of each musical instrument and position information where the musical instrument is arranged.
The sound source object-oriented reproduction method is usually reproduced by a reproduction method (that is, a wavefront synthesis reproduction method) in which a sound wavefront is synthesized by a group of speakers arranged in a straight line or a plane. Among such wavefront synthesis reproduction systems, the Wave Field Synthesis (WFS) system described in Non-Patent Document 1 is one of the practical mounting methods using linearly arranged speaker groups (hereinafter referred to as speaker arrays). Has been actively studied in recent years.

Such a wavefront synthesis reproduction method is different from the above-described multi-channel reproduction method, as shown schematically in FIG. 3, for a listener who is listening at any position in front of the arranged speaker groups 31. However, it has the feature that both good sound image and sound quality can be presented at the same time. That is, the sweet spot 32 in the wavefront synthesis reproduction system is wide as shown in the figure.
In addition, a listener who is listening to sound while facing the speaker array in an acoustic space provided by the WFS method is actually a sound source in which the sound radiated from the speaker array virtually exists behind the speaker array. Feels like being emitted from (virtual sound source).

  This wavefront synthesis reproduction method requires an input signal representing a virtual sound source. In general, one virtual sound source needs to include an audio signal for one channel and position information of the virtual sound source. Taking the above-described music content as an example, for example, it is an audio signal recorded for each musical instrument and position information of the musical instrument. However, the sound signal of each virtual sound source does not necessarily need to be for each musical instrument, but the arrival direction and magnitude of each sound intended by the content creator must be expressed using the concept of virtual sound source. .

Here, since the most widespread method among the above-mentioned multi-channel methods is the stereo (2ch) method, a stereo-type music content will be considered. As shown in FIG. 4, the audio signals of the L (left) channel and the R (right) channel in stereo music contents are installed on the left speaker 41L and on the right using two speakers 41L and 41R, respectively. Playback is performed by the speaker 41R. When such reproduction is performed, as shown in FIG. 4, only when listening at a point equidistant from each of the speakers 41L and 41R, that is, the sweet spot 43, the voice of the vocal and the sound of the bass can be heard from the middle position 42b. The sound image is localized and heard as intended by the producer, such as a piano sound on the left side 42a and a drum sound on the right side 42c.
It is considered that such content is reproduced by the wavefront synthesis reproduction method, and the sound image localization as intended by the content producer is provided to the listener at any position, which is a feature of the wavefront synthesis reproduction method. For this purpose, it is necessary to be able to perceive a sound image when listening in the sweet spot 43 of FIG. 4 from any viewing position, such as the sweet spot 53 shown in FIG. That is, the vocal group and the sound of the bass are heard from the middle position 52b at the wide sweet spot 53 by the speaker group 51 arranged in a straight line or a plane, and the piano sound is heard from the left position 52a and the drum sound. The sound image must be localized and heard as intended by the producer, such as the right position 52c.

  For example, consider the case where the L channel sound and the R channel sound are arranged as virtual sound sources 62a and 62b, respectively, as shown in FIG. In this case, since each L / R channel does not represent a single sound source alone but generates a synthesized sound image by two channels, the sweet spot 63 is still generated even if it is reproduced by the wavefront synthesis reproduction method. The sound image is localized as shown in FIG. 4 only at the position of the sweet spot 63. That is, in order to realize such sound image localization, it is necessary to separate 2ch stereo data into sound for each sound image by some means and generate virtual sound source data from each sound.

  In response to this problem, the method described in Patent Document 1 separates 2ch stereo data into a correlated signal and an uncorrelated signal based on the correlation coefficient of the signal power for each frequency band. And a virtual sound source is generated from the results.

European Patent Application No. 1761110

AJ Berkhout, D. de Vries, and P. Vogel, “Acoustic control by wave field synthesis”, J. Acoust. Soc. Am. Volume 93 (5), United States, Acoustical Society of America, May 1993, pp. 2764- 2778

  However, in the method described in Patent Document 1, the DC component of the left and right channels after the discrete Fourier transform is ignored when analyzing the original audio signal. FIG. 7 is a schematic diagram illustrating an example of a result obtained when a discrete Fourier transform is performed on an audio signal. In FIG. 7, the vertical axis represents the real part, the front axis represents the imaginary part, and reference numeral 71 represents a DC component. In the method described in Patent Document 1, since the direct current component 71 is ignored, the continuity of the waveform between segments after inverse Fourier transform is not guaranteed, and the waveform becomes discontinuous at the segment boundary. Particularly in content including many low-band signals, the generated audio signal waveform includes many discontinuities, which are perceived as noise.

  This noise will be described using the example of the music content 80 shown in FIG. When the audio signal 81 of the left channel and the audio signal 82 of the right channel in the music content 80 are converted into, for example, five channels using the method described in Patent Document 1, a result like the music content 90 shown in FIG. 9 is obtained. . The music content 90 will have five channels of audio signals 91-95. FIG. 10 is an enlarged view of the vicinity of 9 seconds in the audio signal 93 of the third channel from the top of FIG. 9, but in the audio signal 100 shown in FIG. Has occurred. Since many such discontinuous points are included, it is perceived as annoying noise.

  Such a problem is not limited to the case where a multi-channel audio signal is converted into an audio signal to be reproduced by the wavefront synthesis reproduction method, but also for the multi-channel method (even if the number of channels is the same). This may also occur in the case of conversion to a sound signal that may be different. This is because even in the case of such conversion, the discrete Fourier transform / inverse transform as described above may be performed and the DC components of the left and right channels may be ignored.

  The present invention has been made in view of the above-described actual situation, and an object of the present invention is to generate a multichannel audio signal such as 2ch or 5.1ch without generating noise due to discontinuities. An audio signal conversion apparatus, method, program, and recording medium that can be converted are provided.

  In order to solve the above-described problems, a first technical means of the present invention is an audio signal conversion apparatus that converts a multi-channel input audio signal into an audio signal for reproduction by a speaker group. A conversion unit that performs discrete Fourier transform on the input audio signal of one channel; and a correlation signal extraction unit that extracts a correlation signal by ignoring a direct current component for the audio signal of two channels after the discrete Fourier transform by the conversion unit; The correlation signal extracted by the correlation signal extraction unit or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal An inverse transform unit that performs discrete Fourier inverse transform on the audio signal, and a removal unit that removes discontinuous points of the waveform from the audio signal after the discrete Fourier inverse transform by the inverse transform unit; It is obtained characterized by including.

  According to a second technical means, in the first technical means, the removing unit adds a direct current component to the audio signal after the discrete Fourier transform so as to maintain the differential value of the waveform at the boundary of the processing segment. The discontinuous points are removed.

  A third technical means is characterized in that, in the second technical means, the removing unit reduces the magnitude of the amplitude of the DC component to be added in proportion to the elapsed time from the addition time point. is there.

  According to a fourth technical means, in the third technical means, the removing unit changes the proportionality constant for the reduction according to the magnitude of the amplitude of the DC component obtained for addition. It is what.

  According to a fifth technical means, in the fourth technical means, the removing unit, except for a place where the number of times that the waveform of the speech signal after the discrete Fourier inverse transform crosses 0 is greater than or equal to a predetermined number of times within a predetermined time, The addition of the DC component is executed.

  A sixth technical means is the technical means according to any one of the second to fifth technical means, wherein the removing unit is configured to detect the DC component only when the amplitude of the DC component obtained for addition is less than a predetermined value. It is characterized by performing addition.

  A seventh technical means is the technical means according to any one of the first to third technical means, wherein the removing unit is configured such that the number of times that the waveform of the sound signal after the inverse discrete Fourier transform crosses 0 is a predetermined number of times within a predetermined time. The removal of the discontinuous points is executed in places other than the above existing locations.

  According to an eighth technical means, in any one of the first to seventh technical means, the speech signal after the inverse discrete Fourier transform to be processed by the removing unit is the correlation signal or the correlation signal and the non-correlation signal. The correlation signal is subjected to a scaling process in the time domain or the frequency domain to obtain an audio signal after the scaling process.

  According to a ninth technical means, in any one of the first to eighth technical means, the multi-channel input audio signal is an input audio signal of three or more channels, and any of the multi-channel input audio signals is selected. An audio signal to be reproduced by the speaker group by removing the discontinuous points from the two input audio signals by the conversion unit, the correlation signal extraction unit, the inverse conversion unit, and the removal unit. The audio signal conversion apparatus further includes an adder that adds the input audio signals of the remaining channels to the generated audio signal.

  According to a tenth technical means, in any one of the first to ninth technical means, a digital content input unit for inputting digital content including the multi-channel input audio signal, a decoder unit for decoding the digital content, An audio signal extraction unit that separates an audio signal from the digital content decoded by the decoder unit, and a multi-channel audio that is 3 channels or more different from the input audio signal from the audio signal extracted by the audio signal extraction unit An audio signal processing unit that converts the signal into a signal, and the audio signal processing unit includes the conversion unit, the correlation signal extraction unit, the inverse conversion unit, and the removal unit.

  According to an eleventh technical means, in the tenth technical means, the digital content input unit receives the digital content from a recording medium for storing the digital content, a server for distributing the digital content via a network, or a broadcasting station for broadcasting the digital content. It is characterized by inputting.

  The twelfth technical means further comprises a switching unit that switches whether to execute the processing in the audio signal processing unit according to a user operation in any one of the first to eleventh technical means. It is a feature.

  A thirteenth technical means is an audio signal conversion method for converting a multi-channel input audio signal into an audio signal to be reproduced by a speaker group, wherein the conversion unit performs discrete Fourier transform on the input audio signal of two channels. A conversion step for applying a correlation signal extraction unit, an extraction step for ignoring a direct current component and extracting a correlation signal for the two-channel audio signals after discrete Fourier transform in the conversion step, and an inverse conversion unit, The correlation signal extracted in the extraction step or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the voice signal generated from the correlation signal and the non-correlation signal On the other hand, the inverse transform step for performing the discrete Fourier inverse transform, and the removing unit from the speech signal after the discrete Fourier inverse transform in the inverse transform step. It is obtained by comprising: the removal step of removing the discontinuities in the form, the.

In a fourteenth technical means, a computer performs a discrete Fourier transform on the input audio signals of two channels, and ignores the DC component of the audio signals of the two channels after the discrete Fourier transform in the conversion step. An extraction step for extracting a correlation signal; the correlation signal extracted in the extraction step or the correlation signal and the non-correlation signal; or the voice signal generated from the correlation signal; or the correlation signal and the An inverse transform step for performing discrete Fourier inverse transform on the speech signal generated from the uncorrelated signal, and a removal step for removing discontinuous points of the waveform from the speech signal after the discrete Fourier inverse transform in the inverse transform step; Is a program for executing
The fifteenth technical means is a computer-readable recording medium recording the program according to the fourteenth technical means.

  According to the present invention, it is possible to convert a multichannel audio signal such as 2ch or 5.1ch without generating noise caused by discontinuities.

It is a schematic diagram for demonstrating a 2ch system. It is a schematic diagram for demonstrating a 5.1ch surround system. It is a schematic diagram for demonstrating a wavefront synthetic | combination reproduction | regeneration system. It is a schematic diagram which shows a mode that the music content by which the sound of the vocal, the bass, the piano, and the drum was recorded by the stereo system is reproduced using two right and left speakers. FIG. 5 is a schematic diagram showing an ideal sweet spot when the music content of FIG. 4 is reproduced by the wavefront synthesis reproduction method. FIG. 5 is a schematic diagram showing a state of an actual sweet spot when the audio signal of the left / right channel in the music content of FIG. 4 is reproduced by the wavefront synthesis reproduction method by setting a virtual sound source at the position of the left / right speaker, respectively. . It is a schematic diagram which shows an example of a result when carrying out discrete Fourier transform of the audio | voice signal. It is a figure which shows an example of the waveform of the music content which consists of an audio | voice signal of a left channel and a right channel. It is a figure which shows the waveform of the result of having converted the music content of FIG. 8 into five channels using the conventional method. FIG. 10 is an enlarged view of a part of an audio signal of one channel in the music content of FIG. 9. It is a block diagram which shows one structural example of the audio | voice data reproduction apparatus provided with the audio | voice signal converter concerning this invention. It is a block diagram which shows one structural example of the audio | voice signal processing part (audio | voice signal converter concerning this invention) in the audio | voice data reproduction | regeneration apparatus of FIG. FIG. 13 is a flowchart for explaining an example of audio signal processing in an audio signal separation and extraction unit and a noise removal unit in the audio signal processing unit of FIG. 12. It is a figure which shows a mode that audio | voice data are stored in a buffer in the audio | voice signal processing part of FIG. It is a schematic diagram for demonstrating the example of the positional relationship of a listener, a right-and-left speaker, and a synthesized sound image. It is a schematic diagram for demonstrating the example of the positional relationship of the speaker group and virtual sound source which are used with a wavefront synthetic | combination reproduction | regeneration system. It is a schematic diagram for demonstrating the example of the positional relationship of the virtual sound source of FIG. 16, a listener, and a synthesized sound image. FIG. 6 is a schematic diagram for explaining waveform discontinuities occurring at segment boundaries after inverse discrete Fourier transform when the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored. It is a schematic diagram for demonstrating an example of the discontinuous point removal process which concerns on this invention. It is a figure which shows the waveform of the result of having converted the music content which consists of the audio | voice signal of a left-right channel into five channels by applying the discontinuous point removal process of FIG. It is a figure which shows the waveform of the result of having applied the other discontinuous point removal process which concerns on this invention, and having converted the music content same as the music content made into object in FIG. 20 into five channels. 21 shows the waveform obtained as a result of converting music content having a drastic change in audio signal waveform different from the music content targeted in FIGS. 20 and 21 into five channels by applying the same discontinuous point removal processing as in FIG. FIG. It is a figure which shows the waveform of the result of having applied the other discontinuous point removal process which concerns on this invention, and having converted the music content same as the music content made into object in FIG. 22 into five channels. It is a figure which shows the waveform of the result of having applied the discontinuous point removal process of FIG. 23, and having converted the music content of FIG. 8 into five channels. It is the figure which expanded a part of audio | voice signal of one channel among the music contents of FIG. It is a schematic diagram for demonstrating the example of the positional relationship of the speaker group to be used, and a virtual sound source, when reproducing | regenerating a 5.1ch audio | voice signal by a wavefront synthetic | combination reproduction | regeneration system. It is a figure which shows the structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. The figure which shows the structural example of the video projection system provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the video projection system provided with the audio | voice data reproduction | regeneration apparatus of FIG. The figure which shows the structural example of the system which consists of a television board provided with the audio | voice data reproducing apparatus of FIG. 11, and a television apparatus. It is a figure which shows the example of the motor vehicle provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the example of the speaker of reproduction | regeneration object in the audio | voice data reproduction | regeneration apparatus of FIG.

  An audio signal conversion apparatus according to the present invention is an apparatus for converting an audio signal for multi-channel reproduction system into an audio signal for reproduction with a group of speakers having the same or different number of channels, an audio signal for wavefront synthesis reproduction system, or the like. Therefore, it can also be called an audio signal processing device, an audio data conversion device, etc., and can be incorporated into an audio data reproducing device. Of course, the audio signal is not limited to a signal in which a so-called audio is recorded, and can also be called an acoustic signal. The wavefront synthesis reproduction method is a reproduction method in which a wavefront of sound is synthesized by a group of speakers arranged in a straight line or a plane as described above.

Hereinafter, a configuration example and a processing example of an audio signal conversion device according to the present invention will be described with reference to the drawings. In the following description, first, an example in which the audio signal conversion apparatus according to the present invention generates an audio signal for the wavefront synthesis reproduction method by conversion will be given.
FIG. 11 is a block diagram showing an example of the configuration of an audio data reproducing apparatus provided with the audio signal converting apparatus according to the present invention. FIG. 12 shows an audio signal processing unit (according to the present invention) in the audio data reproducing apparatus of FIG. It is a block diagram which shows one structural example of an audio | voice signal converter.

  The audio data reproduction device 110 illustrated in FIG. 11 includes a decoder 111, an audio signal extraction unit 112, an audio signal processing unit 113, a D / A converter 114, an amplifier group 115, and a speaker group 116. The decoder 111 decodes the content of audio only or video with audio, converts it into a signal processable format, and outputs it to the audio signal extraction unit 112. The content is acquired by downloading from the Internet from a digital broadcast content transmitted from a broadcasting station, a server that distributes digital content via a network, or reading from a recording medium such as an external storage device. As described above, although not shown in FIG. 11, the audio data reproducing apparatus 110 includes a digital content input unit that inputs digital content including a multi-channel input audio signal. The decoder 111 decodes the digital content input here. The audio signal extraction unit 112 separates and extracts an audio signal from the obtained signal. Here, it is a 2ch stereo signal. The signals for the two channels are output to the audio signal processing unit 113.

  The audio signal processing unit 113 generates multi-channel audio signals (which will be described as signals corresponding to the number of virtual sound sources in the following example) from three or more channels and different from the input audio signal from the obtained two-channel signals. . That is, the input audio signal is converted into another multi-channel audio signal. The audio signal processing unit 113 outputs the audio signal to the D / A converter 114. The number of virtual sound sources can be determined in advance if there is a certain number or more, but the amount of calculation increases as the number of virtual sound sources increases. Therefore, it is desirable to determine the number in consideration of the performance of the mounted device. In this example, the number is assumed to be 5.

  The D / A converter 114 converts the obtained signal into an analog signal and outputs each signal to the amplifier 115. Each amplifier 115 amplifies the input analog signal and transmits it to each speaker 116, and is output from each speaker 116 as sound into the space.

  FIG. 12 shows a detailed configuration of the audio signal processing unit in this figure. The audio signal processing unit 113 includes an audio signal separation / extraction unit 121, a noise removal unit 122, and an audio output signal generation unit 123.

  The audio signal separation / extraction unit 121 generates an audio signal corresponding to each virtual sound source from the two-channel signals, and outputs it to the noise removal unit 122. The noise removing unit 122 removes a perceptual noise part from the obtained sound signal waveform, and outputs the sound signal after the noise removal to the sound output signal generating unit 123. The audio output signal generation unit 123 generates each output audio signal waveform corresponding to each speaker from the obtained audio signal. The audio output signal generation unit 123 performs processing such as wavefront synthesis reproduction processing. For example, the obtained audio signal for each virtual sound source is assigned to each speaker, and an audio signal for each speaker is generated. A part of the wavefront synthesis reproduction processing may be performed by the audio signal separation / extraction unit 121.

  Next, an example of audio signal processing in the audio signal separation / extraction unit 121 and the noise removal unit 122 will be described with reference to FIG. 13 is a flowchart for explaining an example of the audio signal processing in the audio signal separation / extraction unit and the noise removal unit in the audio signal processing unit in FIG. 12, and FIG. 14 shows the audio in the audio signal processing unit in FIG. It is a figure which shows a mode that data are stored in a buffer.

  First, the audio signal separation / extraction unit 121 reads out audio data having a length of half of one segment from the extraction result of the audio signal extraction unit 112 in FIG. 11 (step S131). Here, the audio data refers to a discrete audio signal waveform sampled at a sampling frequency such as 48 kHz. A segment is an audio data section composed of a group of sample points having a certain length, and here, it is assumed that the section length is an object of discrete Fourier transform later. For example, the value is 1024. In this example, 512 points of audio data that are half the length of one segment are to be read.

  The read 512-point audio data is stored in the buffer 140 illustrated in FIG. This buffer can hold the sound signal waveform for the immediately preceding segment, and the past segments are discarded. Audio data for one segment is created by connecting the data for the previous half segment and the data for the latest half segment, and the process proceeds to window function calculation (step S132). That is, all the sample data is read twice in the window function calculation.

ここで、ｍは自然数、Ｍは１セグメント長で偶数とする。ステレオの入力信号をそれぞれｘ_Ｌ（ｍ）、ｘ_Ｒ（ｍ）とすると、窓関数乗算後の音声信号ｘ′_Ｌ（ｍ）、ｘ′_Ｒ（ｍ）は、 In the window function calculation in step S132, the audio data for one segment is multiplied by a conventionally proposed next Hann window.

Here, m is a natural number, M is an even number of one segment length. If the stereo input signals are x _L (m) and x _R (m), respectively, the audio signals x ′ _L (m) and x ′ _R (m) after the window function multiplication are

x ′ _L (m) = w (m) × _L (m)
x ′ _R (m) = w (m) × _R (m) (2)
Is calculated. Using this Hann window, for example, the input signal x _L (m ₀ ) at the sample point m ₀ (M / 2 ≦ m ₀ <M) is multiplied by sin ² ((m ₀ / M) π). . And in the next reading, the same sample point is read as m ₀ -M / 2.

が乗算される。ここで、ｓｉｎ^２（（ｍ_０／Ｍ）π）＋ｃｏｓ^２（（ｍ_０／Ｍ）π）＝１であるから、もし、何も修正を加えずに読み込んだ信号を半セグメントずつずらして加算すれば、元の信号が完全に復元されることになる。

Is multiplied. Here, since sin ² ((m ₀ / M) π) + cos ² ((m ₀ / M) π) = 1, the signal read without any correction is shifted by half a segment and added. Then, the original signal is completely restored.

The audio data thus obtained is subjected to discrete Fourier transform as in the following formula (3) to obtain audio data in the frequency domain (step S133). Here, DFT represents discrete Fourier transform, k is a natural number, and 0 ≦ k <M. X _L (k) and X _R (k) are complex numbers.
X _L (k) = DFT (x ′ _L (n))
X _R (k) = DFT (x ′ _R (n)) (3)

  Next, the obtained frequency domain audio data is divided into smaller bands, and the processes of steps S135 to S138 are executed for each of the divided bands (steps S134a and S134b). Specific processing will be described.

First, as a division method, an Equivalent Rectangular Band (ERB) is used, and the ERB bandwidth is divided from 0 Hz to half the sampling frequency. Here, how many times the upper limit f _max [Hz] of a given frequency is divided by ERB, that is, the maximum value I of the index of each band divided by ERB is given by the following equation.
I = floor (21.4 log ₁₀ (0.000043 f _max +1)) (4)
However, floor (a) is a floor function and represents the maximum value of an integer not exceeding the real number a.

The center frequency F _c ⁽ⁱ⁾ (1 ≦ i ≦ I) [Hz] of each ERB width band (hereinafter referred to as a small band) is given by the following equation.

Further, the bandwidth b ⁽ⁱ⁾ [Hz] of the ERB at that time is obtained by the following equation.
b ⁽ⁱ⁾ = 24.7 (0.000043F _c ⁽ⁱ⁾ +1) (6)
Therefore, the boundary frequencies F _L ⁽ⁱ⁾ and F _U ⁽ⁱ⁾ on both sides of the i-th small band are obtained by shifting the center frequency from the low frequency side to the high frequency side by the frequency width of ERB / 2. Can do. Accordingly, the i th small band includes the K _U ⁽ⁱ⁾ th line spectrum from the K _L ⁽ⁱ⁾ th line spectrum. Here, K _L ⁽ⁱ⁾ and K _U ⁽ⁱ⁾ are expressed by the following equations (7) and (8), respectively.
^{_{K L (i) = ceil (}} 21.4log 10 (0.00437F L (i) +1)) (7)
K _U ⁽ⁱ⁾ = floor (21.4 log ₁₀ (0.0000437 F _U ⁽ⁱ⁾ +1)) (8)
However, ceil (a) is a ceiling function and represents the minimum value of an integer that is not smaller than the real number a. Further, the line spectrum after the discrete Fourier transform is symmetric with respect to M / 2 (where M is an even number) except for a direct current component, that is, X _L (0), for example. That is, X _L (k) and X _L (M−k) have a complex conjugate relationship in the range of 0 <k <M / 2. Therefore, in the following, the range of K _U ⁽ⁱ⁾ ≦ M / 2 is considered as the object of analysis, and the range of k> M / 2 is treated the same as a symmetric line spectrum having a complex conjugate relationship.

  Specific examples of these will be shown. For example, when the sampling frequency is 48000 Hz, I = 49, which is divided into 49 small bands. However, although there are line spectral components corresponding to frequencies higher than the highest sub-band section, they have almost no audible effect and usually have a small value, so they are the highest sub-band. It can be included in the section.

Next, in each small band determined in this manner, the correlation coefficient is obtained by obtaining the normalized correlation coefficient of the left channel and the right channel by the following equation (step S135).

This normalized correlation coefficient d ⁽ⁱ⁾ represents how much the audio signals of the left and right channels are correlated, and takes a real value between 0 and 1. 1 if the signals are exactly the same, and 0 if the signals are completely uncorrelated. Here, when both the powers P _L ⁽ⁱ⁾ and P _R ⁽ⁱ⁾ of the audio signals of the left and right channels are 0, the correlation signal and the non-correlation signal cannot be extracted for the small band, and the process is performed. Let's move to the next small band processing. Further, when either one of P _L ⁽ⁱ⁾ and P _R ⁽ⁱ⁾ is 0, the calculation cannot be performed in Equation (9), but the normalized correlation coefficient d ⁽ⁱ⁾ = 0 is set, and the smaller Continue processing bandwidth.

Next, using this normalized correlation coefficient d ⁽ⁱ⁾ , conversion coefficients for separating and extracting correlated signals and uncorrelated signals from the left and right channel audio signals are obtained (step S136), and obtained in step S136. Using each transform coefficient, a correlation signal and a non-correlation signal are separated and extracted from the audio signals of the left and right channels (step S137). What is necessary is just to extract both a correlation signal and a non-correlation signal as the estimated audio | voice signal.

A processing example of steps S136 and S137 will be described. Here, as in Patent Document 1, the left and right channel signals are each composed of an uncorrelated signal and a correlated signal, and the same signal is output from the left and right for the correlated signal. The direction of the sound image synthesized from the correlation signals output from the left and right is determined by the balance of the sound pressures on the left and right of the correlation signal. According to the model, the input signals x _L (n), x _R (n) are
x _L (m) = s (m) + n _L (m),
x _R (m) = αs (m) + n _R (m) (13)
It is expressed. Here, s (m) is a left and right correlation signal, and n _L (m) is a signal obtained by subtracting a correlation signal s (m) from an audio signal of the left channel and can be defined as an uncorrelated signal (left channel). , N _R (m) is obtained by subtracting the correlation signal s (m) from the audio signal of the right channel and can be defined as an uncorrelated signal (right channel). Α is a positive real number representing the degree of left / right sound pressure balance of the correlation signal.

From the equation (13), the audio signals x ′ _L (m) and x ′ _R (m) after the window function multiplication described in the equation (2) are expressed by the following equation (14). Here, s ′ (m), n ′ _L (m), and n ′ _R (m) are obtained by multiplying s (m), n _L (m), and n _R (m) by a window function, respectively.
x ′ _L (m) = w (m) {s (m) + n _L (m)} = s ′ (m) + n ′ _L (m),
x ′ _R (m) = w (m) {αs (m) + n _R (m)} = αs ′ (m) + n ′ _R (m)
(14)

The following equation (15) is obtained by subjecting the equation (14) to discrete Fourier transform. However, S (k), N _L (k), and N _R (k) are discrete Fourier transforms of s ′ (m), n ′ _L (m), and n ′ _R (m), respectively.
X _L (k) = S (k) + N _L (k),
X _R (k) = αS (k) + N _R (k) (15)

Therefore, the audio signals X _L ⁽ⁱ⁾ (k) and X _R ⁽ⁱ⁾ (k) in the i-th small band are
X _L ⁽ⁱ⁾ (k) = S ⁽ⁱ⁾ (k) + N _L ⁽ⁱ⁾ (k),
X _R ⁽ⁱ⁾ (k) = α ⁽ⁱ⁾ S ⁽ⁱ⁾ (k) + N _R ⁽ⁱ⁾ (k)
However, K _L ⁽ⁱ⁾ ≦ k ≦ K _U ⁽ⁱ⁾ (16)
It is expressed. Here, α ⁽ⁱ⁾ represents α in the i-th subband. Thereafter, the correlation signal S ⁽ⁱ⁾ (k), the uncorrelated signal N _L ⁽ⁱ⁾ (k), and N _R ⁽ⁱ⁾ (k) in the i-th small band are respectively
S ⁽ⁱ⁾ (k) = S (k),
N _L ⁽ⁱ⁾ (k) = N _L (k),
N _R ⁽ⁱ⁾ (k) = N _R (k)
However, K _L ⁽ⁱ⁾ ≦ k ≦ K _U ⁽ⁱ⁾ (17)
I will leave it.

と表される。ここで、左右の無相関信号の音圧は等しいと仮定している。 From Equation (16), the sound pressures P _L ⁽ⁱ⁾ and P _R ⁽ⁱ⁾ in Equation (12 ⁾ are
P _L ⁽ⁱ⁾ = P _S ⁽ⁱ⁾ + P _N ⁽ⁱ⁾ ,
^{_{P R (i) = [α}} (i)] 2 P S (i) + P N (i) (18)
It is expressed. Here, P _S ⁽ⁱ⁾ and P _N ⁽ⁱ⁾ are the powers of the correlated signal and the uncorrelated signal in the i-th small band, respectively.

It is expressed. Here, it is assumed that the sound pressures of the left and right uncorrelated signals are equal.

と表すことができる。ただし、この算出においてはＳ（ｋ）、Ｎ_Ｌ（ｋ）、Ｎ_Ｒ（ｋ）が互いに直交し、かけ合わされたときの電力は０と仮定している。 Also, from Equations (10) to (12), Equation (9) is

It can be expressed as. However, in this calculation, it is assumed that S (k), N _L (k), and N _R (k) are orthogonal to each other and the power when multiplied is 0.

By solving Equation (18) and Equation (20), the following equation is obtained.

Using these values, a correlated signal and an uncorrelated signal in each small band are estimated. The estimated value est (S ⁽ⁱ⁾ (k)) of the correlation signal S ⁽ⁱ⁾ (k) in the i-th subband is obtained using the parameters μ ₁ and μ ₂ ,
est (S ⁽ⁱ⁾ (k)) = μ ₁ X _L ⁽ⁱ⁾ (k) + μ ₂ X _R ⁽ⁱ⁾ (k) (23)
The estimated error ε is
ε = est (S ⁽ⁱ⁾ (k))-S ⁽ⁱ⁾ (k) (24)
It is expressed. Here, est (A) represents an estimated value of A. And when the square error ε ² is minimized, using the property that ε and X _L ⁽ⁱ⁾ (k), X _R ⁽ⁱ⁾ (k) are orthogonal to each other,
E [ε · X _L ⁽ⁱ⁾ (k)] = 0, E [ε · X _R ⁽ⁱ⁾ (k)] = 0 (25)
This relationship holds. The following simultaneous equations can be derived from Equation (25) by using Equations (16), (19), and (21) to (24).
_{(1-μ 1 -μ 2 α} (i)) P S (i) -μ 1 P N (i) = 0
^{α (i) (1-μ} 1 -μ 2 α (i)) P S (i) -μ 2 P N (i) = 0
(26)

ここで、このようにして求まる推定値est（Ｓ^（ｉ）（ｋ））の電力Ｐ_est（Ｓ） ^（ｉ）が、数式(23）の両辺を二乗して求まる次の式
Ｐ_est（Ｓ） ^（ｉ）＝（μ_１＋α^（ｉ）μ_２）^２Ｐ_Ｓ ^（ｉ）＋（μ_１ ^２＋μ_２ ^２）Ｐ_Ｎ ^（ｉ） (28)
を満たす必要があるため、この式から推定値を次式のようにスケーリングする。なお、est′（Ａ）はＡの推定値をスケーリングしたものを表す。 By solving the equation (26), each parameter is obtained as follows.

Here, the power P _{est (S)} ⁽ⁱ⁾ of the estimated value est (S ⁽ⁱ⁾ (k)) obtained in this way is obtained by squaring both sides of the equation (23), and the following equation P _{est (S ^{) (i) = (μ 1}} + α (i) μ 2) 2 P S (i) + (μ 1 2 + μ 2 2) P N (i) (28)
Therefore, the estimated value is scaled as follows from this equation. Note that est ′ (A) represents a scaled estimate of A.

Then, the estimated values est (N _L ⁽ⁱ⁾ (k)) and est (N _{R for the} uncorrelated signals N _L ⁽ⁱ⁾ (k) and N _R ⁽ⁱ⁾ (k) of the left and right channels in the i-th small band. ^(I) (k))
est (N _L ⁽ⁱ⁾ (k)) = μ ₃ X _L ⁽ⁱ⁾ (k) + μ ₄ X _R ⁽ⁱ⁾ (k) (30)
est (N _R ⁽ⁱ⁾ (k)) = μ ₅ X _L ⁽ⁱ⁾ (k) + μ ₆ X _R ⁽ⁱ⁾ (k) (31)
Thus, in the same manner as the above-described method, the parametric variables μ _{3 to} μ ₆ are

と求めることができる。このようにして求めた推定値est（Ｎ_Ｌ ^（ｉ）（ｋ））、est（Ｎ_Ｒ ^（ｉ）（ｋ））も上述と同様に、次の式によってそれぞれスケーリングする。

It can be asked. The estimated values est (N _L ⁽ⁱ⁾ (k)) and est (N _R ⁽ⁱ⁾ (k)) obtained in this way are also scaled by the following equations, as described above.

The transformation variables obtained in step S136 are the parameters [mu] _{1 to} [mu] ₆ represented by the mathematical expressions (27), (32), and (33) and the scaling coefficients represented by the mathematical expressions (29), (34), and (35). It corresponds to. In step S137, the correlation signal and the uncorrelated signal (the uncorrelated signal of the right channel, the uncorrelated signal of the left channel, and the left channel are estimated by calculation using these transform coefficients (Equations (23), (30), (31)) And uncorrelated signals).

  Next, assignment processing to a virtual sound source is performed (step S138). First, in this allocation process, the direction of the synthesized sound image generated by the correlation signal estimated for each small band is estimated as preprocessing. This estimation process will be described with reference to FIGS. FIG. 15 is a schematic diagram for explaining an example of the positional relationship between the listener, the left and right speakers, and the synthesized sound image, and FIG. 16 is an example of the positional relationship between the speaker group used in the wavefront synthesis reproduction method and the virtual sound source. FIG. 17 is a schematic diagram for explaining an example of the positional relationship between the virtual sound source of FIG. 16, the listener, and the synthesized sound image.

Now, as in the positional relationship 150 shown in FIG. 15, a line drawn from the listener to the midpoint of the left and right speakers 151L and 151R and a line drawn from the listener 153 to the center of one of the speakers 151L / 151R. The spread angle formed is θ ₀ , and the spread angle formed by the line drawn from the listener 153 to the position of the estimated synthesized sound image 152 is θ. Here, when the same audio signal is output from the left and right speakers 151L and 151R while changing the sound pressure balance, the direction of the synthesized sound image 152 generated by the output sound is the following using the parameter α representing the sound pressure balance. It is generally known that the following equation can be approximated (hereinafter referred to as the sign law in stereophonic sound).

  Here, in order to be able to reproduce the 2ch stereo audio signal by the wavefront synthesis reproduction method, the audio signal separation and extraction unit 121 shown in FIG. 12 converts the 2ch signal into a signal of a plurality of channels. For example, when the number of channels after conversion is five, it is regarded as virtual sound sources 162a to 162e in the wavefront synthesis reproduction method as in the positional relationship 160 shown in FIG. 16, and behind the speaker group (speaker array) 161. Deploy. Note that the virtual sound sources 162a to 162e are equally spaced from adjacent virtual sound sources. Therefore, the conversion here converts the audio signal of 2ch into the audio signal of the number of virtual sound sources. As already described, the audio signal separation and extraction unit 121 first separates the 2ch audio signal into one correlation signal and two uncorrelated signals for each small band. In the audio signal separation / extraction unit 121, it is necessary to determine in advance how to allocate these signals to the virtual sound sources of the number of virtual sound sources (here, five virtual sound sources). The assignment method may be user-configurable from a plurality of methods, or may be presented to the user by changing the selectable method according to the number of virtual sound sources.

  As an example of the allocation method, the following method is adopted. First, the left and right uncorrelated signals are assigned to both ends (virtual sound sources 162a and 162e) of the five virtual sound sources, respectively. Next, the synthesized sound image generated by the correlation signal is assigned to two adjacent virtual sound sources out of the five. As for the premise of assigning to two adjacent virtual sound sources, first, it is assumed that the synthesized sound image generated by the correlation signal is inside of both ends (virtual sound sources 162a and 162e) of the five virtual sound sources, that is, 2ch stereo reproduction It is assumed that five virtual sound sources 162a to 162e are arranged so as to fall within a spread angle formed by two speakers at the time. Then, two adjacent virtual sound sources that sandwich the synthesized sound image are determined from the estimated direction of the synthesized sound image, and the allocation of the sound pressure balance to the two virtual sound sources is adjusted, and the two virtual sound sources are synthesized. An allocation method is adopted in which reproduction is performed so as to generate a sound image.

Therefore, as shown in the positional relationship 170 shown in FIG. 17, the spread angle formed by the line drawn from the listener 173 to the midpoint of the virtual sound sources 162a and 162e at both ends and the line drawn from the virtual sound source 162e at the end is θ ′. ₀ , the spread angle formed by the line drawn from the listener 173 to the synthesized sound image 171 is θ ′. Further, a line drawn from the listener 173 at the midpoint between the two virtual sound sources 162c and 162d sandwiching the synthesized sound image 171 and a line drawn from the listener 173 at the midpoint between the virtual sound sources 162a and 162e (from the listener 173). A spread angle formed by a line drawn on the virtual sound source 162c) is φ ₀ , and a spread angle formed by a line drawn from the listener 173 on the synthesized sound image 171 is φ. Here, φ ₀ is a positive real number. A method of assigning the synthesized sound image 152 in FIG. 15 (corresponding to the synthesized sound image 171 in FIG. 17) whose direction has been estimated as described in Expression (36) to a virtual sound source using these variables will be described.

First, scaling by the difference in spread angle is performed as in the following equation.
θ ′ = (θ ′ ₀ / θ ₀ ) θ (37)
Thereby, the difference in the spread angle due to the placement of the virtual sound source is taken into consideration. However, the values of θ ′ ₀ and θ ₀ only need to be adjusted when the audio data reproducing apparatus is installed, and there is no particular problem even if the values of θ ′ ₀ and θ ₀ are not equal. , Θ ₀ = π / 6 [rad], and θ ′ ₀ = π / 4 [rad].

Next, if the direction θ ⁽ⁱ⁾ of the i-th synthesized sound image is estimated by Expression (36), for example, θ ⁽ⁱ⁾ = π / 15 [rad], then θ ′ ⁽ⁱ⁾ from Expression (37 ^). = Π / 10 [rad]. When there are five virtual sound sources, the synthesized sound image 171 is located between the third virtual sound source 162c and the fourth virtual sound source 162d as counted from the left as shown in FIG. Further, when there are five virtual sound sources, φ ₀ ≈0.078 [rad] from θ ′ ₀ = π / 4 [rad] between the third virtual sound source 162c and the fourth virtual sound source 162d. When φ in the i-th small band is φ ⁽ⁱ⁾ , φ ⁽ⁱ⁾ = θ ′ ⁽ⁱ⁾ −φ ₀ ≈0.022π [rad]. In this way, the direction of the synthesized sound image generated by the correlation signal in each small band is represented by a relative angle from the directions of the two virtual sound sources sandwiching the direction. Then, as described above, it is considered that the synthesized sound image is generated by the two virtual sound sources 162c and 162d. For that purpose, the sound pressure balance of the output audio signals from the two virtual sound sources 162c and 162d may be adjusted, and as the adjustment method, the sign law in the stereophonic sound used again as the equation (36) is used.

を満たせばよい。 Here, of the two virtual sound sources 162c and 162d sandwiching the synthesized sound image generated by the correlation signal in the i-th small band, the scaling coefficient for the third virtual sound source 162c is denoted by g ₁ , and the scaling coefficient for the fourth virtual sound source 162d is denoted by When _{g 2,} g ₁ · est from the third virtual sound source ^{162c '(S (i) (} k)), from the fourth virtual source ^{_{162d g 2 · est' (S}} (i) (k)) The audio signal is output. And g ₁ and g ₂ are based on the sign law in stereophonic sound,

Should be satisfied.

On the other hand, when g ₁ and g ₂ are normalized such that the total power from the third virtual sound source 162c and the fourth virtual sound source 162d is equal to the power of the original 2ch stereo correlation signal,
g ₁ ² + g ₂ ² = 1 + [α ⁽ⁱ⁾ ] ² (39)
It becomes.

と求められる。この数式(40)に上述のφ^（ｉ）、φ_０を代入することによって、ｇ_１、ｇ_２を算出する。このようにして算出したスケーリング係数に基づき、上述したように３番目の仮想音源１６２ｃにはｇ_１・est′（Ｓ^（ｉ）（ｋ））の音声信号を、４番目の仮想音源１６２ｄからはｇ_２・est′（Ｓ^（ｉ）（ｋ））の音声信号を割り当てる。そして、これも上述したように、無相関信号は両端の仮想音源１６２ａ，１６２ｅに割り当てられる。すなわち、１番目の仮想音源１６２ａにはest′（Ｎ_Ｌ ^（ｉ）（ｋ））を、５番目の仮想音源１６２ｅにはest′（Ｎ_Ｒ ^（ｉ）（ｋ））を割り当てる。 By bringing these together,

Is required. By substituting the above-mentioned φ ⁽ⁱ⁾ and φ ₀ into this mathematical formula (40), g ₁ and g ₂ are calculated. Based on the scaling coefficient calculated in this way, as described above, the third virtual sound source 162c receives the audio signal of g ₁ · est ′ (S ⁽ⁱ⁾ (k)) from the fourth virtual sound source 162d. The audio signal of g ₂ · est ′ (S ⁽ⁱ⁾ (k)) is assigned. As described above, the uncorrelated signal is assigned to the virtual sound sources 162a and 162e at both ends. In other words, _'the ^{(N L (i) (k} )), the 5 th virtual source 162e _est' est is the first virtual sound source 162a assigns the ^{(N R (i) (k} )).

Unlike this example, if the estimated direction of the synthesized sound image is between the first and second virtual sound sources, g ₁ · est ′ (S ⁽ⁱ⁾ (k) ) And est ′ (N _L ⁽ⁱ⁾ (k)) will be assigned. If the estimated direction of the synthesized sound image is between the fourth and fifth virtual sound sources, the second virtual sound source includes g ₂ · est ′ (S ⁽ⁱ⁾ (k)) and est ′. (N _R ⁽ⁱ⁾ (k)) will be assigned.

As described above, in step S138, the left and right channel correlation signals and uncorrelated signals are assigned to the i-th small band. This is performed for all the small bands by the loop of steps S134a and S134b. As a result, if the number of virtual sound sources is J, output audio signals Y ₁ (k),..., Y _J (k) in the frequency domain for each virtual sound source (output channel) are obtained.

  Then, the processing of steps S140 to S142 is executed for each obtained output channel (steps S139a and S139b). Hereinafter, the processing of steps S140 to S142 will be described.

First, the output speech signal y ′ _j (m) in the time domain is obtained by performing inverse discrete Fourier transform on each output channel (step S140). Here, DFT ⁻¹ represents discrete Fourier inverse transform.
y ′ _j (m) = DFT ⁻¹ (Y _j (k)) (1 ≦ j ≦ J) (41)
Here, as described in Equation (3), the signal subjected to the discrete Fourier transform is a signal after the window function multiplication, and therefore the signal y ′ _j (m) obtained by the inverse transformation is also multiplied by the window function. It is in the state. The window function is a function as shown in Equation (1), and reading is performed while shifting by half segment length. As described above, the window function is added to the output buffer while shifting by half segment length from the beginning of the previous segment. By doing so, the converted data is obtained.

However, in this state, as described above as the prior art, many discontinuous points as indicated by the central portion 101 in FIG. 10 are included in the converted data, and these are perceived as noise during reproduction. As described above, such a discontinuous point is caused by not considering the line spectrum of the DC component. FIG. 18 is a waveform graph schematically showing this. More specifically, FIG. 18 is a diagram for explaining the discontinuity points of the waveform generated at the segment boundary after the inverse discrete Fourier transform when the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored. It is a schematic diagram. In the graph 180 shown in FIG. 18, the horizontal axis represents time. For example, the symbol (M-2) ^(l) indicates that it is the M-2th sample point of the lth segment. The vertical axis of the graph 180 is the value of the output signal for those sample points. As can be seen from the graph 180, a discontinuity occurs in the portion from the end of the l-th segment to the beginning of the (l + 1) -th segment.

  In order to solve the problem described with reference to FIG. 18, the audio signal conversion apparatus according to the present invention is configured as follows. That is, the audio signal conversion apparatus according to the present invention includes a conversion unit, a correlation signal extraction unit, an inverse conversion unit, and a removal unit. The conversion unit performs discrete Fourier transform on the input audio signals of the two channels. The correlation signal extraction unit extracts the correlation signal from the two-channel audio signals after the discrete Fourier transform by the conversion unit while ignoring the DC component. That is, the extraction unit extracts a correlation signal between the input audio signals of the two channels. The inverse transform unit is (a1) for the correlation signal extracted by the correlation signal extraction unit, or (a2) for the correlation signal and the non-correlation signal (signal excluding the correlation signal), or (b1) the Discrete Fourier inverse transform is performed on the audio signal generated from the correlation signal or (b2) the audio signal generated from the correlation signal and the non-correlation signal. In the example here, the inverse transform unit is an example of the audio signal of (b2) above, and the discontinuous points are removed from the audio signal after allocation to the virtual sound source for the wavefront synthesis reproduction method. However, this is not a limitation. For example, discontinuous points with respect to an audio signal before allocation to a virtual sound source, which is an example of the above (a1) or (a2), that is, with respect to an extracted correlation signal or an extracted correlation signal and an uncorrelated signal May be removed and then assigned.

And a removal part removes the discontinuous point of a waveform from the audio | voice signal after discrete Fourier inverse transform in an inverse transformation part. That is, the removing unit removes the discontinuous points of the waveform from the signal after the inverse discrete Fourier transform of the correlation signal or the sound signal generated therefrom.
In the example of the audio signal processing unit 113 in FIG. 12, the above-described conversion unit, correlation signal extraction unit, and inverse conversion unit are included in the audio signal separation / extraction unit 121, and the above-described removal unit is the noise removal unit 122. It can be illustrated.

  With reference to FIG. 19, such processing for solving the problem described with reference to FIG. 18 will be specifically described. FIG. 19 is a schematic diagram for explaining an example of the discontinuous point removal process according to the present invention. When the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored, the discrete Fourier inverse transform is performed. It is a schematic diagram for demonstrating the method of removing the discontinuous point of the waveform which arises in the segment boundary.

In the discontinuous point removal processing according to the present invention, as shown in the graph 190 of FIG. 19 with respect to the graph 180 of FIG. 18, the differential value of the last waveform of the l-th segment and the beginning of the (l + 1) -th segment. Ensure that the differential values match. Specifically, the noise removal unit 122 applies the waveform of the (l + 1) th segment so that the first value of the (l + 1) th segment is maintained so that the slope of the last two points of the lth segment is maintained. Add DC component (bias). As a result, the processed output audio signal y ″ _j (m) is
y ″ _j (m) = y ′ _j (m) + B (42)
It becomes. B is a constant representing a bias, and after the output audio signal of the previous time and the output audio signal of the current process are added by the output buffer, the waveform is determined so as to be continuous as shown in the graph 190 of FIG. .

  As described above, the noise removing unit 122 adds a direct current component to the audio signal after the inverse discrete Fourier transform (correlation signal or the audio signal generated therefrom) so as to maintain the differential value of the waveform at the boundary of the processing segment. Thus, it is preferable to remove discontinuous points. In this example, a negative bias is applied, but naturally a positive bias may be applied in order to match the differential values.

  Thus, according to the present invention, an audio signal for reproducing a multichannel audio signal such as 2ch or 5.1ch by a wavefront synthesis reproduction method without generating noise due to discontinuous points. Can be converted to As a result, it is possible to enjoy the effect of providing sound image localization as intended by the content producer to the listener at any position, which is a feature of the wavefront synthesis reproduction method.

  Further, the speech signal after inverse discrete Fourier transform to be processed by the noise removing unit 122 is subjected to scaling processing in the time domain or the frequency domain with respect to the correlation signal or the correlation signal and the non-correlation signal, as exemplified by each equation. And the audio signal after the scaling processing may be used. That is, the scaling process may be performed on the correlation signal or the non-correlation signal, and the discontinuous points may be removed from the correlation signal or the non-correlation signal after the scaling process.

  A more preferable example of the present invention will be described with reference to FIGS. FIG. 20 is a diagram showing a waveform obtained as a result of converting a certain music content composed of audio signals of left and right channels into five channels by applying the discontinuous point removal processing of FIG. 19, and FIG. It is a figure which shows the waveform of the result of having applied the other discontinuous point removal process which concerns, and having converted the music content same as the music content made into object in FIG. 20 into five channels. That is, FIG. 21 is a diagram for explaining a method of removing the waveform discontinuity generated at the segment boundary after the discrete Fourier inverse transform when the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored. FIG.

  Only the discontinuous point removal processing described with reference to FIG. 19 may accumulate bias components and overflow the amplitude of the waveform. In the music content 200 after conversion illustrated in FIG. 20, among the five channels of audio signals 201 to 205, in particular, accumulation of bias components is often seen in the audio signals 202 and 203 of the second and third channels from the top, It can be seen that the audio signal 203 has overflowed.

Therefore, in the present invention, it is preferable to converge by decreasing the magnitude of the amplitude of the bias component (DC component) to be added as shown in the following equation. Note that “decrease in time” means to decrease in proportion to the elapsed time from the addition time, for example, the elapsed time from the start point of each processing segment or the start point of the discontinuous point.
y ″ _j (m) = y ′ _j (m) + B × ((M−mσ) / M) (43)
However, σ is a parameter for adjusting the degree of the decrease, and is set to 0.5, for example. For the purpose of reduction, both B and σ are positive. Furthermore, when the absolute value of the bias value obtained for addition exceeds a certain value, σ may be dynamically increased or decreased according to the value. The timing to increase or decrease may be in the next processing segment. Not limited to this, the feedback function works if σ corresponding to the proportional constant to be reduced is changed (changed) according to the absolute value of the bias value (the magnitude of the amplitude of the DC component). A similar effect can be obtained. However, these methods do not guarantee that the amplitude of the speech waveform does not overflow.

  Therefore, for example, when the bias value becomes a certain value (predetermined value) or more, a process of not adding the bias term of the second term of the equation (43) may be added as a function of the safety valve. That is, it is preferable that the noise removing unit 122 executes the addition of the DC component (executes the removal of the discontinuous points) only when the amplitude of the DC component obtained for the addition is less than a predetermined value. By adopting this method, the output result output as the music content 200 of FIG. 20 becomes an output result like the music content 210 shown in FIG. 21, and the bias component is not accumulated. In particular, in the audio signals corresponding to the audio signals 202 and 203 of the music content 200, that is, the audio signals 212 and 213 of the second and third channels from the top among the audio signals 211 to 215 of the five channels of the music content 210, It can be seen that the bias component is not accumulated.

  A more preferred example of the present invention will be described with reference to FIGS. FIG. 22 shows the result of converting the music content, which is different from the music content targeted in FIG. 20 and FIG. It is a figure which shows these waveforms. FIG. 23 is a diagram showing waveforms obtained as a result of converting the same music content as the target music content in FIG. 22 into five channels by applying still another discontinuous point removal process according to the present invention. . That is, FIG. 23 shows discontinuous points of the waveform generated at the segment boundary after the discrete Fourier inverse transform when the left and right channel sound signals having a drastic change in the sound signal waveform are subjected to the discrete Fourier transform and the direct current components of the left and right channels are ignored. It is a schematic diagram for demonstrating the method to remove.

  For example, when the audio signal is close to white noise, such as the consonant part of the audio, there are cases where the change in the audio signal waveform is so drastic that the original waveform is already close to discontinuity. If the discontinuous point removal processing of the present invention is applied when converting such left and right channel audio signals into audio signals for the wavefront synthesis reproduction system, the waveform may be distorted. In other words, if the discontinuous point removal processing of the present invention is applied to an audio signal in which the original waveform is close to discontinuity, this processing will force the waveform close to the original discontinuity to be continuous. Therefore, the waveform may be distorted. One example is shown in FIG. In the music content 220 after conversion shown in FIG. 22, the distortion is particularly large at the locations indicated by arrows in the first and fifth audio signals 221 and 225 out of the five channels of audio signals 221 to 225. Perceived.

  In order to solve this problem, it is preferable to employ the following method in the discontinuous point removal processing in the audio signal conversion processing according to the present invention. That is, when the signal is close to white noise, such as the consonant part of the voice, the number of times that the waveform of the input voice signal crosses 0 within a predetermined time (for example, within the processing segment or half thereof) compared to the other parts. Take advantage of extreme increases. In addition, what is necessary is just to decide where to take 0. Therefore, the number of times that the output audio signal (at least the audio signal after the inverse discrete Fourier transform) crosses 0 in the half segment length is counted, and if it is equal to or greater than a certain value (predetermined number), the next And the second term on the right-hand side in Equation (42) or Equation (43) is not added in the next segment processing. That is, the discontinuous point removal process is executed only at other points. The count may be performed for a speech waveform for a certain time regardless of the segment boundary, or may be performed for speech waveforms for a plurality of segment processes. What is necessary is just to determine whether a bias term is added by segment processing.

  By adopting such a method, the locations indicated by arrows in the music content 220, particularly the audio signals 221 and 225, are among the audio signals 231 to 235 of the five channels of the music content 230 after conversion shown in FIG. As indicated by the arrows in the audio signals 231, 235, there is no distortion and no noise is generated.

  The effect of the more preferable discontinuous point removal processing described with reference to FIG. 23 will be described in comparison with FIG. FIG. 24 is a diagram showing waveforms as a result of converting the music content of FIG. 8 into five channels by applying the discontinuous point removal processing of FIG. 23, and FIG. 25 is the music content (after conversion) of FIG. It is the figure which expanded a part of audio | voice signal of one channel among these. When the music content 80 shown in FIG. 8 is an input audio signal by the discontinuous point removal processing (noise removal processing) as described above, the audio signals 241 to 245 of five channels in the music content 240 shown in FIG. Is converted to In particular, the discontinuous points corresponding to FIG. 9 in the audio signal 243 of the third channel from the top in the music content 240 become continuous with the discontinuities being eliminated as shown by the audio signal 250 in FIG. I understand that. In this way, discontinuities can be eliminated and noise can be removed. 24 and 25 have been described as a result of applying the preferred discontinuous point removal process described with reference to FIG. 23, but some processing such as Expression (42) or Expression (43) may be performed somewhat. Although there is a difference, it becomes a continuous audio signal as indicated by the audio signal 250 in the same manner.

  In the above, the audio signal conversion processing according to the present invention has been described with reference to an example in which the input audio signal is a 2ch audio signal. However, it can be applied to other multi-channel audio signals. To do. Here, a 5.1ch input audio signal is taken as an example with reference to FIG. 26, but the present invention can be similarly applied to other multi-channel input audio signals.

  FIG. 26 is a schematic diagram for explaining an example of a positional relationship between a speaker group to be used and a virtual sound source when a 5.1ch audio signal is reproduced by the wavefront synthesis reproduction method. Consider applying the audio signal conversion processing according to the present invention to 5.1ch input audio. In general, 5.1ch speakers are arranged as shown in FIG. 2, and three speakers 21L, 22C, and 21R are arranged in front of the listener. And especially in content such as movies, the so-called center channel at the front center is often used for applications such as human speech. That is, there are not many places where sound pressure control is performed so as to generate a synthesized sound image between the center channel and the left channel or between the center channel and the right channel.

  By utilizing this property, the input audio signals to the 5.1ch front left and right speakers 262a and 262c are converted by this method (audio signal conversion processing according to the present invention) as in the positional relationship 260 shown in FIG. For example, after allocating to the five virtual sound sources 263a to 263e, the audio signal of the center channel (the channel for the center speaker) is added to the middle virtual sound source 263c. In this way, the output audio signal is reproduced as a sound image for the virtual sound source by the speaker array 261 by the wavefront synthesis reproduction method. As for the input audio signals for the left and right channels, speakers 262d and 262e may be installed at the rear as in 5.1ch, and output without any change from there.

  As described above, on the premise that the multi-channel input audio signal is an input audio signal of three or more channels, the present invention relates to any two input audio signals among the multi-channel input audio signals. The audio signal conversion process as described above is performed to generate an audio signal to be reproduced by the wavefront synthesis reproduction method, and the input audio signals of the remaining channels are added to the generated audio signal and output. Good. For this addition, for example, an adder may be provided in the audio output signal generator 123.

  Next, the implementation of the present invention will be briefly described. The present invention can be used for an apparatus accompanied with an image such as a television. Various examples of apparatuses to which the present invention can be applied will be described with reference to FIGS. 27 to 29 are diagrams showing an example of the configuration of a television apparatus provided with the audio data reproducing apparatus of FIG. 11, and FIGS. 30 and 31 are diagrams of a video projection system provided with the audio data reproducing apparatus of FIG. 11, respectively. FIG. 32 is a diagram illustrating a configuration example, FIG. 32 is a diagram illustrating a configuration example of a system including a television board and a television device including the audio data reproducing device of FIG. 11, and FIG. 33 is an audio data reproducing device of FIG. It is a figure which shows the example of a motor vehicle. In any of FIGS. 27 to 33, an example is shown in which eight speakers indicated by LSP1 to LSP8 are arranged as the speaker array, but the number of speakers may be plural.

  The audio signal conversion apparatus according to the present invention and the audio data reproduction apparatus including the same can be used for a television apparatus. The arrangement of these devices in the television device may be determined freely. As in the television device 270 shown in FIG. 27, a speaker group 272 in which the speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line may be provided below the television screen 271. Like the television device 280 shown in FIG. 28, a speaker group 282 in which the speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line may be provided above the television screen 281. As in the television device 290 shown in FIG. 29, a speaker group 292 in which transparent film type speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line may be embedded in the television screen 291.

  Moreover, the audio signal conversion apparatus according to the present invention and the audio data reproduction apparatus including the same can be used in a video projection system. As in the video projection system 300 shown in FIG. 30, the speaker group 302 of the speakers LSP1 to LSP8 may be embedded in the projection screen 301b that projects the video by the video projection device 301a. As in the video projection system shown in FIG. 31, a speaker group 312 in which speakers LSP1 to LSP8 are arranged behind a sound transmission type screen 311b for projecting an image by the video projection device 311a may be arranged. In addition, the audio signal conversion apparatus according to the present invention and the audio data reproduction apparatus including the same can be embedded in a TV stand (TV board). As in a system (home theater system) 320 shown in FIG. 32, a speaker group 322b in which speakers LSP1 to LSP8 are arranged may be embedded in a television stand 322a for mounting the television device 321. Furthermore, the audio signal conversion device according to the present invention and the audio data reproduction device including the same can also be applied to car audio. As in an automobile 330 shown in FIG. 33, a speaker group 332 in which speakers LSP1 to LSP8 are arranged in a curved shape may be embedded in a dashboard inside the vehicle.

  In addition, when the audio signal conversion process according to the present invention is applied to an apparatus or the like described with reference to FIGS. 27 to 33, the listener can perform the conversion process (in the audio signal processing unit 113 in FIGS. 11 and 12). It is also possible to provide a switching unit that switches whether or not to perform processing by a user operation performed by a button operation or a remote controller operation provided in the apparatus main body. When this conversion processing is not performed, the 2ch audio data may be reproduced by arranging the virtual sound source as shown in FIG. Alternatively, reproduction may be performed using only the speakers 341L and 341R at both ends of the array speaker 341 as in the positional relationship 340 shown in FIG. Similarly, 5.1ch audio data may be assigned to three virtual sound sources, or may be reproduced using only one or two speakers at both ends and the middle.

  In addition, as a wavefront synthesis reproduction method applicable in the present invention, any method may be used as long as it includes a speaker array (a plurality of speakers) and outputs a sound image for a virtual sound source from those speakers. In addition to the WFS method described in Patent Document 1, there are various methods such as a method using a preceding sound effect (Haas effect) as a phenomenon related to human sound image perception. Here, the preceding sound effect means that if the same sound is played from multiple sound sources and each sound reaching the listener from each sound source has a small time difference, the sound image is localized in the sound source direction of the sound that has arrived in advance. It points out the effect to do. If this effect is used, a sound image can be perceived at the virtual sound source position. However, it is difficult to clearly perceive the sound image only by the effect. Here, humans also have the property of perceiving a sound image in the direction in which the sound pressure is felt highest. Therefore, in the audio data reproducing apparatus, the above-described effect of the preceding sound and the effect of perceiving the maximum sound pressure direction are combined, so that a sound image can be perceived in the direction of the virtual sound source even with a small number of speakers.

  As described above, the audio signal conversion apparatus according to the present invention has been described on the assumption that the audio signal for the multi-channel method is converted into an audio signal for reproduction by the wavefront synthesis reproduction method. The present invention can be similarly applied to the case of converting to a multi-channel audio signal (the number of channels may be the same or different). The converted audio signal may be an audio signal to be reproduced by a speaker group including at least a plurality of speakers, although the arrangement is not limited. This is because even in the case of such conversion, the DC component may be ignored in order to perform the discrete Fourier transform / inverse transform as described above and obtain a correlation signal. As a method of reproducing the audio signal converted in this way, for example, each of the signals extracted for each virtual sound source is associated with one speaker at a time, and is normally output and reproduced instead of the wavefront synthesis reproduction method. Can be considered. Further, various reproduction methods such as a method of assigning the uncorrelated signals on both sides to different speakers installed on the side and the rear can be considered.

  Further, for example, each component of the audio signal conversion device according to the present invention, such as each component in the audio signal processing unit 113 illustrated in FIG. (Or DSP: Digital Signal Processor), hardware such as a memory, a bus, an interface, and a peripheral device, and software that can be executed on these hardware. Part or all of the hardware can be mounted as an integrated circuit / IC (Integrated Circuit) chip set, and in this case, the software may be stored in the memory. In addition, all the components of the present invention may be configured by hardware, and in that case as well, part or all of the hardware can be mounted as an integrated circuit / IC chip set. .

  In addition, a recording medium on which a program code of software for realizing the functions in the various configuration examples described above is recorded is supplied to a device such as a general-purpose computer serving as an audio signal conversion device, and the microprocessor or DSP in the device is used. The object of the present invention is also achieved by executing the program code. In this case, the software program code itself realizes the functions of the above-described various configuration examples. Even if the program code itself or a recording medium (external recording medium or internal storage device) on which the program code is recorded is used. The present invention can be configured by the control side reading and executing the code. Examples of the external recording medium include various media such as an optical disk such as a CD-ROM or a DVD-ROM and a non-volatile semiconductor memory such as a memory card. Examples of the internal storage device include various devices such as a hard disk and a semiconductor memory. The program code can be downloaded from the Internet and executed, or received from a broadcast wave and executed.

  Although the audio signal conversion apparatus according to the present invention has been described above, as illustrated in the flowchart of the processing flow, the present invention converts a multi-channel input audio signal into an audio signal for reproduction by a speaker group. A form as an audio signal conversion method may also be adopted.

  This audio signal conversion method includes the following conversion step, extraction step, inverse conversion step, and removal step. The conversion step is a step in which the conversion unit performs a discrete Fourier transform on the input audio signals of the two channels. The extraction step is a step in which the correlation signal extraction unit extracts a correlation signal by ignoring a direct current component of the audio signals of the two channels after the discrete Fourier transform in the conversion step. In the inverse conversion step, the inverse conversion unit performs the correlation signal or the correlation signal and the non-correlation signal extracted in the extraction step, the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal. This is a step of performing inverse discrete Fourier transform on the generated audio signal. The removal step is a step in which the removal unit removes the discontinuous points of the waveform from the audio signal after the discrete Fourier inverse transform in the inverse transform step. Other application examples are the same as those described for the audio signal converter, and the description thereof is omitted.

  Note that the program code itself is a program for causing a computer to execute the audio signal conversion method. That is, this program performs a correlation step in which a computer performs a discrete Fourier transform on an input audio signal of two channels and an audio signal of two channels after the discrete Fourier transform in the conversion step, ignoring a direct current component. An extraction step for extracting a correlation signal, a correlation signal or a correlation signal and a non-correlation signal extracted in the extraction step, a voice signal generated from a correlation signal, or a correlation signal and a non-correlation signal This is a program for executing an inverse transform step for performing inverse discrete Fourier transform on an audio signal, and a removing step for removing waveform discontinuities from the audio signal after the discrete Fourier inverse transform in the inverse transform step. .

  DESCRIPTION OF SYMBOLS 110 ... Audio | voice data reproduction apparatus, 111 ... Decoder, 112 ... Audio signal extraction part, 113 ... Audio signal processing part, 114 ... D / A converter, 115 ... Amplifier, 116 ... Speaker, 121 ... Audio signal separation extraction part, 122 ... Noise removal unit, 123... Audio output signal generation unit.

Claims (15)

An audio signal converter for converting a multi-channel input audio signal into an audio signal for reproduction by a group of speakers,
A conversion unit for performing discrete Fourier transform on the input audio signals of the two channels;
A correlation signal extraction unit that extracts a correlation signal by ignoring a direct current component for two-channel audio signals after discrete Fourier transform in the conversion unit;
The correlation signal extracted by the correlation signal extraction unit or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal An inverse transform unit that performs discrete Fourier inverse transform on the audio signal;
A removing unit for removing the discontinuous points of the waveform from the audio signal after the discrete Fourier inverse transform in the inverse transform unit;
An audio signal conversion device comprising:   The said removal part removes the said discontinuous point by adding a DC component to the audio | voice signal after the said discrete Fourier transform so that the differential value of a waveform may be maintained in the boundary of a process segment. Item 2. The audio signal conversion device according to Item 1.   The audio signal conversion apparatus according to claim 2, wherein the removing unit reduces the magnitude of the amplitude of the DC component to be added in proportion to the elapsed time from the addition time.   The audio signal conversion apparatus according to claim 3, wherein the removing unit changes the proportional constant for the reduction according to the magnitude of the amplitude of the DC component obtained for addition.   The said removal part performs addition of the said DC component except the location where the frequency | count that the waveform of the audio | voice signal after said discrete Fourier transform crosses 0 exists more than predetermined times within predetermined time, It is characterized by the above-mentioned. Item 5. The audio signal converter according to Item 4.   The said removal part performs addition of the said DC component only when the amplitude of the said DC component calculated | required in order to add is less than predetermined value, The said any one of Claims 2-5 characterized by the above-mentioned. Audio signal converter.   The removing unit performs the removal of the discontinuous points except where the number of times that the waveform of the audio signal after the discrete Fourier inverse crosses 0 exceeds a predetermined number of times within a predetermined time. The audio | voice signal converter of any one of Claims 1-3.   The audio signal after the inverse discrete Fourier transform to be processed by the removing unit performs a scaling process in the time domain or the frequency domain on the correlated signal or the correlated signal and the uncorrelated signal, and after the scaling process The audio signal conversion apparatus according to any one of claims 1 to 7, wherein the audio signal conversion apparatus according to any one of claims 1 to 7 is used. The multi-channel input audio signal is an input audio signal of three or more channels, and for any two input audio signals of the multi-channel input audio signals, the conversion unit, the correlation signal extraction unit, The discontinuous point is removed by an inverse transform unit and the removal unit, and an audio signal to be reproduced by the speaker group is generated,
9. The audio signal conversion according to claim 1, further comprising: an adder that adds the input audio signals of the remaining channels to the generated audio signal. 10. apparatus.   A digital content input unit that inputs digital content including the multi-channel input audio signal; a decoder unit that decodes the digital content; and an audio signal extraction unit that separates the audio signal from the digital content decoded by the decoder unit; An audio signal processing unit that converts the audio signal extracted by the audio signal extraction unit into a multi-channel audio signal that has three or more channels and is different from the input audio signal, and the audio signal processing unit includes: The audio signal conversion apparatus according to claim 1, further comprising a conversion unit, the correlation signal extraction unit, the inverse conversion unit, and the removal unit.   The digital content input unit inputs digital content from a recording medium storing digital content, a server that distributes digital content via a network, or a broadcasting station that broadcasts digital content. Audio signal converter.   The audio signal converter according to any one of claims 1 to 11, further comprising a switching unit that switches whether to execute processing in the audio signal processing unit according to a user operation. An audio signal conversion method for converting a multi-channel input audio signal into an audio signal for reproduction by a speaker group,
A conversion step in which the conversion unit performs discrete Fourier transform on the input audio signals of the two channels;
An extraction step in which the correlation signal extraction unit extracts a correlation signal by ignoring a direct current component for the audio signals of the two channels after the discrete Fourier transform in the conversion step;
An inverse transform unit for the correlation signal extracted in the extraction step, the correlation signal and the non-correlation signal, the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal; An inverse transform step for performing an inverse discrete Fourier transform on the generated audio signal;
A removing unit for removing discontinuous points of the waveform from the audio signal after discrete Fourier inverse transform in the inverse transform step;
An audio signal conversion method comprising: On the computer,
A transform step for performing a discrete Fourier transform on the input audio signals of the two channels;
An extraction step of ignoring a direct current component and extracting a correlation signal for the audio signals of the two channels after the discrete Fourier transform in the conversion step;
The correlation signal extracted in the extraction step or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the voice signal generated from the correlation signal and the non-correlation signal An inverse transform step for performing an inverse discrete Fourier transform,
A removal step of removing discontinuous points of the waveform from the audio signal after the discrete Fourier inverse transform in the inverse transform step;
A program for running   The computer-readable recording medium which recorded the program of Claim 14.

Images