WO2013176073A1 - Audio signal conversion device, method, program, and recording medium - Google Patents

Abstract

An audio signal conversion device (exemplified by an audio signal processing section (73)) includes: a conversion section that reads out each input audio signal in two channels while shifting by one fourth the length of a processing segment for each read-out, multiplies the read audio signals in the processing segment by the Hann window function, and then applies a discrete Fourier transform; an extraction section that ignores the DC components of the audio signals in the two channels, to which the discrete Fourier transform has been applied, and extracts correlation signals; an inverse conversion section that applies an inverse discrete Fourier transform to the correlation signals extracted by the extraction section or audio signals generated from the correlation signals; and a window function multiplication section (83) that multiplies again the audio signals in the processing segment among the audio signals to which the inverse discrete Fourier transform has been applied, by the Hann window function, shifts the resultant signals by one fourth the length of the processing segment, and adds the signals to the audio signals in the prior processing segment. As a result, this audio signal conversion device can convert audio signals in multiple channels without generating noise caused by discontinuous points.

Description

Audio signal conversion apparatus, method, program, and recording medium

The present invention relates to an audio signal conversion apparatus, method, program, and recording medium for converting an audio signal for a multi-channel playback method.

Conventionally proposed sound reproduction systems include a stereo (2ch) system and a 5.1ch surround system (ITU-R BS.775-1), which are widely used for consumer use. The 2ch system is a system for generating different audio data from the left speaker 11L and the right speaker 11R as schematically illustrated in FIG. The 5.1ch surround system is, as schematically illustrated in FIG. 2, a left front speaker 21L, a right front speaker 21R, a center speaker 22C, a left rear speaker 23L, a right rear speaker 23R disposed between them, This is a method of inputting and outputting different audio data to a subwoofer dedicated to a low sound range (generally 20 Hz to 100 Hz) not shown.

In addition to the 2ch system and 5.1ch surround system, various sound reproduction systems such as 7.1ch, 9.1ch, and 22.2ch have been proposed. In any of the methods described above, each speaker is arranged on a circumference or a spherical surface centered on the listener (listener), and ideally a listening position (listening position) that is equidistant from each speaker, so-called sweet. It is preferable to listen at a spot. For example, it is preferable to listen to the sweet spot 12 in the 2ch system and the sweet spot 24 in the 5.1ch surround system. When listening at the sweet spot, the synthesized sound image based on the balance of sound pressure is localized where the producer intended. Conversely, when listening at a position other than the sweet spot, the sound image / quality is generally deteriorated. Hereinafter, these methods are collectively referred to as a multi-channel reproduction method.

On the other hand, apart from the multi-channel playback method, there is also a sound source object-oriented playback method. This method is a method in which all sounds are sounds emitted by any sound source object, and each sound source object (hereinafter referred to as “virtual sound source”) includes its own position information and audio signal. It is out. Taking music content as an example, each virtual sound source includes the sound of each musical instrument and position information where the musical instrument is arranged.
The sound source object-oriented reproduction method is usually reproduced by a reproduction method (that is, a wavefront synthesis reproduction method) in which a sound wavefront is synthesized by a group of speakers arranged in a straight line or a plane. Among such wavefront synthesis reproduction systems, the Wave Field Synthesis (WFS) system described in Non-Patent Document 1 is one of the practical mounting methods using linearly arranged speaker groups (hereinafter referred to as speaker arrays). Has been actively studied in recent years.

Such a wavefront synthesis reproduction method is different from the above-described multi-channel reproduction method, as shown schematically in FIG. 3, for a listener who is listening at any position in front of the arranged speaker groups 31. However, it has the feature that both good sound image and sound quality can be presented at the same time. That is, the sweet spot 32 in the wavefront synthesis reproduction system is wide as shown in the figure.
In addition, a listener who is listening to sound while facing the speaker array in an acoustic space provided by the WFS method is actually a sound source in which the sound radiated from the speaker array virtually exists behind the speaker array. Feels like being emitted from (virtual sound source).

This wavefront synthesis playback method requires an input signal representing a virtual sound source. In general, one virtual sound source needs to include an audio signal for one channel and position information of the virtual sound source. Taking the above-described music content as an example, for example, it is an audio signal recorded for each musical instrument and position information of the musical instrument. However, the sound signal of each virtual sound source does not necessarily need to be for each musical instrument, but the arrival direction and magnitude of each sound intended by the content creator must be expressed using the concept of virtual sound source. .

Here, since the most widespread method among the aforementioned multi-channel methods is the stereo (2ch) method, the music content of the stereo method will be considered. As shown in FIG. 4, the audio signals of the L (left) channel and the R (right) channel in stereo music contents are installed on the left speaker 41L and on the right using two speakers 41L and 41R, respectively. Playback is performed by the speaker 41R. When such reproduction is performed, as shown in FIG. 4, only when listening at a point equidistant from each of the speakers 41L and 41R, that is, the sweet spot 43, the voice of the vocal and the sound of the bass can be heard from the middle position 42b. The sound image is localized and heard as intended by the producer, such as a piano sound on the left side 42a and a drum sound on the right side 42c.
It is considered that such content is reproduced by the wavefront synthesis reproduction method, and the sound image localization as intended by the content producer is provided to the listener at any position, which is a feature of the wavefront synthesis reproduction method. For this purpose, it is necessary to be able to perceive a sound image when listening in the sweet spot 43 of FIG. 4 from any viewing position, such as the sweet spot 53 shown in FIG. That is, the vocal group and the sound of the bass are heard from the middle position 52b at the wide sweet spot 53 by the speaker group 51 arranged in a straight line or a plane, and the piano sound is heard from the left position 52a and the drum sound. The sound image must be localized and heard as intended by the producer, such as the right position 52c.

For this problem, for example, consider the case where the L channel sound and the R channel sound are arranged as virtual sound sources 62a and 62b as shown in FIG. In this case, since each L / R channel does not represent a single sound source alone but generates a synthesized sound image by two channels, the sweet spot 63 is still generated even if it is reproduced by the wavefront synthesis reproduction method. The sound image is localized as shown in FIG. 4 only at the position of the sweet spot 63. That is, in order to realize such sound image localization, it is necessary to separate 2ch stereo data into sound for each sound image by some means and generate virtual sound source data from each sound.

In response to this problem, the method described in Patent Document 1 separates 2ch stereo data into a correlated signal and an uncorrelated signal based on the correlation coefficient of the signal power for each frequency band. Is generated, a virtual sound source is generated from the results, and a discontinuous point of the waveform generated at that time is removed.

Japanese Patent No. 4810621

However, in the method described in Patent Document 1, it is determined whether or not a signal other than a human voice is a consonant part by counting the number of zero crossings, and a waveform is generated for a part other than the consonant part. A bias value is added so as to be continuous. Here, the musical sound signal or the like is usually a mixture of components that are close to white noise, such as consonant parts, and other components. There are also many parts of human speech that have characteristics like consonants and vowels, such as muddy sounds. In this method, since it is determined whether or not a bias value is added to such an audio signal only by the number of zero crossings, an erroneous determination occurs naturally, and this portion is discontinuous in the generated audio signal waveform. Dots are included and perceived as noise.

The present invention has been made in view of the above-described actual situation, and an object of the present invention is to generate a multichannel audio signal such as 2ch or 5.1ch without generating noise due to discontinuities. An audio signal conversion apparatus, method, program, and recording medium that can be converted are provided.

In order to solve the above-described problems, a first technical means of the present invention is an audio signal conversion apparatus that converts a multi-channel input audio signal into an audio signal for reproduction by a speaker group. Each of the input audio signals of the two channels is read while being shifted by ¼ of the length of the processing segment, the audio signal of the read processing segment is multiplied by a Hann window function, and then a discrete Fourier transform is performed. A correlation signal extraction unit that extracts a correlation signal by ignoring a direct current component of the two-channel audio signals after the discrete Fourier transform by the conversion unit, and the correlation signal extracted by the correlation signal extraction unit or the correlation signal And an uncorrelated signal, an audio signal generated from the correlated signal, or from the correlated signal and the uncorrelated signal The inverse transform unit that performs discrete Fourier inverse transform on the generated speech signal, and the speech signal of the processing segment of the speech signal after the discrete Fourier inverse transform by the inverse transform unit is multiplied by the Hann window function again. And a window function multiplier for adding to the audio signal of the previous processing segment with a shift of ¼ of the length of the processing segment.

According to a second technical means, in the first technical means, the speech signal after the inverse discrete Fourier transform to be processed by the window function multiplication unit is the correlation signal or the correlation signal and the non-correlation signal. The audio signal is subjected to scaling processing in the time domain or the frequency domain.

A third technical means is an audio signal conversion method for converting a multi-channel input audio signal into an audio signal for reproduction by a group of speakers, wherein the conversion unit is configured for each of the input audio signals of two channels. The conversion step of performing the discrete Fourier transform after multiplying the audio signal of the read processing segment by shifting by a quarter of the length of the processing segment and multiplying by the Hann window function, and the correlation signal extracting unit, the conversion step And an extraction step for extracting a correlation signal by ignoring a direct current component of the two-channel audio signals after the discrete Fourier transform, and the inverse transformation unit extracts the correlation signal extracted in the extraction step or the correlation signal and the non-correlation To a signal or to an audio signal generated from the correlated signal, or to the correlated signal and the uncorrelated The inverse transform step for performing discrete Fourier inverse transform on the speech signal generated from the signal, and the window function multiplication unit for the speech signal of the processing segment among the speech signals after the discrete Fourier inverse transform in the inverse transform step A window function multiplying step of multiplying the Hann window function again, shifting by 1/4 of the length of the processing segment, and adding to the audio signal of the previous processing segment.

According to a fourth technical means, the computer reads out each of the input audio signals of the two channels while shifting each 1/4 of the length of the processing segment, and multiplies the read audio signal of the processing segment by the Hann window function. Then, a conversion step for performing a discrete Fourier transform, an extraction step for ignoring a direct current component and extracting a correlation signal for the two-channel audio signals after the discrete Fourier transform in the conversion step, and an extraction step for extracting the correlation signal Discrete Fourier for the correlated signal or for the correlated signal and the uncorrelated signal, for the speech signal generated from the correlated signal, or for the speech signal generated from the correlated signal and the uncorrelated signal An inverse transform step for performing an inverse transform, and a processing section of the audio signal after the discrete Fourier inverse transform at the inverse transform step. A window function multiplying step of multiplying the audio signal of the transmission segment by the Hann window function again, shifting by a quarter of the length of the processing segment, and adding to the audio signal of the previous processing segment. It is.

The fifth technical means is a computer-readable recording medium in which the program in the fourth technical means is recorded.

According to the present invention, it is possible to convert a multi-channel audio signal such as 2ch or 5.1ch without generating noise due to discontinuous points.

It is a schematic diagram for demonstrating a 2ch system. It is a schematic diagram for demonstrating a 5.1ch surround system. It is a schematic diagram for demonstrating a wavefront synthetic | combination reproduction | regeneration system. It is a schematic diagram which shows a mode that the music content by which the sound of the vocal, the bass, the piano, and the drum was recorded by the stereo system is reproduced using two right and left speakers. FIG. 5 is a schematic diagram showing an ideal sweet spot when the music content of FIG. 4 is reproduced by the wavefront synthesis reproduction method. FIG. 5 is a schematic diagram showing a state of an actual sweet spot when the audio signal of the left / right channel in the music content of FIG. 4 is reproduced by the wavefront synthesis reproduction method with virtual sound sources set at the positions of the left / right speakers, respectively. . It is a block diagram which shows one structural example of the audio | voice data reproduction apparatus provided with the audio | voice signal converter concerning this invention. FIG. 8 is a block diagram illustrating a configuration example of an audio signal processing unit (an audio signal conversion device according to the present invention) in the audio data reproduction device of FIG. 7. It is a flowchart for demonstrating an example of the audio | voice signal process in the audio | voice signal processing part of FIG. It is a figure which shows a mode that audio | voice data are stored in a buffer in the audio | voice signal processing part of FIG. It is a figure which shows a Hann window function. It is a figure which shows the window function multiplied once per 1/4 segment in the first window function multiplication process in the audio | voice signal process of FIG. It is a schematic diagram for demonstrating the example of the positional relationship of a listener, a right-and-left speaker, and a synthesized sound image. It is a schematic diagram for demonstrating the example of the positional relationship of the speaker group and virtual sound source which are used with a wavefront synthetic | combination reproduction | regeneration system. It is a schematic diagram for demonstrating the example of the positional relationship of the virtual sound source of FIG. 14, a listener, and a synthesized sound image. FIG. 6 is a schematic diagram for explaining waveform discontinuities occurring at segment boundaries after inverse discrete Fourier transform when the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored. It is a figure which shows the segment after discrete Fourier inverse transform which performs the discontinuous point removal process which concerns on this invention. It is a figure which shows an example of the graph of the waveform of the input audio | voice signal. It is a figure which shows an example of the graph of the waveform after performing the 1st multiplication process by the Hann window function with respect to the audio | voice signal of FIG. It is a figure which shows an example of the graph of the waveform which performed discrete Fourier inverse transform with respect to the audio | voice signal of FIG. It is a figure which shows an example of the graph of the waveform after performing the 2nd multiplication process by a Hann window function with respect to the audio | voice signal of FIG. It is a schematic diagram for demonstrating the process in case a shift width is 1/2 and a window function calculation is performed only once. It is a schematic diagram for demonstrating the process in case a shift width is 1/4 and a window function calculation is performed twice. It is a schematic diagram for demonstrating the example of the positional relationship of the speaker group to be used, and a virtual sound source, when reproducing | regenerating a 5.1ch audio | voice signal by a wave-front synthetic | combination reproduction | regeneration system. It is a figure which shows the structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the television apparatus provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the structural example of the video projection system provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the other structural example of the video projection system provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the structural example of the system which consists of a television board provided with the audio | voice data reproduction | regeneration apparatus of FIG. 7, and a television apparatus. It is a figure which shows the example of the motor vehicle provided with the audio | voice data reproduction | regeneration apparatus of FIG. It is a figure which shows the example of the speaker of reproduction | regeneration object in the audio | voice data reproduction | regeneration apparatus of FIG.

An audio signal conversion apparatus according to the present invention is an apparatus for converting an audio signal for a multi-channel playback system into an audio signal for playback on a speaker group having the same or different number of channels, an audio signal for a wavefront synthesis playback system, or the like. Therefore, it can also be called an audio signal processing device, an audio data conversion device, etc., and can be incorporated into an audio data reproducing device. Of course, the audio signal is not limited to a signal in which a so-called audio is recorded, and can also be called an acoustic signal. The wavefront synthesis reproduction method is a reproduction method in which a wavefront of sound is synthesized by a group of speakers arranged in a straight line or a plane as described above.

Hereinafter, a configuration example and a processing example of an audio signal conversion device according to the present invention will be described with reference to the drawings. In the following description, first, an example in which the audio signal conversion apparatus according to the present invention generates an audio signal for the wavefront synthesis reproduction method by conversion will be given.
FIG. 7 is a block diagram showing an example of the configuration of an audio data reproducing apparatus including the audio signal converting apparatus according to the present invention. FIG. 8 is an audio signal processing unit (according to the present invention) in the audio data reproducing apparatus of FIG. It is a block diagram which shows one structural example of an audio | voice signal converter.

7 includes a decoder 71, an audio signal extraction unit 72, an audio signal processing unit 73, a D / A converter 74, an amplifier group 75, and a speaker group 76. The decoder 71 decodes the content of only audio or video with audio, converts it into a signal processable format, and outputs it to the audio signal extraction unit 72. The content is acquired by downloading from the Internet from a digital broadcast content transmitted from a broadcasting station, a server that distributes digital content via a network, or reading from a recording medium such as an external storage device. Thus, although not shown in FIG. 7, the audio data reproducing device 70 includes a digital content input unit that inputs digital content including a multi-channel input audio signal. The decoder 71 decodes the digital content input here. The audio signal extraction unit 72 separates and extracts an audio signal from the obtained signal. Here, it is a 2ch stereo signal. The signals for the two channels are output to the audio signal processing unit 73.

The audio signal processing unit 73 generates multi-channel audio signals (which will be described as signals corresponding to the number of virtual sound sources in the following example) from three or more channels and different from the input audio signal from the obtained two-channel signals. . That is, the input audio signal is converted into another multi-channel audio signal. The audio signal processing unit 73 outputs the audio signal to the D / A converter 74. The number of virtual sound sources can be determined in advance if there is a certain number or more, but the amount of calculation increases as the number of virtual sound sources increases. Therefore, it is desirable to determine the number in consideration of the performance of the mounted device. In this example, the number is assumed to be 5.

The D / A converter 74 converts the obtained signals into analog signals and outputs each signal to the amplifier 75. Each amplifier 75 amplifies the input analog signal, transmits it to each speaker 76, and outputs it from each speaker 76 as sound.

FIG. 8 shows the detailed configuration of the audio signal processing unit in this figure. The audio signal processing unit 73 includes a window function multiplication unit 81, an audio signal separation / extraction unit 82, a window function multiplication unit 83, and an audio output signal generation unit 84.

The window function multiplication unit 81 reads out the 2-channel audio signal, multiplies it by the Hann window function, and outputs it to the audio signal separation / extraction unit 82. The audio signal separation / extraction unit 82 generates an audio signal corresponding to each virtual sound source from the two-channel signals, and outputs it to the window function multiplication unit 83. The window function multiplication unit 83 removes a perceptual noise part from the obtained audio signal waveform, and outputs the audio signal after noise removal to the audio output signal generation unit 84. Thus, the window function multiplication unit 83 functions as a noise removal unit. The audio output signal generation unit 84 generates each output audio signal waveform corresponding to each speaker from the obtained audio signal. The audio output signal generation unit 84 performs processing such as wavefront synthesis reproduction processing, for example, assigns the obtained audio signal for each virtual sound source to each speaker, and generates an audio signal for each speaker. A part of the wavefront synthesis reproduction processing may be performed by the audio signal separation / extraction unit 82.

Next, an example of audio signal processing in the window function multiplication unit 81, the audio signal separation / extraction unit 82, and the window function multiplication unit 83 will be described with reference to FIG. FIG. 9 is a flowchart for explaining an example of the audio signal processing in the audio signal processing unit in FIG. 8, and FIG. 10 is a diagram showing a state in which audio data is stored in the buffer in the audio signal processing unit in FIG. . FIG. 11 is a diagram showing the Hann window function, and FIG. 12 is a diagram showing one 1 per 1/4 segment in the first window function multiplication process (window function multiplication process in the window function multiplication unit 81) in the audio signal processing of FIG. It is a figure which shows the window function multiplied by times.

First, the window function multiplying unit 81 reads out audio data of 1/4 length of one segment from the extraction result in the audio signal extracting unit 72 in FIG. 7 (step S1). Here, the audio data refers to a discrete audio signal waveform sampled at a sampling frequency such as 48 kHz. A segment is an audio data section composed of a group of sample points having a certain length. Here, the segment refers to a section length to be subjected to discrete Fourier transform later, and is also called a processing segment. For example, the value is 1024. In this example, 256 points of audio data that is ¼ of one segment are to be read.

The read 256-point audio data is stored in the buffer 100 as illustrated in FIG. This buffer can hold the sound signal waveform for the immediately preceding segment, and the past segments are discarded. The immediately previous 3/4 segment data (768 points) and the latest 1/4 segment data (256 points) are connected to create audio data for one segment, and the process proceeds to window function calculation (step S2). That is, all sample data is read four times in the window function calculation.

Next, the window function multiplication unit 81 executes a window function calculation process for multiplying the audio data for one segment by the next Hann window that has been conventionally proposed (step S2). This Hann window is illustrated as the window function 110 in FIG.

Here, m is a natural number, M is an even number of one segment length. If the stereo input signals are x _L (m) and x _R (m), respectively, the audio signals x ′ _L (m) and x ′ _R (m) after the window function multiplication are

x ′ _L (m) = w (m) × _L (m)
x ′ _R (m) = w (m) × _R (m) (2)
Is calculated. Using this Hann window, for example, the input signal x _L (m ₀ ) at the sample point m ₀ (where 0 ≦ m ₀ <M / 4) is multiplied by sin ² ((m ₀ / M) π). . In the next reading, the same sample point is read as m ₀ + M / 4, then as m ₀ + M / 2, and then as m ₀ + (3M) / 4. Further, although details will be described later, this window function is calculated again at the end. Accordingly, the above input signal x _L (m ₀ ) is multiplied by sin ⁴ ((m ₀ / M) π). If this is illustrated as a window function, a window function 120 shown in FIG. 12 is obtained. Since this window function 120 is added a total of four times while being shifted every quarter segment,

Will be multiplied. If this expression is transformed, the value becomes 3/2 (a constant value). If no modification is made, the read signal is multiplied by the Hann window twice, and the reciprocal of 2/2 of the above 3/2 is obtained. By multiplying / 3 and shifting it by 1/4 segment (or shifting by 1/4 segment and applying 2/3 after the addition), the original signal is completely restored.

The audio data thus obtained is subjected to discrete Fourier transform as in the following formula (3) to obtain frequency domain audio data (step S3). The processing in steps S3 to S10 may be performed by the audio signal separation / extraction unit 82. Here, DFT represents discrete Fourier transform, k is a natural number, and 0 ≦ k <M. X _L (k) and X _R (k) are complex numbers.
X _L (k) = DFT (x ′ _L (n))
X _R (k) = DFT (x ′ _R (n)) (3)

Next, the processing of steps S5 to S8 is performed for each line spectrum on the obtained frequency domain audio data (steps S4a and S4b). Specific processing will be described. Here, an example of performing processing such as obtaining a correlation coefficient for each line spectrum will be described. However, as described in Patent Document 1, a band (small size) divided using Equivalent Rectangular Band (ERB) is used. Processing such as obtaining a correlation coefficient may be executed for each (band).

Here, the line spectrum after the discrete Fourier transform is symmetrical with respect to M / 2 (where M is an even number) except for the DC component, that is, for example, X _L (0). That is, X _L (k) and X _L (Mk) have a complex conjugate relationship in the range of 0 <k <M / 2. Therefore, in the following, the range of k ≦ M / 2 is considered as the object of analysis, and the range of k> M / 2 is treated the same as a symmetric line spectrum having a complex conjugate relationship.

Next, the correlation coefficient is acquired by calculating | requiring the normalized correlation coefficient of a left channel and a right channel with following Formula with respect to each line spectrum (step S5).

This normalized correlation coefficient d ⁽ⁱ⁾ represents how much the audio signals of the left and right channels are correlated, and takes a real value between 0 and 1. 1 if the signals are exactly the same, and 0 if the signals are completely uncorrelated. Here, when both the powers P _L ⁽ⁱ⁾ and P _R ⁽ⁱ⁾ of the audio signals of the left and right channels are 0, it is impossible to extract the correlated signal and the uncorrelated signal with respect to the line spectrum, and the processing is performed. Let's move on to the processing of the next line spectrum. Also, if either P _L ⁽ⁱ⁾ or P _R ⁽ⁱ⁾ is 0, the calculation cannot be performed in Equation (4), but the normalized correlation coefficient d ⁽ⁱ⁾ = 0 and the line Continue processing the spectrum.

Next, using this normalized correlation coefficient d ⁽ⁱ⁾ , a conversion coefficient for separating and extracting the correlation signal and the non-correlation signal from the audio signals of the left and right channels is obtained (step S6), and obtained in step S6. Using each conversion coefficient, the correlation signal and the non-correlation signal are separated and extracted from the audio signals of the left and right channels (step S7). What is necessary is just to extract both a correlation signal and a non-correlation signal as the estimated audio | voice signal.

A processing example of steps S6 and S7 will be described. Here, as in Patent Document 1, each signal of the left and right channels is composed of an uncorrelated signal and a correlated signal, and for the correlated signal, a signal waveform that differs only in gain from the left and right (that is, a signal waveform composed of the same frequency component) is output. Adopt the model to be done. Here, the gain corresponds to the amplitude of the signal waveform and is a value related to the sound pressure. In this model, it is assumed that the direction of the sound image synthesized by the correlation signals output from the left and right is determined by the balance of the left and right sound pressures of the correlation signal. According to the model, the input signals x _L (n), x _R (n) are
x _L (m) = s (m) + n _L (m),
x _R (m) = αs (m) + n _R (m) (8)
It is expressed. Here, s (m) is a left and right correlation signal, n _L (m) is a subtracted correlation signal s (m) from a left channel audio signal, and can be defined as an uncorrelated signal (left channel). , N _R (m) is obtained by subtracting the correlation signal s (m) multiplied by α from the right channel audio signal, and can be defined as an uncorrelated signal (right channel). Α is a positive real number representing the degree of left / right sound pressure balance of the correlation signal.

From Equation (8), the audio signals x ′ _L (m) and x ′ _R (m) after the window function multiplication described in Equation (2) are expressed by the following Equation (9). However, s ′ (m), n ′ _L (m), and n ′ _R (m) are obtained by multiplying s (m), n _L (m), and n _R (m) by a window function, respectively.
x ′ _L (m) = w (m) {s (m) + n _L (m)} = s ′ (m) + n ′ _L (m),
x ′ _R (m) = w (m) {αs (m) + n _R (m)} = αs ′ (m) + n ′ _R (m) (9)

The following equation (10) is obtained by performing a discrete Fourier transform on the equation (9). However, S (k), N _L (k), and N _R (k) are discrete Fourier transforms of s ′ (m), n ′ _L (m), and n ′ _R (m), respectively.
X _L (k) = S (k) + N _L (k),
X _R (k) = αS (k) + N _R (k) (10)

Therefore, the audio signals X _L ⁽ⁱ⁾ (k) and X _R ⁽ⁱ⁾ (k) in the i-th line spectrum are
X _L ⁽ⁱ⁾ (k) = S ⁽ⁱ⁾ (k) + N _L ⁽ⁱ⁾ (k),
X _R ⁽ⁱ⁾ (k) = α ⁽ⁱ⁾ S ⁽ⁱ⁾ (k) + N _R ⁽ⁱ⁾ (k) (11) Here, α ⁽ⁱ⁾ represents α in the i-th line spectrum. Thereafter, the correlation signal S ⁽ⁱ⁾ (k), the non-correlation signal N _L ⁽ⁱ⁾ (k), and N _R ⁽ⁱ⁾ (k) in the i-th line spectrum, respectively,
S ⁽ⁱ⁾ (k) = S (k),
N _L ⁽ⁱ⁾ (k) = N _L (k),
N _R ⁽ⁱ⁾ (k) = N _R (k) (12)
I will leave it.

From Equation (11), the sound pressures P _L ⁽ⁱ⁾ and P _R ⁽ⁱ⁾ in Equation (7 ⁾ are
P _L ⁽ⁱ⁾ = P _S ⁽ⁱ⁾ + P _N ⁽ⁱ⁾
P _R ⁽ⁱ⁾ = [α ⁽ⁱ⁾ ] ² P _S ⁽ⁱ⁾ + P _N ⁽ⁱ⁾ (13)
It is expressed. Here, P _S ⁽ⁱ⁾ and P _N ⁽ⁱ⁾ are the powers of the correlated signal and the uncorrelated signal in the i-th line spectrum, respectively.

It is expressed. Here, it is assumed that the sound pressures of the left and right uncorrelated signals are equal.

From Equations (5) to (7), Equation (4) is

It can be expressed as. However, in this calculation, it is assumed that S (k), N _L (k), and N _R (k) are orthogonal to each other and the power when multiplied is 0.

By solving Equation (13) and Equation (15), the following equation is obtained.

Using these values, a correlation signal and a non-correlation signal in each line spectrum are estimated. Estimate the estimated value est (S ⁽ⁱ⁾ (k)) of the correlation signal S ⁽ⁱ⁾ (k) in the i-th line spectrum using the parameters μ ₁ and μ ₂ ,
est (S ⁽ⁱ⁾ (k)) = μ ₁ X _L ⁽ⁱ⁾ (k) + μ ₂ X _R ⁽ⁱ⁾ (k) (18)
The estimated error ε is
ε = est (S ⁽ⁱ⁾ (k)) − S ⁽ⁱ⁾ (k) (19)
It is expressed. Here, est (A) represents an estimated value of A. And when the square error ε ² is minimized, using the property that ε and X _L ⁽ⁱ⁾ (k) and X _R ⁽ⁱ⁾ (k) are orthogonal to each other,
E [ε · X _L ⁽ⁱ⁾ (k)] = 0, E [ε · X _R ⁽ⁱ⁾ (k)] = 0 (20)
This relationship holds. The following simultaneous equations can be derived from Equation (20) by using Equations (11), (14), and (16) to (19).
(1-μ ₁ −μ ₂ α ⁽ⁱ⁾ ) P _S ⁽ⁱ⁾ −μ ₁ P _N ⁽ⁱ⁾ = 0
α ⁽ⁱ⁾ (1-μ ₁ −μ ₂ α ⁽ⁱ⁾ ) P _S ⁽ⁱ⁾ −μ ₂ P _N ⁽ⁱ⁾ = 0
(twenty one)

By solving the equation (21), each parameter is obtained as follows.

Here, the power P _{est (S)} ⁽ⁱ⁾ of the estimated value est (S ⁽ⁱ⁾ (k)) obtained in this way is obtained by squaring both sides of the equation (18), and the following equation P _{est (S )} ⁽ⁱ⁾ = (μ ₁ + α ⁽ⁱ⁾ μ ₂ ) ² P _S ⁽ⁱ⁾ + (μ ₁ ² + μ ₂ ² ) P _N ⁽ⁱ⁾ (23)
Therefore, the estimated value is scaled as follows from this equation. Note that est ′ (A) represents a scaled estimate of A.

The estimated values est (N _L ⁽ⁱ⁾ (k)) and est (N _R ) for the left and right channel uncorrelated signals N _L ⁽ⁱ⁾ (k) and N _R ⁽ⁱ⁾ (k) in the i-th line spectrum. ⁽ⁱ⁾ (k))
est (N _L ⁽ⁱ⁾ (k)) = μ ₃ X _L ⁽ⁱ⁾ (k) + μ ₄ X _R ⁽ⁱ⁾ (k) (25)
est (N _R ⁽ⁱ⁾ (k)) = μ ₅ X _L ⁽ⁱ⁾ (k) + μ ₆ X _R ⁽ⁱ⁾ (k) (26)
Thus, in the same way as the above calculation, the parameters μ ₃ to μ ₆ are

It can be asked. The estimated values est (N _L ⁽ⁱ⁾ (k)) and est (N _R ⁽ⁱ⁾ (k)) obtained in this way are also scaled by the following equations, as described above.

The respective transformation variables μ ₁ to μ ₆ represented by the mathematical expressions (22), (27), and (28) and the scaling coefficients represented by the mathematical expressions (24), (29), and (30) are converted coefficients obtained in step S6. It corresponds to. In step S7, the correlation signal and the non-correlated signal (the uncorrelated signal of the right channel, the uncorrelated signal of the left channel, and the left channel) are estimated by calculation using these transform coefficients (formulas (18), (25), (26)). And uncorrelated signals).

Next, the assignment process to the virtual sound source is performed (step S8). First, in this allocation process, as a pre-process, the direction of the synthesized sound image generated by the correlation signal estimated for each line spectrum is estimated. This estimation process will be described with reference to FIGS. FIG. 13 is a schematic diagram for explaining an example of the positional relationship between the listener, the left and right speakers, and the synthesized sound image, and FIG. 14 shows an example of the positional relationship between the speaker group used in the wavefront synthesis reproduction method and the virtual sound source. FIG. 15 is a schematic diagram for explaining an example of the positional relationship between the virtual sound source of FIG. 14, the listener, and the synthesized sound image.

Now, as in the positional relationship 130 shown in FIG. 13, a line drawn from the listener to the middle point of the left and right speakers 131L and 131R and a line drawn from the listener 133 to the center of one of the speakers 131L / 131R. The spread angle formed is θ ₀ , and the spread angle formed by the line drawn from the listener 133 to the position of the estimated synthesized sound image 132 is θ. Here, when the same audio signal is output from the left and right speakers 131L and 131R while changing the sound pressure balance, the direction of the synthesized sound image 132 generated by the output sound is the following using the parameter α representing the sound pressure balance. It is generally known that the following equation can be approximated (hereinafter referred to as the sign law in stereophonic sound).

Here, in order to be able to reproduce the 2ch stereo audio signal by the wavefront synthesis reproduction method, the audio signal separation and extraction unit 82 shown in FIG. 8 converts the 2ch signal into a signal of a plurality of channels. For example, when the number of channels after conversion is five, it is regarded as virtual sound sources 142a to 142e in the wavefront synthesis reproduction system as shown in the positional relationship 140 shown in FIG. 14, and behind the speaker group (speaker array) 141. Deploy. Note that the virtual sound sources 142a to 142e are equally spaced from adjacent virtual sound sources. Therefore, the conversion here converts the audio signal of 2ch into the audio signal of the number of virtual sound sources. As already described, the audio signal separation / extraction unit 82 first separates the 2ch audio signal into one correlation signal and two uncorrelated signals for each line spectrum. In the audio signal separation / extraction unit 82, it is necessary to determine in advance how to allocate these signals to the virtual sound sources of the number of virtual sound sources (here, five virtual sound sources). The assignment method may be user-configurable from a plurality of methods, or may be presented to the user by changing the selectable method according to the number of virtual sound sources.

The following method is adopted as an example of the allocation method. First, the left and right uncorrelated signals are assigned to both ends ( virtual sound sources 142a and 142e) of the five virtual sound sources, respectively. Next, the synthesized sound image generated by the correlation signal is assigned to two adjacent virtual sound sources out of the five. Regarding which two virtual sound sources are adjacent to each other, first, as a premise, the synthesized sound image generated by the correlation signal is assumed to be inside both ends ( virtual sound sources 142a and 142e) of the five virtual sound sources, that is, 2ch stereo reproduction. Assume that five virtual sound sources 142a to 142e are arranged so as to fall within a spread angle formed by two speakers at the time. Then, two adjacent virtual sound sources that sandwich the synthesized sound image are determined from the estimated direction of the synthesized sound image, and the allocation of the sound pressure balance to the two virtual sound sources is adjusted, and the two virtual sound sources are synthesized. An allocation method is adopted in which reproduction is performed so as to generate a sound image.

Therefore, as in the positional relationship 150 shown in FIG. 15, the spread angle formed by the line drawn from the listener 153 to the midpoint of the virtual sound sources 142a and 142e at both ends and the line drawn from the virtual sound source 142e at the end is θ _0. A spread angle formed by a line drawn from the listener 153 to the synthesized sound image 151 is defined as θ. Further, a line drawn from the listener 153 at the midpoint between the two virtual sound sources 142c and 142d sandwiching the synthesized sound image 151 and a line drawn from the listener 153 at the midpoint between the virtual sound sources 142a and 142e at both ends (from the listener 153). A spread angle formed by a line drawn on the virtual sound source 142c) is φ ₀ , and a spread angle formed by a line drawn from the listener 153 on the synthesized sound image 151 is φ. Here, φ ₀ is a positive real number. A method of assigning the synthesized sound image 132 in FIG. 13 (corresponding to the synthesized sound image 151 in FIG. 15) whose direction has been estimated as described in Equation (31) to the virtual sound source using these variables will be described.

First, it is assumed that the direction θ ⁽ⁱ⁾ of the i-th synthesized sound image is estimated by Expression (31), and for example, θ ⁽ⁱ⁾ = π / 15 [rad]. When there are five virtual sound sources, the synthesized sound image 151 is positioned between the third virtual sound source 142c and the fourth virtual sound source 142d as counted from the left as shown in FIG. When there are five virtual sound sources, φ ₀ ≈0.11 [rad] is obtained by simple geometric calculation using a trigonometric function between the third virtual sound source 142c and the fourth virtual sound source 142d. When φ in the i-th line spectrum is φ ⁽ⁱ⁾ , φ ⁽ⁱ⁾ = θ ⁽ⁱ⁾ −φ ₀ ≈0.088 [rad]. In this way, the direction of the synthesized sound image generated by the correlation signal in each line spectrum is represented by a relative angle from the directions of the two virtual sound sources sandwiching the synthetic sound image. Then, as described above, it is considered that the synthesized sound image is generated by the two virtual sound sources 142c and 142d. For that purpose, the sound pressure balance of the output audio signals from the two virtual sound sources 142c and 142d may be adjusted, and as the adjustment method, the sign law in the stereophonic sound used again as Equation (31) is used.

Here, of the two virtual sound sources 142c and 142d sandwiching the synthesized sound image generated by the correlation signal in the i-th line spectrum, the scaling coefficient for the third virtual sound source 142c is g ₁ , and the scaling coefficient for the fourth virtual sound source 142d is When g _2, g ₁ · est from the third virtual sound source ^{142c '(S (i) (} k)), from the fourth virtual source _{142d g 2 · est' (S} (i) (k)) The audio signal is output. And g ₁ and g ₂ are due to the sign law in stereophonic sound,

Should be satisfied.

On the other hand, when g ₁ and g ₂ are normalized so that the total power from the third virtual sound source 142c and the fourth virtual sound source 142d is equal to the power of the original 2ch stereo correlation signal,
g ₁ ² + g ₂ ² = 1 + [α ⁽ⁱ⁾ ] ² (33)
It becomes.

By bringing these together,

Is required. By substituting the aforementioned φ ⁽ⁱ⁾ and φ ₀ into the mathematical formula (34), g ₁ and g ₂ are calculated. Based on the scaling coefficient thus calculated, the audio signal of g ₁ · est ′ (S ⁽ⁱ⁾ (k)) is transmitted from the fourth virtual sound source 142d to the third virtual sound source 142c as described above. The audio signal of g ₂ · est ′ (S ⁽ⁱ⁾ (k)) is assigned. As described above, the uncorrelated signal is assigned to the virtual sound sources 142a and 142e at both ends. In other words, 'the ^{_{(N L (i) (k}} )), the 5 th virtual source 142e est' est is the first virtual sound source 142a assigns the ^{_{(N R (i) (k}} )).

Unlike this example, if the estimated direction of the synthesized sound image is between the first and second virtual sound sources, g ₁ · est ′ (S ⁽ⁱ⁾ (k) ) And est ′ (N _L ⁽ⁱ⁾ (k)) will be assigned. Also, if the estimated direction of the synthesized sound image is between the fourth and fifth virtual sound sources, the second virtual sound source has g ₂ · est ′ (S ⁽ⁱ⁾ (k)) and est ′. Both (N _R ⁽ⁱ⁾ (k)) will be assigned.

As described above, the left and right channel correlation signals and uncorrelated signals are assigned to the i-th line spectrum in step S8. This is performed for all line spectra by the loop of steps S4a and S4b. For example, if 256 discrete Fourier transforms are performed, the first to 127th line spectrum up to 512 points. If 512 discrete Fourier transforms are performed, all the segment points up to 1st to 255th line spectrum (1024 points). When the discrete Fourier transform is performed for, the first to 511th line spectra are obtained. As a result, if the number of virtual sound sources is J, output audio signals Y ₁ (k),..., Y _J (k) in the frequency domain for each virtual sound source (output channel) are obtained.

Then, the processing of steps S10 to S12 is executed for each obtained output channel (steps S9a and S9b). Hereinafter, the processing of steps S10 to S12 will be described.

First, the output speech signal y ′ _J (m) in the time domain is obtained by performing inverse discrete Fourier transform on each output channel (step S10). Here, DFT ⁻¹ represents discrete Fourier inverse transform.
y ′ _J (m) = DFT ⁻¹ (Y _J (k)) (1 ≦ j ≦ J) (35)
Here, as described in Equation (3), since the signal subjected to the discrete Fourier transform is a signal after the window function multiplication, the signal y ′ _J (m) obtained by the inverse transformation is also multiplied by the window function. It is in the state. The window function is a function as shown in Formula (1), and reading is performed while shifting by a ¼ segment length. Therefore, as described above, the window function is shifted by a ¼ segment length from the head of the previous processed segment. The converted data is obtained by adding to the output buffer.

However, in this state, as described above as the prior art, many discontinuous points are included in the converted data, and these are perceived as noise during reproduction. As described above, such a discontinuous point is caused by not considering the line spectrum of the DC component. FIG. 16 is a waveform graph schematically showing this. More specifically, FIG. 16 is a diagram for explaining the discontinuity points of the waveform generated at the segment boundary after the inverse discrete Fourier transform when the left and right channel audio signals are discrete Fourier transformed and the left and right channel DC components are ignored. It is a schematic diagram. In the graph 160 shown in FIG. 16, the horizontal axis represents time. For example, the symbol (0) ^(l) indicates the first sample point of the l-th segment, and (M−1) ⁽ The symbol ^l) indicates the Mth sample point of the lth segment. The vertical axis of the graph 160 is the value of the output signal for those sample points. As can be seen from the graph 160, a discontinuous point occurs in the portion from the end of the (l-1) th segment to the beginning of the lth segment.

In order to solve the problem described with reference to FIG. 16, the audio signal converter according to the present invention is configured as follows. That is, the audio signal conversion apparatus according to the present invention includes a conversion unit, a correlation signal extraction unit, an inverse conversion unit, and a window function multiplication unit. The conversion unit reads out each of the input audio signals of the two channels while shifting by ¼ of the length of the processing segment, multiplies the read audio signal of the processing segment by a Hann window function, and then performs a discrete Fourier transform. Apply. The correlation signal extraction unit extracts the correlation signal from the two-channel audio signals after the discrete Fourier transform by the conversion unit while ignoring the DC component. That is, the correlation signal extraction unit extracts the correlation signal of the input audio signals of the two channels.

The inverse transform unit is (a1) for the correlation signal extracted by the correlation signal extraction unit, or (a2) for the correlation signal and the non-correlation signal (signal excluding the correlation signal), or (b1) the Discrete Fourier inverse transform is performed on the audio signal generated from the correlation signal or (b2) the audio signal generated from the correlation signal and the non-correlation signal. In the example here, the inverse transform unit is an example of the audio signal of (b2) above, and the discontinuous points are removed from the audio signal after allocation to the virtual sound source for the wavefront synthesis reproduction method. However, it is not limited to this. For example, discontinuous points with respect to an audio signal before allocation to a virtual sound source, which is an example of the above (a1) or (a2), that is, with respect to an extracted correlation signal or an extracted correlation signal and an uncorrelated signal May be removed and then assigned.

The window function multiplying unit again multiplies the audio signal of the processing segment by the Hann window function among the audio signals after the inverse discrete Fourier transform by the inverse transform unit (that is, the correlation signal or the audio signal generated therefrom). Then, the waveform is shifted by ¼ of the length of the processing segment and added to the audio signal of the previous processing segment, whereby the discontinuous point of the waveform is removed from the audio signal after the inverse discrete Fourier transform by the inverse transform unit. Here, the previous processing segment refers to the previous processing segment, which is actually shifted by ¼, and refers to the previous, second, and third previous processing segments.

In the example of the audio signal processing unit 73 in FIG. 8, the conversion unit described above is included in the window function multiplication unit 81 and the audio signal separation extraction unit 82, and the correlation signal extraction unit and the inverse conversion unit described above are included in the audio signal separation extraction unit 82. The window function multiplication unit described above can be exemplified by the window function multiplication unit 83.

Referring to FIGS. 17 to 21 as well, such discontinuous point removal processing for solving the problem described in FIG. 16 will be specifically described. FIG. 17 is a diagram showing segments after discrete Fourier inverse transform to which discontinuous point removal processing according to the present invention is performed. FIG. 18 is a diagram illustrating an example of a waveform graph of the input audio signal, and FIG. 19 is a waveform graph after the first multiplication process by the Hann window function is performed on the audio signal of FIG. FIG. 20 is a diagram illustrating an example of a waveform graph obtained by performing inverse discrete Fourier transform on the audio signal of FIG. 19, and FIG. 21 is a Hann window function for the audio signal of FIG. It is a figure which shows an example of the graph of the waveform after performing the 2nd multiplication process.

In the discontinuous point removal processing in the present invention, the average value of the first value of the waveform of the segment (processing segment) 170 after the inverse discrete Fourier transform as shown in FIG. Subtract from the value. As described above, this is because the Hann window is calculated before performing the discrete Fourier transform. That is, since the values of both end points of the Hann window are 0, if no spectral component value is changed after the discrete Fourier transform and the inverse discrete Fourier transform is performed again, the end points of the segment become 0, There is no discontinuity between them. However, in actuality, in the frequency domain after the discrete Fourier transform, each spectral component is changed as described above. Therefore, both end points of the segment after the inverse discrete Fourier transform are not 0, and discontinuous points between the segments are generated.

Therefore, in order to set the both end points to 0, the Hann window is calculated again as described above. Regarding how the multiplication process by the second Hann window function works, the process from the first input speech signal until the multiplication process by the second Hann window function is performed is based on the simplified input waveform. explain. First, assuming that an audio signal waveform such as a graph 180 shown in FIG. 18 is inputted, an audio signal waveform like a graph 190 shown in FIG. 19 is generated by the first Hann window function calculation, and the steps of FIG. The process proceeds to the discrete Fourier transform process of S3. Then, as a result of the arithmetic processing, it is assumed that the waveform of the audio signal as a result of the discrete Fourier inverse transform processing in step S10 in FIG. 9 is shifted from 0 at both end points as in the graph 200 shown in FIG.

In this state, both end points become discontinuous points and are perceived as noise. Therefore, when the Hann window function is calculated again as in step S11 in FIG. 9, both end points are obtained as shown in a graph 210 in FIG. Is a waveform of an audio signal that is guaranteed to be zero. Therefore, the second Hann window function multiplication process ensures that no discontinuity occurs. Thus, the waveform of the processing segment 170 in FIG. 17 does not have a discontinuous point as in the graph 160 in FIG. 16, and a portion that is a discontinuous point in the graph 160 (segment boundary portion) has a value of 0. It becomes continuous and the slope (differential value) also coincides.

After that, as described above, the processing segment after the second Hann window function multiplication process is multiplied by 2/3, which is the inverse of 3/2, and this is multiplied by the audio signal of the previous processing segment (actually The original waveform can be completely restored if it is added to the audio signals of the previous, second, and third processing segments. Actually, at this point, the waveform of the audio signal of the previous three processing segments can be completely restored. In this way, if the processing segments after the second Hann window function multiplication process multiplied by 2/3 are added while being shifted by 1/4 segment, the original waveform can be completely restored. Alternatively, the processing segment after the second Hann window function multiplication processing is added while shifting by 1/4 segment, and the processing segment (the processing segment before the above three) that has been completely added is multiplied by 2/3. In this case, the original signal is completely restored. Of course, the process of multiplying 2/3 does not have to be executed, but only the amplitude is increased.

Next, with reference to FIG. 22 and FIG. 23, the effect of removing discontinuous points according to the present invention will be schematically described by taking a simple sine wave as an example. FIG. 22 is a schematic diagram for explaining the processing in the case where the shift width is 1/2 segment and the window function calculation is performed only once, and FIG. 23 is the processing of the present invention (the shift width is 1/4 segment and the window It is a schematic diagram for demonstrating the process in the case of performing a function calculation twice.

As shown in FIG. 22, when the input waveform 221 is subjected to window function calculation once with a shift width of ½ segment and subjected to discrete Fourier transform, speech signal separation extraction, and inverse discrete Fourier transform, As in the graph 200 of FIG. 20, the segment waveforms 222 and 223 of FIG. Since the output waveform 224 is obtained by adding the segment waveforms 222 and 223, discontinuous points 224a and 224b are generated.

On the other hand, in the processing of the present invention, as shown in FIG. 23, the first window function calculation is performed on the input waveform 231 that is the same as the input waveform 221 with a shift width of 1/4 segment, and the discrete Fourier transform is performed. After performing speech signal separation and extraction and inverse discrete Fourier transform, a second window function calculation is performed. Thus, each segment waveform 232, 233, 234, 235 obtained by performing the second window function operation on each segment waveform after the inverse discrete Fourier transform has both ends as shown in the graph 210 of FIG. Always 0. Here, assuming that the segment waveform 235 is the current processing segment, the segment waveforms 234, 233, and 232 correspond to the waveforms of the previous 1, 2, and 3 processing segments, respectively. Since the output waveform 236 is obtained by adding the segment waveforms 232, 233, 234, and 235, no discontinuity occurs in the output waveform 236 even by the addition.

As described above, in the discontinuous point removal processing according to the present invention, the discontinuous points of the waveform are removed from the speech signal after the inverse discrete Fourier transform by performing the second Hann window function multiplication processing. Therefore, according to the present invention, an audio signal for a multi-channel system such as 2ch or 5.1ch is converted into an audio signal for reproduction by a wavefront synthesis reproduction system without generating noise due to discontinuous points. It becomes possible to do. In particular, according to the present invention, unlike the method described in Patent Document 1, a musical sound signal in which a component close to white noise and a component other than the white noise, such as a consonant portion, are mixed, or a consonant among human voices. Noise caused by discontinuities is not generated even for voice signals such as muddy sounds having characteristics between the vowels and vowels.

According to the present invention, since it can be converted into an audio signal to be reproduced by the wavefront synthesis reproduction method without generating noise, it is possible for the listener at any position which is a feature of the wavefront synthesis reproduction method. Can also enjoy the effect of providing sound image localization as intended by the content creator.

Further, the speech signal after inverse discrete Fourier transform to be processed by the window function multiplication unit 83 is scaled in the time domain or the frequency domain with respect to the correlation signal or the correlation signal and the non-correlation signal as illustrated in each equation. It is good also as an audio | voice signal after processing and the scaling process. That is, scaling processing may be performed on the correlation signal or non-correlation signal, and the discontinuous points may be removed by multiplying the correlation signal or non-correlation signal after the scaling processing by the Hann window function.

In the above, the audio signal conversion processing according to the present invention has been described with reference to an example in which the input audio signal is a 2ch audio signal. However, it can be applied to other multi-channel audio signals. To do. Here, a 5.1ch input audio signal will be described as an example with reference to FIG. 24, but the present invention can be similarly applied to other multi-channel input audio signals.

FIG. 24 is a schematic diagram for explaining an example of the positional relationship between a speaker group to be used and a virtual sound source when a 5.1ch audio signal is reproduced by the wavefront synthesis reproduction method. Consider applying the audio signal conversion processing according to the present invention to 5.1ch input audio. In general, 5.1ch speakers are arranged as shown in FIG. 2, and three speakers 21L, 22C, and 21R are arranged in front of the listener. And especially in content such as movies, the so-called center channel at the front center is often used for applications such as human speech. That is, there are not many places where sound pressure control is performed so as to generate a synthesized sound image between the center channel and the left channel or between the center channel and the right channel.

Using this property, the input audio signals to the 5.1ch front left and right speakers 242a and 242c are converted by this method (audio signal conversion processing according to the present invention) as in the positional relationship 240 shown in FIG. For example, after allocating to five virtual sound sources 243a to 243e, an audio signal of the center channel (center speaker channel) is added to the middle virtual sound source 243c. In this way, the output audio signal is reproduced as a sound image for the virtual sound source by the speaker array 241 by the wavefront synthesis reproduction method. As for the input audio signals for the left and right channels, speakers 242d and 242e may be installed behind 5.1ch and output without any change from there.

As described above, on the premise that the multi-channel input audio signal is an input audio signal of three or more channels, the present invention relates to any two input audio signals among the multi-channel input audio signals. The audio signal conversion process as described above is performed to generate an audio signal to be reproduced by the wavefront synthesis reproduction method, and the input audio signals of the remaining channels are added to the generated audio signal and output. Good. For this addition, for example, an adder may be provided in the audio output signal generator 84.

Next, the implementation of the present invention will be briefly described. The present invention can be used for an apparatus accompanied with an image such as a television. Various examples of apparatuses to which the present invention can be applied will be described with reference to FIGS. FIGS. 25 to 27 are diagrams showing examples of the configuration of the television apparatus provided with the audio data reproducing apparatus of FIG. 7, and FIGS. 28 and 29 are diagrams of the video projection system provided with the audio data reproducing apparatus of FIG. 7, respectively. FIG. 30 is a diagram illustrating a configuration example, FIG. 30 is a diagram illustrating a configuration example of a system including a television board and a television device including the audio data reproduction device of FIG. 7, and FIG. 31 is provided with the audio data reproduction device of FIG. It is a figure which shows the example of a motor vehicle. In any of FIGS. 25 to 31, an example is shown in which eight speakers indicated by LSP1 to LSP8 are arranged as the speaker array, but the number of speakers may be plural.

The audio signal conversion apparatus according to the present invention and the audio data reproduction apparatus including the same can be used for a television apparatus. The arrangement of these devices in the television device may be determined freely. Like the television device 250 shown in FIG. 25, a speaker group 252 in which the speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line may be provided below the television screen 251. Like the television device 260 shown in FIG. 26, a speaker group 262 in which the speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line may be provided above the television screen 261. As in the television device 270 shown in FIG. 27, the television screen 271 may be embedded with a speaker group 272 in which transparent film type speakers LSP1 to LSP8 in the audio data reproducing device are arranged in a straight line.

Also, the audio signal conversion device according to the present invention and the audio data reproduction device provided with the same can be used in a video projection system. Like the video projection system 280 shown in FIG. 28, the speaker group 282 of the speakers LSP1 to LSP8 may be embedded in the projection screen 281b for projecting the video by the video projection device 281a. Like the video projection system 290 shown in FIG. 29, a speaker group 292 in which the speakers LSP1 to LSP8 are arranged behind the sound transmission type screen 291b for projecting an image by the video projection device 291a may be arranged. In addition, the audio signal conversion apparatus according to the present invention and the audio data reproduction apparatus including the same can be embedded in a TV stand (TV board). As in a system (home theater system) 300 shown in FIG. 30, a speaker group 302b in which speakers LSP1 to LSP8 are arranged may be embedded in a TV stand 302a on which the TV device 301 is mounted. Furthermore, the audio signal conversion device according to the present invention and the audio data reproduction device including the same can also be applied to car audio. As in an automobile 310 shown in FIG. 31, a speaker group 312 in which speakers LSP1 to LSP8 are arranged in a curved line may be embedded in a dashboard inside the vehicle.

In addition, when the audio signal conversion process according to the present invention is applied to the apparatus described with reference to FIGS. 25 to 31, the listener can perform the conversion process (in the audio signal processing unit 73 in FIGS. 7 and 8). It is also possible to provide a switching unit that switches whether or not to perform processing by a user operation performed by a button operation or a remote controller operation provided in the apparatus main body. When this conversion processing is not performed, the 2ch audio data may be reproduced by arranging the virtual sound source as shown in FIG. Or you may reproduce | regenerate using only the speakers 321L and 321R of the both ends of the array speaker 321 like the positional relationship 320 shown in FIG. Similarly, 5.1ch audio data may be assigned to three virtual sound sources, or may be reproduced using only one or two speakers at both ends and the middle.

In addition, as a wavefront synthesis reproduction method applicable in the present invention, any method may be used as long as it includes a speaker array (a plurality of speakers) and outputs a sound image for a virtual sound source from those speakers. In addition to the WFS method described in Patent Document 1, there are various methods such as a method using a preceding sound effect (Haas effect) as a phenomenon related to human sound image perception. Here, the preceding sound effect means that if the same sound is played from multiple sound sources and each sound reaching the listener from each sound source has a small time difference, the sound image is localized in the sound source direction of the sound that has arrived in advance. It points out the effect to do. If this effect is used, a sound image can be perceived at the virtual sound source position. However, it is difficult to clearly perceive the sound image only by the effect. Here, humans also have the property of perceiving a sound image in the direction in which the sound pressure is felt highest. Therefore, in the audio data reproducing apparatus, the above-described effect of the preceding sound and the effect of perceiving the maximum sound pressure direction are combined, so that a sound image can be perceived in the direction of the virtual sound source even with a small number of speakers.

As described above, the audio signal conversion apparatus according to the present invention has been described on the assumption that the audio signal for the multi-channel method is converted into an audio signal for reproduction by the wavefront synthesis reproduction method. The present invention can be similarly applied to the case of converting to a multi-channel audio signal (the number of channels may be the same or different). The converted audio signal may be an audio signal to be reproduced by a speaker group including at least a plurality of speakers, although the arrangement is not limited. This is because even in the case of such conversion, the DC component may be ignored in order to perform the discrete Fourier transform / inverse transform as described above and obtain a correlation signal. As a method of reproducing the audio signal converted in this way, for example, each of the signals extracted for each virtual sound source is associated with one speaker at a time, and is normally output and reproduced instead of the wavefront synthesis reproduction method. Can be considered. Further, various reproduction methods such as a method of assigning the uncorrelated signals on both sides to different speakers installed on the side and the rear can be considered.

Further, for example, each component of the audio signal conversion apparatus according to the present invention such as each component in the audio signal processing unit 73 illustrated in FIG. (Or DSP: Digital Signal Processor), hardware such as a memory, a bus, an interface, and a peripheral device, and software that can be executed on these hardware. Part or all of the hardware can be mounted as an integrated circuit / IC (Integrated Circuit) chip set, and in this case, the software may be stored in the memory. In addition, all the components of the present invention may be configured by hardware, and in that case as well, part or all of the hardware can be mounted as an integrated circuit / IC chip set. .

In addition, a recording medium on which a program code of software for realizing the functions in the various configuration examples described above is recorded is supplied to a device such as a general-purpose computer serving as an audio signal conversion device, and the microprocessor or DSP in the device is used. The object of the present invention is also achieved by executing the program code. In this case, the software program code itself realizes the functions of the above-described various configuration examples. Even if the program code itself or a recording medium (external recording medium or internal storage device) on which the program code is recorded is used. The present invention can be configured by the control side reading and executing the code. Examples of the external recording medium include various media such as an optical disk such as a CD-ROM or a DVD-ROM and a nonvolatile semiconductor memory such as a memory card. Examples of the internal storage device include various devices such as a hard disk and a semiconductor memory. The program code can be downloaded from the Internet and executed, or received from a broadcast wave and executed.

Although the audio signal conversion apparatus according to the present invention has been described above, as illustrated in the flowchart of the processing flow, the present invention converts a multi-channel input audio signal into an audio signal for reproduction by a speaker group. A form as an audio signal conversion method may also be adopted.

This audio signal conversion method has the following conversion step, extraction step, inverse conversion step, and window function multiplication step. In the conversion step, after the conversion unit reads each of the input audio signals of the two channels while shifting by ¼ of the length of the processing segment, the audio signal of the read processing segment is multiplied by the Hann window function. This is a step of performing discrete Fourier transform. The extraction step is a step in which the correlation signal extraction unit extracts a correlation signal by ignoring a direct current component of the audio signals of the two channels after the discrete Fourier transform in the conversion step. In the inverse conversion step, the inverse conversion unit performs the correlation signal or the correlation signal and the non-correlation signal extracted in the extraction step, the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal. This is a step of performing inverse discrete Fourier transform on the generated audio signal. In the window function multiplication step, the window function multiplication unit again multiplies the audio signal of the processing segment among the audio signals after the inverse discrete Fourier transform in the inverse conversion step by the Hann window function, and 1 / of the length of the processing segment. This is a step of shifting by 4 and adding to the audio signal of the previous processing segment. Other application examples are the same as those described for the audio signal converter, and the description thereof is omitted.

The program code itself is a program for causing a computer to execute this audio signal conversion method. In other words, this program reads out each of the input audio signals of the two channels from the computer while shifting each 1/4 of the length of the processing segment, and multiplies the read audio signal of the processing segment by the Hann window function. Thereafter, a transform step for performing discrete Fourier transform, an extraction step for extracting a correlation signal by ignoring a direct current component of the two-channel audio signals after the discrete Fourier transform in the transform step, and a correlation signal extracted in the extraction step Alternatively, an inverse transform step for performing discrete Fourier inverse transform on the correlated signal and the uncorrelated signal, or on the sound signal generated from the correlated signal, or on the sound signal generated from the correlated signal and the uncorrelated signal. And the sound of the processing segment of the audio signal after the discrete Fourier inverse transform in the inverse transform step. Signal to a program for causing again multiplied by the Hann window function, shifted by 1/4 of the length of the processing segments, it is executed and the window function multiplier step of adding to the audio signal before the processing segment, a. Other application examples are the same as those described for the audio signal converter, and the description thereof is omitted.

DESCRIPTION OF SYMBOLS 70 ... Audio | voice data reproduction apparatus, 71 ... Decoder, 72 ... Audio signal extraction part, 73 ... Audio signal processing part, 74 ... D / A converter, 75 ... Amplifier, 76 ... Speaker, 81 ... Window function multiplication part, 82 ... Audio | voice Signal separation and extraction unit, 83... Window function multiplication unit, 84.

Claims (5)

1. An audio signal converter for converting a multi-channel input audio signal into an audio signal for reproduction by a group of speakers,
   Each of the input audio signals of the two channels is read while being shifted by ¼ of the length of the processing segment, and the conversion unit that performs discrete Fourier transform after multiplying the read audio signal of the processing segment by the Hann window function When,
   A correlation signal extraction unit that extracts a correlation signal by ignoring a direct current component for two-channel audio signals after discrete Fourier transform in the conversion unit;
   The correlation signal extracted by the correlation signal extraction unit or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal An inverse transform unit that performs discrete Fourier inverse transform on the audio signal;
   The audio signal of the processing segment among the audio signals after the discrete Fourier inverse transform in the inverse transform unit is multiplied by the Hann window function again, and shifted by 1/4 of the length of the processing segment, so that the audio of the previous processing segment is obtained. A window function multiplier for adding to the signal;
   An audio signal conversion device comprising:
2. The audio signal after the inverse discrete Fourier transform to be processed by the window function multiplication unit is subjected to scaling processing in the time domain or the frequency domain with respect to the correlation signal or the correlation signal and the non-correlation signal. The audio signal conversion apparatus according to claim 1, wherein the audio signal conversion apparatus is an audio signal.
3. An audio signal conversion method for converting a multi-channel input audio signal into an audio signal for reproduction by a speaker group,
   The conversion unit reads out each of the input audio signals of the two channels while shifting by ¼ of the length of the processing segment, and multiplies the read audio signal of the processing segment by the Hann window function, and then performs a discrete Fourier transform. A conversion step for applying
   An extraction step in which the correlation signal extraction unit extracts a correlation signal by ignoring a direct current component for the audio signals of the two channels after the discrete Fourier transform in the conversion step;
   An inverse transform unit for the correlation signal extracted in the extraction step, the correlation signal and the non-correlation signal, the voice signal generated from the correlation signal, or the correlation signal and the non-correlation signal; An inverse transform step for performing an inverse discrete Fourier transform on the generated audio signal;
   The window function multiplication unit again multiplies the audio signal of the processing segment among the audio signals after the discrete Fourier inverse transform in the inverse transformation step, and shifts by 1/4 of the length of the processing segment, A window function multiplication step to add to the audio signal of the previous processing segment;
   An audio signal conversion method comprising:
4. On the computer,
   A conversion step in which each of the input audio signals of the two channels is read while being shifted by ¼ of the length of the processing segment, the audio signal of the read processing segment is multiplied by the Hann window function, and then a discrete Fourier transform is performed. When,
   An extraction step of ignoring a direct current component and extracting a correlation signal for the audio signals of the two channels after the discrete Fourier transform in the conversion step;
   The correlation signal extracted in the extraction step or the correlation signal and the non-correlation signal, or the voice signal generated from the correlation signal, or the voice signal generated from the correlation signal and the non-correlation signal An inverse transform step for performing an inverse discrete Fourier transform,
   Of the audio signals after the discrete Fourier inverse transform in the inverse transform step, the audio signal of the processing segment is again multiplied by the Hann window function and shifted by ¼ of the length of the processing segment to obtain the audio of the previous processing segment. And a window function multiplication step for adding to the signal.
5. A computer-readable recording medium on which the program according to claim 4 is recorded.

Images