JP6161962B2 - Audio signal reproduction apparatus and method - Google Patents

Description

  The present invention relates to an audio signal reproduction apparatus and method for reproducing an audio signal.

  Conventionally proposed sound reproduction methods include multi-channel reproduction such as stereo (2ch) method, 5.1ch surround method (ITU-R BS.775-1), 7.1ch, 9.1ch, 22.2ch, etc. And a sound source object-oriented reproduction method. The latter method is a method in which all sounds are sounds emitted by any sound source object, and each sound source object (hereinafter referred to as “virtual sound source”) has its own position information and audio signal. Contains. Taking music content as an example, each virtual sound source includes the sound of each musical instrument and position information where the musical instrument is arranged.

  The sound source object-oriented reproduction method is usually reproduced by a reproduction method (that is, a wavefront synthesis reproduction method) in which a sound wavefront is synthesized by a group of speakers arranged in a straight line or a plane. Among such wavefront synthesis reproduction systems, the Wave Field Synthesis (WFS) system described in Non-Patent Document 1 is a practical mounting method using a speaker group (hereinafter referred to as a speaker array) arranged linearly or on a curve. Recently, it has been actively researched as one of these.

  Such a wavefront synthesis playback system, unlike the above-mentioned multi-channel playback system with a narrow sweet spot, provides a good sound image and sound quality for listeners listening at any position in front of the arranged speaker groups. It has the feature that both can be presented simultaneously. That is, the sweet spot in the wavefront synthesis reproduction method is wide.

  Patent Document 1 proposes a playback method that converts 5.1ch audio signals, which are normally used for movie contents, etc., to audio signals of a plurality of channels and reproduces them using a wavefront synthesis playback method. Specifically, focusing on the left front channel signal and the right front channel signal among the left front channel signal, right front channel signal, center channel signal, left rear channel signal, right rear channel signal, and subwoofer channel signal. By separating the correlation signal component and the non-correlation signal component, assigning the correlation signal component to a plurality of virtual sound sources, and then superimposing the center channel signal on the central virtual sound source, wavefront synthesis of the 5.1ch audio signal Playback is in the playback mode.

  By the way, conventionally, various technologies have been proposed in acoustic systems and audio software that change visual display in conjunction with music to increase the immersive feeling. For example, there are a method of displaying the absolute value of the sound pressure power as a bar graph for each certain frequency band, a method of displaying colors and figures at random according to temporal changes in sound, and the like.

  In addition, a method of visualizing the spatial sound image position by linking the display of the sound pressure level with the sound image position has been proposed (see, for example, Patent Document 2). In the method described in Patent Document 2, the sound image position is determined only by the absolute value of the level difference between the left and right audio signals. Therefore, if the left and right audio signals differ only in amplitude, or if the amplitudes are also the same, the perceptual sound image position matches the display position.

Japanese Patent No. 4810621 Japanese Utility Model Publication No. 60-177600

AJ Berkhout, D. de Vries, and P. Vogel, “Acoustic control by wave field synthesis”, J. Acoust. Soc. Am. Volume 93 (5), United States, Acoustical Society of America, May 1993, pp. 2764- 2778

  However, in the method described in Patent Document 2, since the sound image position is determined based only on the absolute value of the level difference between the left and right audio signals, information indicating whether the sound image is closer to the left or right is not displayed.

  Also, in normal music content, vocals mixed with different sound image positions and sounds of each instrument are mixed, and the sound image differs for each component sound. In the method, since the sound image position is displayed as a representative value, the sound image position does not match between hearing and perception.

  Furthermore, when different audio signals are input to the left and right, a sound image should not be generated for perception, but in the method described in Patent Document 2, the position of the sound image is determined only by the absolute value of the level difference between the left and right audio signals. Therefore, a sound image is displayed at any location, and the sound image position does not match between hearing and perception.

  The present invention has been made in view of the above situation, and an object of the present invention is to provide an audio signal reproduction device and an audio signal capable of displaying a sound image position so that the sound image position matches between hearing and perception. It is to provide a reproduction method.

  In order to solve the above-mentioned problem, a first technical means of the present invention is an audio signal reproduction device, comprising two audio signals, a correlation signal and a non-correlation signal for each frequency or frequency band. A separation unit that separates the correlation signals, a calculation unit that calculates a sound image direction and a signal power value for each correlation signal separated by the separation unit, and a plurality of predetermined sound images based on the sound image direction for each correlation signal An adder that assigns the signal power value to one of the direction groups and obtains an added value of the signal power value for each sound image direction group, and each sound image direction at a position corresponding to each of the plurality of sound image direction groups And a sound image display unit for displaying information indicating the added value of the group.

  According to a second technical means of the present invention, in the first technical means, the sound image display unit displays the addition value for each sound image direction group in a bar graph format at a position corresponding to each of the plurality of sound image direction groups. The information to be displayed is displayed.

  According to a third technical means of the present invention, in the first technical means, the sound image display unit includes a plurality of light emitting units, and the emission color of the light emitting unit at a position corresponding to each of the plurality of sound image direction groups. Is changed according to information indicating the added value for each sound image direction group.

  According to a fourth technical means of the present invention, in the first technical means, the sound image display unit includes a plurality of light emitting units, and the light emission intensity of the light emitting unit at a position corresponding to each of the plurality of sound image direction groups. Is changed according to information indicating the added value for each sound image direction group.

  According to a fifth technical means of the present invention, there is provided an audio signal reproducing method, wherein the separation unit separates the two audio signals into a correlated signal and a non-correlated signal for each frequency or frequency band. And a calculation unit that calculates a sound image direction and a signal power value for each correlation signal separated in the separation step, and an addition unit is predetermined based on the sound image direction for each correlation signal. An addition step of assigning the signal power value to one of a plurality of sound image direction groups and obtaining an addition value of the signal power values for each sound image direction group; and a sound image display unit, And a sound image display step for displaying information indicating the added value for each sound image direction group at a position corresponding to the sound image direction group.

  According to the present invention, in the audio signal reproduction device, it is possible to display the sound image position so that the sound image position matches between hearing and perception.

It is a block diagram which shows the example of 1 structure of the audio | voice signal reproduction | regeneration apparatus based on this invention. FIG. 2 is a block diagram illustrating a configuration example of an audio signal processing unit in the audio signal reproduction device of FIG. 1. It is a flowchart for demonstrating an example of the separation extraction process in the separation extraction part in the audio | voice signal processing part of FIG. It is a schematic diagram for demonstrating an example of the positional relationship of a listener, right and left speakers, and a synthesized sound image. It is a schematic diagram for demonstrating an example of the positional relationship of the speaker for reproduction | regeneration, a listener, and a synthesized sound image. It is a flowchart for demonstrating an example of the sound image display process performed in the audio | voice signal reproduction | regeneration apparatus of FIG. It is a figure which shows an example of the display part in the audio | voice signal reproduction | regeneration apparatus of FIG. It is a figure which shows the other example of the display part in the audio | voice signal reproduction | regeneration apparatus of FIG. This is an arrangement example of five speakers excluding LFE from the speaker group of the 5.1ch surround system. It is a figure which shows the example of arrangement | positioning of the output object speaker after the downmix in the example of arrangement | positioning of FIG. It is a flowchart for demonstrating an example of the focused pair determination process in the isolation | separation extraction part in the audio | voice signal processing part of FIG. It is a schematic diagram for demonstrating the other example of the positional relationship of a listener, a right-and-left speaker, and a virtual sound source. It is a schematic diagram for demonstrating an example of the positional relationship of a listener, left and right speakers, and left and right surround speakers, and a virtual sound source. It is a schematic diagram for demonstrating the other example of the positional relationship of a listener, a left-right speaker, a right-and-left surround speaker, and a virtual sound source. It is a schematic diagram for demonstrating the other example of the positional relationship of a listener, a left-right speaker, a right-and-left surround speaker, and a virtual sound source. It is a schematic diagram for demonstrating the other example of the positional relationship of a listener, a left-right speaker, a right-and-left surround speaker, and a virtual sound source. It is a schematic diagram for demonstrating an example of the positional relationship between the left and right speakers, the left and right surround speakers, and all virtual sound sources. It is a schematic diagram for demonstrating the other example of the positional relationship of a right-and-left speaker, a right-and-left surround speaker, and all the virtual sound sources. In a speaker group of a 6.1ch surround system, it is a figure which shows the example of arrangement | positioning of the output object speaker after a downmix among six speakers except LFE. In a speaker group of a 7.1ch surround system, it is a figure which shows the example of arrangement | positioning of the output object speaker after a downmix among seven speakers except LFE. In the technique of nonpatent literature 1, when a virtual sound source is provided behind the speaker group arranged on one straight line, it is a mimetic diagram for explaining a speaker which outputs a sound corresponding to each virtual sound source. . It is a schematic diagram for demonstrating the example of arrangement | positioning of the speaker group in the audio | voice signal reproduction | regeneration apparatus of FIG. It is a schematic diagram for demonstrating the other example of arrangement | positioning of the speaker group in the audio | voice signal reproduction | regeneration apparatus of FIG. It is a schematic diagram for demonstrating the other example of arrangement | positioning of the speaker group in the audio | voice signal reproduction | regeneration apparatus of FIG. It is a figure which shows the structural example of the video display system provided with the audio | voice signal reproduction | regeneration apparatus of FIG.

  The audio signal reproduction apparatus according to the present invention outputs an audio signal for a multi-channel reproduction method of two or more channels as it is from a corresponding speaker group, or from the speaker group by another reproduction method such as a wavefront synthesis reproduction method. This is an apparatus that outputs the sound signal after converting it into a sound signal that can provide a more appropriate sound image, and is characterized in that it performs a sound image display process as described later. The audio signal reproduction device according to the present invention may be configured to be able to execute audio reproduction processing and sound image display processing. For example, as various AV (Audio Visual) devices such as a display device such as a television device and an audio system. Can be configured.

  Regarding the audio playback function, below, basically, the input audio signals of two or more channels are converted into audio signals to be reproduced by a speaker group (a plurality of speakers) as a sound image for a virtual sound source by the wavefront synthesis playback method. A case of reproduction will be described as an example. In the following, a case will be described in which playback speakers having the same number and the same positions as the number of virtual sound sources are prepared, and audio signals to be output from the respective virtual sound sources are played back one-to-one from the corresponding playback speakers. .

  However, even when the virtual sound source is set so that the number and position of the speakers are different from those of the virtual sound source, it can be similarly applied by assigning each virtual sound source to the speaker group. Further, as described above, it is possible to adopt a configuration in which an input audio signal is output as it is. In this case as well, processing for sound image display processing according to the present invention, for example, processing up to separation and extraction of audio signals is necessary.

  Hereinafter, a configuration example and a processing example of an audio signal reproduction device according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of an audio signal reproduction device according to the present invention, and FIG. 2 is a block diagram illustrating a configuration example of an audio signal processing unit in the audio signal reproduction device of FIG.

  An audio signal reproduction device (audio data reproduction device) 10 illustrated in FIG. 1 includes a decoder 11, an audio signal extraction unit 12, and an audio signal processing unit 13, a D / A converter 14, a plurality of amplifiers 15, and a plurality of A speaker array 17 including speakers 16 is provided. The number of amplifiers 15 and speakers 16 is basically the same. The audio signal reproduction device 10 includes a display control unit 18 and a display unit 19 for sound image display processing.

  The decoder 11 decodes the content of audio only or video with audio, converts it into a signal processable format, and outputs it to the audio signal extraction unit 12. The content is acquired by downloading from the Internet from a digital broadcast content transmitted from a broadcasting station, a server that distributes digital content via a network, or reading from a recording medium such as an external storage device. Thus, although not shown in FIG. 1, the audio signal reproduction device 10 includes a digital content input unit that inputs digital content including a multi-channel input audio signal. The decoder 11 decodes the digital content input here.

  The audio signal extraction unit 12 separates and extracts an audio signal from the obtained signal. In the example described here, the audio signal to be extracted is a 2ch stereo signal, and the audio signal extraction unit 12 outputs the signals for the two channels to the audio signal processing unit 13. Of course, if the originally input signal is a 2ch stereo signal, the audio signal extraction unit 12 may extract the 2ch stereo signal.

  On the other hand, for example, when the input audio signal has a channel number exceeding 2 ch, such as 5.1 ch, the audio signal extraction unit 12 is defined by, for example, ARIB STD-B21 “Digital Broadcasting Receiver Standard”. Then, it is downmixed to 2 ch by the normal downmix method of the following formula (1) and output to the audio signal processing unit 13.

In Equation (1), L _t and R _t are left and right channel signals after downmixing, and L, R, C, L _S and R _S are 5.1ch signals (left front channel signal, right front channel signal, Center channel signal, a left rear channel signal, the right rear channel signal), a overload reduction factor, for example, 1 / √2, _{k d} is, for example, 1 / √2 downmix coefficients or 1/2, or 1/2, √2 or 0.

  As described above, the multi-channel input audio signal may be an input audio signal of a multi-channel reproduction method having three or more channels. In this case, the audio signal processing unit 13 converts the multi-channel input audio signal to 2 The audio signals of the two channels after being downmixed to the audio signals of one channel may be subjected to sound image display processing described later. Alternatively, two channel audio signals may be simply extracted from an input audio signal of a multi-channel reproduction system having three or more channels, and only the audio signals of the two channels may be subjected to sound image display processing described later.

  The audio signal processing unit 13 generates a multi-channel audio signal (preferably 5 channels or more) different from the input audio signal as a part of the audio reproduction process from the obtained 2-channel signal.

  The audio signal processing unit 13 outputs these audio signals to the D / A converter 14. The D / A converter 14 converts the obtained signals into analog signals and outputs the respective signals to the amplifier 15. Each amplifier 15 amplifies the input analog signal and transmits it to each speaker 16, and this amplified analog signal is output from each speaker 16 as sound into the space.

  In the speaker array 17, the speakers 16 are arranged in one or a plurality of rows, and each shape thereof may be any shape such as a circle, an ellipse, or a rhombus. Further, the direction of arrangement is not limited to a straight line, and the centers of the speakers 16 may be arranged in a curved line. Note that the audio signal processing unit 13 may determine the delay amount and output level for each speaker 16 according to the arrangement of each speaker 16 in the speaker array 17.

  Further, the audio signal processing unit 13 generates sound image direction information from the obtained two-channel signal as part of the sound image display process which is the main feature of the present invention. The sound image direction information is the estimated direction angle (value indicating the sound image direction) of the sound image and the power value (signal power value) of the signal component for each frequency (for each line spectrum) or for each frequency band (for each frequency region). Point to. The audio signal processing unit 13 outputs the sound image direction information generated for each line spectrum or each frequency band to the display control unit 18.

  The display control unit 18 includes an adding unit 18a that obtains information for sound image display. The adder 18a assigns a calculated signal power value to one of a plurality of predetermined sound image direction groups (sound image direction ranges) based on the calculated sound image direction for each correlation signal, and the sound image direction An added value of the signal power value is obtained for each group. The display control unit 18 outputs the addition value for each sound image direction group obtained by the addition unit 18 a to the display unit 19.

  The display unit 19 is an example of a sound image display unit, and the sound image display unit is provided with each sound image at a position corresponding to each of the plurality of sound image direction groups (preferably a position that coincides with the central direction of each sound image direction group). Information indicating the added value for the direction group is displayed. A specific display example will be described later. The signal power value for each sound image direction group changes in time series according to the 2-channel signal to be displayed, and the display unit 19 can change the display of the sound image direction in accordance with such change. As described above, in the present invention, the sound image position can be displayed so that the sound image position matches between hearing and perception, and the listener's immersive feeling can be enhanced.

  Hereinafter, a detailed configuration example of the audio signal processing unit 13 that performs audio reproduction processing and sound image display processing will be described with reference to FIG. The audio signal processing unit 13 includes a conversion unit 21, a separation / extraction unit 22, an inverse conversion unit 23, and an audio output signal generation unit 24.

  The conversion unit 21 reads out each of the input signals of the two channels that have been input as much as ¼ of one segment of audio data. Here, the audio data refers to a discrete audio signal waveform sampled at a sampling frequency such as 48 kHz. A segment is an audio data section composed of a group of sample points having a certain length. Here, the segment refers to a section length to be subjected to discrete Fourier transform later, and is also called a processing segment. For example, the value is 1024. In this example, 256 points of audio data that is ¼ of one segment are to be read.

  The read out 256-point audio data is stored in the buffer. This buffer can hold the sound signal waveform for the immediately preceding segment, and the past segments are discarded. Audio data for one segment is created by connecting the previous 3/4 segment data (768 points) and the latest 1/4 segment data (256 points), and is multiplied by a window function. That is, all sample data is read four times in the window function calculation.

Here, the multiplication of the window function executes a window function calculation process for multiplying the audio data for one segment by the next Hann window that has been conventionally proposed.

m is a natural number, M is a segment length and an even number. Assuming that the input signals to the converter 21 are x _L (m) and x _R (m), respectively, the sound signals x ′ _L (m) and x ′ _R (m) after the window function multiplication are
x ′ _L (m) = w (m) × _L (m),
x ′ _R (m) = w (m) × _R (m) (3)
Is calculated.

The transform unit 21 performs discrete Fourier transform on the sound data obtained in this way as in the following equation (4) to obtain sound data in the frequency domain. That is, the conversion unit 21 performs a discrete Fourier transform on the two channels (speech data). Here, DFT represents discrete Fourier transform, k is a natural number, and 0 <k ≦ M / 2. X _L (k) and X _R (k) are complex numbers.
X _L (k) = DFT (x ′ _L (m)),
X _R (k) = DFT (x ′ _R (m)) (4)

  The separation / extraction unit 22 separates (extracts) the correlation signal and the non-correlation signal for each of the two channels converted by the conversion unit 21 for each line spectrum (that is, for each frequency). Below, the part which performs such isolation | separation (extraction) among the isolation | separation extraction parts 22 is demonstrated as the isolation | separation part 22a. That is, the separation unit 22a separates the two audio signals into a correlation signal and a non-correlation signal for each frequency (or for each frequency band). Here, the correlation signal refers to a signal that correlates with respect to frequency, and the non-correlation signal refers to a signal that does not correlate with respect to frequency.

  The separation / extraction unit 22 includes a calculation unit 22b for sound image display processing, and the calculation unit 22b calculates a sound image direction and a signal power value for each correlation signal separated by the separation unit 22a. More specifically, for each line spectrum, the calculation unit 22b estimates the angle in the sound image direction of the correlation signal and obtains the signal power value of the correlation signal. In addition, when the sound image direction and the signal power value are necessary in the sound reproduction process as in the example described below, the calculation unit 22b serves as both the sound image display process and the sound reproduction process. As a case where the sound image direction and the signal power value are not necessary for the sound reproduction process, as described above, there is a configuration in which the input sound signal is directly output as sound.

  Further, the separation unit 22a may separate and extract the correlation signal and the non-correlation signal for each frequency band (small band) instead of for each line spectrum. The signal power value is also calculated for the correlation signal for each small band. That is, here, an example of performing processing such as obtaining a correlation coefficient for each line spectrum will be described. However, as described in Patent Document 1, a band (frequency) divided using an Equivalent Rectangular Band (ERB) is used. A process such as obtaining a correlation coefficient may be executed for each area (also called a small band).

  After the calculation by the calculation unit 22b, the addition unit 18a separates (corresponds to) the correlation signals for each direction based on the estimated angle, and calculates an addition value for the signal power values of the correlation signals belonging to each sound image direction group. Ask. Then, the display unit 19 displays the sound image direction group and information indicating the added value in association with each other.

  The specific contents of the processing in the separation / extraction unit 22 will be described with reference to FIG. The separation / extraction unit 22 performs the processes of steps S32 to S34 for each line spectrum on the two-channel audio signals after the discrete Fourier transform by the conversion unit 21 (steps S31a and S31b. Specifically, the individual processes will be described). To do.

The line spectrum after the discrete Fourier transform is symmetrical with respect to M / 2 (where M is an even number) except for the DC component, that is, X _L (0), for example. That is, X _L (k) and X _L (M−k) have a complex conjugate relationship in the range of 0 <k <M / 2. Therefore, in the following, the range of k ≦ M / 2 is considered as the object of analysis, and the range of k> M / 2 is treated the same as a symmetric line spectrum having a complex conjugate relationship.

  Next, the correlation coefficient is acquired by calculating | requiring the normalization correlation coefficient of the left channel and the right channel with following Formula with respect to each line spectrum. Here, when the gain (amplitude) of the audio signal X of the target line spectrum is G, P (X) represents the power (corresponding to the sound pressure) of the audio signal X and can be expressed by the square value of the gain G. .

This normalized correlation coefficient d ^(k) represents how much the audio signals of the left and right channels are correlated, and takes a real value between 0 and 1. 1 if the signals are exactly the same, and 0 if the signals are completely uncorrelated. Here, when both the powers P _L ^(k) and P _R ^(k) of the audio signals of the left and right channels are 0, it is impossible to extract the correlated signal and the uncorrelated signal with respect to the line spectrum, and processing is performed. Let's move on to the processing of the next line spectrum. Also, if either one of P _L ^(k) or P _R ^(k) is 0, the calculation cannot be performed using Equation (5), but the normalized correlation coefficient d ^(k) = 0 and the line Continue processing the spectrum.

Next, using this normalized correlation coefficient d ^(k) , a conversion coefficient for separating and extracting the correlation signal and the non-correlation signal from the left and right channel audio signals is obtained (step S32), and obtained in step S32. Using each conversion coefficient, a correlation signal and a non-correlation signal are separated and extracted from the audio signals of the left and right channels (step S33). What is necessary is just to extract both a correlation signal and a non-correlation signal as the estimated audio | voice signal. The processing in steps S32 and S33 may be performed mainly by the separation unit 22a in the separation / extraction unit 22, but also serves as processing in the calculation unit 22b.

  A processing example of steps S32 and S33 will be described. Here, as in Patent Document 1, each signal of the left and right channels is composed of an uncorrelated signal and a correlated signal, and for the correlated signal, a signal waveform that differs only in gain from the left and right channels (that is, a signal waveform composed of the same frequency component) A model is assumed that is output. Here, the gain corresponds to the amplitude of the signal waveform and is a value related to the sound pressure. In this model, the direction of the sound image synthesized by the correlation signal output from the left and right channel signals is determined by the balance of the left and right sound pressures of the correlation signal.

According to the model, the input signals x _L (m), x _R (m) are
x _L (m) = s (m) + n _L (m),
x _R (m) = αs (m) + n _R (m) (9)
It is expressed. Here, s (m) is a left and right correlation signal, n _L (m) is a subtracted correlation signal s (m) from a left channel audio signal, and can be defined as an uncorrelated signal (left channel). , N _R (m) is obtained by subtracting the correlation signal s (m) multiplied by α from the right channel audio signal, and can be defined as an uncorrelated signal (right channel). Α is a positive real number representing the degree of left and right sound pressure balance of the correlation signal.

From Equation (9), the audio signals x ′ _L (m) and x ′ _R (m) after the window function multiplication described in Equation (3) are expressed by the following Equation (10). However, s ′ (m), n ′ _L (m), and n ′ _R (m) are obtained by multiplying s (m), n _L (m), and n _R (m) by a window function, respectively.
x ′ _L (m) = w (m) {s (m) + n _L (m)} = s ′ (m) + n ′ _L (m),
x ′ _R (m) = w (m) {αs (m) + n _R (m)} = αs ′ (m) + n ′ _R (m)
(Ten)

The following equation (11) is obtained by performing a discrete Fourier transform on the equation (10). However, S (k), N _L (k), and N _R (k) are discrete Fourier transforms of s ′ (m), n ′ _L (m), and n ′ _R (m), respectively.
X _L (k) = S (k) + N _L (k),
X _R (k) = αS (k) + N _R (k) (11)

Therefore, the audio signals X _L (k), X _R (k) in the k-th line spectrum are
X _L (k) = S (k) + N _L (k),
X _R (k) = α ^(k) S (k) + N _R (k) (12)
It is expressed. Here, α ^(k) represents α in the k-th line spectrum.

と表される。ここで、左右のチャネル間の無相関信号の音圧は等しいと仮定している。 From Equation (12), the sound pressures P _L ^(k) and P _R ^(k) in Equation (8 ⁾ are
P _L ^(k) = P _S ^(k) + P _N ^(k)
P _R ^(k) = [α ^(k) ] ² P _S ^(k) + P _N ^(k) (13)
It is expressed. Here, P _S ^(k) and P _N ^(k) are the powers of the correlated signal and uncorrelated signal in the k-th line spectrum, respectively.

It is expressed. Here, it is assumed that the sound pressures of the uncorrelated signals between the left and right channels are equal.

Further, from the equations (6) to (8), d ^(k) can be expressed by the following equation (15). However, in this calculation, it is assumed that S (k), N _L (k), and N _R (k) are orthogonal to each other and the power when multiplied is 0.

By solving Equation (13) and Equation (15), the following equation is obtained.

Using these values, a correlation signal and a non-correlation signal in each line spectrum are estimated. Estimate the estimated value est (S (k)) of the correlation signal S (k) in the kth line spectrum using the parameters μ ₁ and μ ₂ ,
est (S (k)) = μ ₁ X _L (k) + μ ₂ X _R (k) (18)
The estimated error ε is
ε = est (S (k)) − S (k) (19)
It is expressed. Here, est (A) represents an estimated value of A. And when the square error ε ² is minimized, using the property that ε and X _L (k) and X _R (k) are orthogonal to each other,
E [ε · X _L (k)] = 0, E [ε · X _R (k)] = 0 (20)
This relationship holds.

The following simultaneous equations can be derived from Equation (20) using Equations (12), (14), and (16) to (19).
_{(1-μ 1 -μ 2 α} (k)) P S (k) -μ 1 P N (k) = 0
α ^(k) (1-μ ₁ −μ ₂ α ^(k) ) P _S ^(k) −μ ₂ P _N ^(k) = 0
(twenty one)

ここで、このようにして求まる推定値ｅｓｔ（Ｓ（ｋ））の電力Ｐ_{ｅｓｔ（Ｓ）} ^（ｋ）が、数式(18)の両辺を二乗して求まる次の式
Ｐ_{ｅｓｔ（Ｓ）} ^（ｋ）＝（μ_１＋α^（ｋ）μ_２）^２Ｐ_Ｓ ^（ｋ）＋（μ_１ ^２＋μ_２ ^２）Ｐ_Ｎ ^（ｋ） (23)
を満たす必要があるため、この式から推定値を次式のようにスケーリングする。なお、ｅｓｔ′（Ａ）はＡの推定値をスケーリングしたものを表す。 By solving the equation (21), each parameter is obtained as follows.

Here, the power P _{est (S)} ^(k) of the estimated value est (S (k)) obtained in this way is obtained by squaring both sides of the equation (18), and the following equation P _{est (S)} ^{(k _{) = (μ 1 + α (}} k) μ 2) 2 P S (k) + (μ 1 2 + μ 2 2) P N (k) (23)
Therefore, the estimated value is scaled as follows from this equation. Note that est ′ (A) represents a scaled estimate of A.

と求めることができる。このようにして求めた推定値ｅｓｔ（Ｎ_Ｌ（ｋ））、ｅｓｔ（Ｎ_Ｒ（ｋ））も上述と同様に、次の式によってそれぞれスケーリングする。 The estimated values est (N _L (k)) and est (N _R (k)) for the left and right channel uncorrelated signals N _L (k) and N _R (k) in the k-th line spectrum are
est (N _L (k)) = μ ₃ X _L (k) + μ ₄ X _R (k) (25)
est (N _R (k)) = μ ₅ X _L (k) + μ ₆ X _R (k) (26)
Thus, in the same manner as the above-described method, the parametric variables μ _{3 to} μ ₆ are

It can be asked. The estimated values est (N _L (k)) and est (N _R (k)) obtained in this way are also scaled by the following equations in the same manner as described above.

The respective transformation variables μ _{1 to} μ ₆ represented by the mathematical expressions (22), (27), and (28) and the scaling coefficients represented by the mathematical expressions (24), (29), and (30) are converted coefficients obtained in step S32. It corresponds to. In step S33, the correlation signal and the non-correlated signal (the uncorrelated signal of the right channel, the uncorrelated signal of the left channel, and the left channel) are estimated by calculation using these conversion coefficients (Equations (18), (25), (26)). And uncorrelated signals).

  Next, an allocation process to the reproduction speaker 16 is performed (step S34). This process corresponds to a process in which a virtual sound source is set for each of the playback speakers 16. When a virtual sound source is set at a position different from the speaker 16, the sound signal processing unit 13 assigns the sound source to the virtual sound source in the same manner as the processing described here, and then depends on the arrangement of each speaker 16. Thus, the delay amount and output level for each speaker 16 may be determined.

  In the allocation process of step S34, the direction of the synthesized sound image generated by the correlation signal estimated for each line spectrum is estimated as preprocessing. This estimation process is performed by the calculation unit 22b of the separation / extraction unit 22 of FIG. This estimation process will be described with reference to FIGS. FIG. 4 is a schematic diagram for explaining an example of the positional relationship between the listener, the left and right speakers, and the synthesized sound image. FIG. 5 is a diagram for explaining an example of the positional relationship between the reproduction speaker, the listener, and the synthesized sound image. FIG.

Now, as in the positional relationship 40 shown in FIG. 4, a line drawn from the listener to the midpoint of the left and right speakers 41L and 41R, and a line drawn from the listener 43 to the center of one of the speakers 41L / 41R. The spread angle formed is θ ₀ , and the spread angle formed by the line drawn from the listener 43 to the position of the estimated synthesized sound image 42 is θ. Here, when the same audio signal is output from the left and right speakers 41L and 41R while changing the sound pressure balance, the direction of the synthesized sound image 42 generated by the output sound is determined using the parameter α representing the sound pressure balance. It is generally known that the following equation can be approximated (hereinafter referred to as the sign law in stereophonic sound).

  Here, in order to be able to reproduce the 2ch stereo audio signal by the wavefront synthesis reproduction method, the separation unit 22a shown in FIG. 2 converts the 2ch signal into a signal of a plurality of channels. For example, when the number of channels after conversion is five, the channels are reproduced using the reproduction speakers 52a to 52e that are equally arranged as the speaker array 17 as in the positional relationship 50 shown in FIG. The reproduction speakers 52a to 52e correspond to the plurality of speakers 16 in FIG. As already described, the separation unit 22a first separates the 2ch audio signal into one correlated signal and two uncorrelated signals for each line spectrum. In the separation unit 22a, it is necessary to determine in advance how to allocate those signals to the reproduction speakers (here, five reproduction speakers). The assignment method may be user-configurable from a plurality of methods, or may be presented to the user by changing the selectable method according to the number of playback speakers.

  As an example of the allocation method, the following method is adopted. First, left and right uncorrelated signals are assigned to both ends (speakers 52a and 52e) of five reproduction speakers, respectively. Next, the synthesized sound image generated by the correlation signal is assigned to two adjacent reproduction speakers out of the five. As for pre-assignment to which two adjacent reproduction speakers, first, it is assumed that the synthesized sound image generated by the correlation signal is located inside both ends (reproduction speakers 52a and 52e) of the five reproduction speakers. It is assumed that five reproduction speakers 52a to 52e are arranged so as to fall within a spread angle formed by two speakers during 2ch stereo reproduction. Then, two adjacent playback speakers that sandwich the synthesized sound image are determined from the estimated direction of the synthesized sound image, the sound pressure balance allocation to the two playback speakers is adjusted, and the two playback speakers are adjusted. An allocation method is adopted in which reproduction is performed so that a synthesized sound image is generated by a speaker.

Therefore, as in the positional relationship 50 shown in FIG. 5, the spread angle formed by the line drawn from the listener 53 to the middle point of the reproduction speakers 52a and 52e at both ends and the line drawn from the reproduction speaker 52e at the end is θ. ₀ ′, the spread angle formed by the line drawn at the midpoint and the line drawn from the listener 53 on the synthesized sound image 51 is θ ′. Further, the spread angle formed by the line drawn at the midpoint between the two reproduction speakers 52c and 52d sandwiching the synthesized sound image 51 from the listener 53 and the reproduction speaker 52d from the listener 53 is φ ₀ , and the synthesis is performed from the listener 53. The spread angle formed by the line drawn on the sound image 51 is φ. Here, φ ₀ is a positive real number. A method of assigning the synthesized sound image 42 in FIG. 4 (corresponding to the synthesized sound image 51 in FIG. 5) whose direction has been estimated as described in Expression (31) to the reproduction speaker using these variables will be described.

First, it is assumed that the direction θ ^(k) of the k-th synthesized sound image is estimated by Expression (31), and for example, θ ^(k) = π / 15 [rad]. When there are five reproduction speakers, the synthesized sound image 51 is positioned between the third reproduction speaker 52c and the fourth reproduction speaker 52d as counted from the left as shown in FIG. Further, when there are five playback speakers, φ ₀ ≈0.11 [0.1121 [between the third playback speaker 52c and the fourth playback speaker 52d by a simple geometric calculation using a trigonometric function. rad], and φ in the k-th line spectrum is φ ^(k) , then φ ^(k) = θ ^(k) −φ ₀ ≈0.088 [rad]. In this way, the direction of the synthesized sound image generated by the correlation signal in each line spectrum is represented by a relative angle from the direction of the two playback speakers sandwiching the direction. Then, as described above, it is considered that the synthesized sound image is generated by the two reproduction speakers 52c and 52d. For this purpose, it is only necessary to adjust the sound pressure balance of the output audio signals from the two reproduction speakers 52c and 52d. As the adjustment method, the law of sign in the stereophonic sound used again as Equation (31) is used.

を満たせばよい。 Here, of the two playback speakers 52c and 52d sandwiching the synthesized sound image generated by the correlation signal in the kth line spectrum, the scaling coefficient for the third playback speaker 52c is g ₁ and the fourth playback speaker 52d is set. When the scaling factor and _{g 2,} of the third reproducing speaker _{52c g 1 · est '(S} (k)), g 2 · est from the fourth reproducing speaker 52d' (S (k)) An audio signal is output. And g ₁ and g ₂ are based on the sign law in stereophonic sound,

Should be satisfied.

On the other hand, when g ₁ and g ₂ are normalized so that the total power from the third playback speaker 52c and the fourth playback speaker 52d is equal to the power of the original 2ch stereo correlation signal,
g ₁ ² + g ₂ ² = 1 + [α ^(k) ] ² (33)
It becomes.

By combining these, Equation (34) is obtained.

By substituting the aforementioned φ ^(k) and φ ₀ into this mathematical formula (34), g ₁ and g ₂ are calculated. Based on the scaling coefficient thus calculated, the audio signal of g ₁ · est ′ (S (k)) is transmitted to the third reproduction speaker 52c as described above, and the g signal from the fourth reproduction speaker 52d is g. ₂ · est ′ (S (k)) audio signals are assigned. As described above, the uncorrelated signal is assigned to the reproduction speakers 52a and 52e at both ends. That is, est ′ (N _L (k)) is assigned to the first reproduction speaker 52a, and est ′ (N _R (k)) is assigned to the fifth reproduction speaker 52e.

Unlike this example, if the estimated direction of the synthesized sound image is between the first and second reproduction speakers, the first reproduction speaker has g ₁ · est ′ (S (k)). And est ′ (N _L (k)) will be assigned. If the estimated direction of the synthesized sound image is between the fourth and fifth reproduction speakers, the second reproduction speaker has g ₂ · est ′ (S (k)) and est ′ ( N _R (k)) will be assigned.

  The process as described above is performed for all line spectra by the loop of steps S31a and S31b. For example, when 256 discrete Fourier transforms are performed, all the points of the segment (1024 points) up to the 1st to 127th line spectrum, and when 512 discrete Fourier transforms are performed, the 1st to 255th line spectrum. When the discrete Fourier transform is performed for, the first to 511th line spectra are obtained.

As a result, when the number of reproduction speakers is J (J = 5 in this example), output audio signals Y ₁ (k),..., Y _J (k) in the frequency domain for each reproduction speaker (output channel). ) Is obtained. These outputs are output results to the inverse transform unit 23 of the separation / extraction unit 22 in FIG. As described above, the processing of the separation / extraction unit 22 in FIG. 2 is performed.

  Next, processing in the inverse transform unit 23 in FIG. 2 is performed. The inverse transform unit 23 performs discrete Fourier inverse transform on the correlation signal extracted by the separation unit 22a. The inverse transform unit 23 generates (a1) the correlation signal and the non-correlation signal (signal excluding the correlation signal) or (a2) the correlation signal extracted from the correlation signal instead of the correlation signal extracted by the separation unit 22a. The discrete Fourier inverse transform may be applied to the audio signal generated or (a3) the audio signal generated from the correlated signal and the uncorrelated signal.

Specifically, the processing of the inverse transform unit 23 is exemplified. The inverse transform unit 23 performs discrete Fourier inverse transform on each output channel output from the separation and extraction unit 22 to thereby output the time-domain output speech signal y ′ _J (m ) Here, DFT ⁻¹ represents discrete Fourier inverse transform.

y ′ _J (m) = DFT ⁻¹ (Y _J (k)) (1 ≦ j ≦ J) (35)
Here, as described in the equations (3) and (4), the signal obtained by performing the discrete Fourier transform is a signal after the window function multiplication, so that the signal y ′ _J (m) obtained by the inverse transform is also a window. The function has been multiplied. Therefore, the signal obtained in this way is multiplied by the window function shown in Equation (2) again, and added to the output buffer while shifting by 1/4 segment length from the head of the previous processed segment. To obtain the converted data.

  The audio output signal generation unit 24 in FIG. 2 generates an audio signal for each speaker from the audio signal of each virtual sound source generated by the inverse conversion unit 23 according to the technique described in Non-Patent Document 1 described above. Here, from the audio signal of each virtual sound source, the delay amount and the output level for each speaker are determined according to the arrangement of each speaker, and the audio signal corresponding to that is generated. In this way, the audio signal processing unit 13 can convert the input audio signal of the channel of the multi-channel reproduction system so that it can be reproduced by the speaker group. However, when a virtual sound source is set for each of the playback speakers as in the example given here, such processing in the audio output signal generation unit 24 becomes unnecessary. Whether or not the virtual sound source is matched with the playback speaker, the audio output signal generation unit 24 performs other processing such as a process of adding audio signals of channels that are not used in the input audio signal as necessary. What is necessary is just to comprise so that the process of this may be performed.

  The sound reproduction process is performed as described above. As described above, sound image display processing is also performed in the present invention. Regarding sound image display processing, sound image direction information output from the audio signal processing unit 13 to the display control unit 18 in FIG. 1 will be described. As described above, the sound image direction information includes the sound image direction and the signal power value, and is generated by the separation and extraction unit 22.

Specifically, the sound image direction is exemplified by the angle (estimated angle) of the estimated direction shown as θ in FIG. 4, and the signal power value is exemplified by P _S ^(k) shown in Equation (14). These are all values generated for each line spectrum (that is, for each correlation signal of each line spectrum). Although not mentioned in particular, the calculation unit 22b of the separation / extraction unit 22 in FIG. 2 also performs processing for calculating the signal power value as described above. The calculation unit 22b outputs the estimated angle and signal power value calculated for the correlation signal of each line spectrum to the display control unit 18.

  Next, processing in the display control unit 18 and the display unit 19 will be described with reference to FIGS. FIG. 6 is a flowchart for explaining an example of a sound image display process mainly executed by the adding unit 18a in the audio signal reproduction device 10 of FIG. FIG. 7 is a diagram illustrating an example of the display unit 19 in the audio signal reproduction device 10 of FIG.

  The display control unit 18 executes the processes of steps S62 and S63 for each line spectrum (steps S61a and S61b). In step S62, the adding unit 18a specifies a sound image direction group based on the estimated angle θ of the target line spectrum correlation signal. The sound image direction group is obtained by dividing the possible angle θ of the sound image direction into groups, and is hereinafter simply referred to as a group. The range that θ can take is the range of the spread angle from the rightmost speaker to the leftmost speaker at the assumed listening position, and is a predetermined range.

  For example, when this range is −30 ° to 30 ° and the sound image direction is displayed on the display unit 19 as 15 bar graphs as shown in FIG. 7, this range may be divided into 15 groups. At that time, the first group is divided into groups of 4 °, such as −30 ° or more and less than −26 °, and the second group of −26 ° or more and less than −22 °. Here, although an example of dividing equally is shown, it may not be equally, and it may be divided into groups according to the way of display on the display unit 19.

In step S63, the adding unit 18a adds the signal power value for the correlation signal to the group specified in step S62. More specifically, the adding unit 18a adds the power value P _S (k) of the correlation signal to a buffer for each group prepared in advance. Here, an example of simple addition is shown, but if the power value is weighted and added based on an equal loudness curve (ISO 226: 2003), this numerical value can be brought close to the sound pressure received for hearing. It becomes possible. That is, the mere sum is given as an example of the addition value, but the present invention is not limited to this. For example, a weighted addition value obtained by weighting for each line spectrum (or for each frequency band) can also be adopted.

By repeating steps S62 and S63 for all the line spectra, the signal power value is added for each group, and the sum of the signal power values for each group is calculated.
The calculated added value is displayed on the display unit 19. The display control unit 18 also performs control for this display.

  After completing the power value addition for all the line spectra, the display control unit 18 executes the processes of steps S65 and S66 for each group (steps S64a and 64b). In step S65, the display control unit 18 converts the added value of the signal power value into a value for displaying for each group. That is, the sum of signal power values for each group is converted into a value for display. In step S <b> 66, the display control unit 18 generates a control signal for displaying the converted value and outputs the control signal to the display unit 19. The display unit 19 displays a sound image according to the control signal.

A specific example of the sound image displayed on the display unit 19 will be described with reference to FIG.
The television device 70 illustrated in FIG. 7 is an example of a display device. The display device reproduces an audio signal and passes it to the speaker array 17 and an image for displaying an image indicated by the image signal. And a display panel 71 as an example of a display unit (a liquid crystal panel, an organic electroluminescence panel, or the like). The display panel 71 is also an example of the display unit (sound image display unit) 19. Actually, a video can be displayed on the screen of the video display unit, and information indicating an added value for each group, that is, information indicating a sound image may be included in the video. Alternatively, this information can be displayed as an OSD (On Screen Display) image superimposed on the screen of the video display unit. Therefore, an example of the control signal in step S66 is a signal for displaying such a video or OSD image.

  The television device 70 incorporates a speaker array 17 in which a plurality of speakers 16 (speakers 16a, 16b, 16c,...) Shown in FIG. However, the speaker array 17 may be connected to the outside. When externally connected, the speaker array 17 can be embedded in a TV stand (TV board), or can be embedded as an integrated speaker system placed under a TV device called a sound bar. In this example, the speaker array 17 is provided on the lower side of the display panel 71, but may be provided on the upper side, or on the upper side and the lower side.

  On the display panel 71, information indicating the added value is displayed as a bar graph 72. In the bar graph 72, information 73a, 73b, 73c,... That indicates an added value for each group at a position corresponding to each of the plurality of groups (in this example, the installation positions of the speakers 16a, 16b, 16c,...). . . It is the graph which displayed. That is, in the bar graph 72, positions corresponding to each of a plurality of groups are arranged on one axis, and a bar having a length corresponding to an added value (signal power value of a correlation signal included in each group) for each group. It is a graph with the other axis.

  An added value of signal power values for outputting correlation signals included in one group corresponds to the strength of sound pressure. Therefore, such a bar graph 72 can express the strength of sound pressure from the direction in which the bar exists by changing the height of the bar.

  Thus, when it is desired to display information indicating the added value for each group in the form of a bar graph on the display panel 71, in step S65, the added value for each group (the added value of the signal power value) is set to the height of each bar. In other words, the higher the added value, the higher the bar is drawn.

  Since the addition value for each group is calculated for each processing segment as described above, by performing drawing for each processing segment, in synchronization with the sound output from the speaker array 17 having a plurality of speakers 16, The bar graph 72 will move. As a result, the sound image display can be changed according to the sound reproduction, the direction of the sound image that can be heard matches the place of the visual effect (the sound image position is the same for hearing and perception), and the listener is immersed. A feeling can be given. In particular, according to the display in the bar graph format such as the bar graph 72, the listener can recognize the sound image at a glance, so that the listener's immersive feeling can be further increased compared to other sound image display methods.

  In addition, the number of bars (that is, the number of groups) in the bar graph is basically equal to the resolution of the speaker group (that is, the number of speakers installed) as illustrated in FIG. This is preferable because it can be matched with a bar that can be formed. However, the number of speakers may be different from the number of groups.

  In the above description, the audio signal reproducing device according to the present invention is described on the assumption that the audio signal is reproduced from the speaker array. However, for example, the audio signal reproducing device may simply output from the left and right speakers. However, the sound image display is executed as described here, and the sound reproduction only needs to be output from the two left and right speakers, so that the present invention can be similarly applied. In addition, the present invention can be similarly applied to the case of outputting in another reproduction system, for example, in the case of outputting in a multi-channel reproduction system from a 5.1ch speaker system.

Next, another configuration example of the display unit (sound image display unit) 19 will be described with reference to FIG. FIG. 8 is a diagram showing another example of the display unit 19 in the audio signal reproduction device 10 of FIG.
The display unit 19 exemplified here includes a plurality of light emitting units regardless of whether or not the audio signal reproduction device 10 includes a video display unit, and performs sound image display using these light emitting units. Examples of the audio signal reproduction device 10 that does not include a video display unit include various audio systems.

  The speaker array 17 illustrated in FIG. 8 includes the speakers 16a, 16b, 16c,. . . 8 LEDs (Light Emitting Diodes) 81a, 81b, 81c,. . . Is arranged. That is, one of the light emitting units is composed of eight LEDs 81a. However, the number of LEDs 81a corresponding to the light emitting unit is not limited to eight, and may be only one, for example. Moreover, not only LED but another kind of light emission part can also be provided.

  In addition, it is preferable that the light emitting unit is basically prepared for the resolution of the speaker group (that is, the number of speakers to be installed) because the sound image that can be heard matches the light emission, but this is limited to this. is not. In the case of the speaker array 17, a plurality of the light emitting units may be arranged along the longitudinal direction. However, it is not limited to those arranged along the longitudinal direction of the speaker array 17 and may be arranged at a position corresponding to the group. Further, the LED arrangement for one speaker is not limited to the arrangement surrounding the speaker as illustrated in FIG. It is also possible to display a plurality of light emitting units arranged in groups for each group in a bar graph format.

  Using the plurality of light emitting units having such a configuration, the display unit 19 changes the light emission color of the light emitting unit at the position corresponding to each of the plurality of groups according to information indicating the added value for each group. As described above, the display unit 19 may express the intensity of sound pressure from the direction in which the LED exists by changing the color of the LED.

  For example, when a sound image exists only in the direction of the speaker 16b, the corresponding eight LEDs 81b are displayed in a color such as blue when the added value is weak, red when strong, and so on. Change colors in sync. Further, a strong sound (a sound with a large addition value) is output from the direction of the speaker 16b, a weak sound (a sound with a small addition value) is output from the direction of the speaker 16c, and no sound is output from the direction of other speakers. In such a case, the color of the eight LEDs 81b may be changed to red, and the color of the eight LEDs 81c may be changed to blue, for example.

  Further, the display unit 19 may change the light emission intensity (light emission luminance) of the LED at the position corresponding to each of the plurality of groups according to information indicating the added value for each group. If the added value is converted to the brightness of the LED, the brightness of the LED changes in synchronization with the position of the sound image. That is, the intensity of sound pressure from the direction (sound image position) where the LED exists may be expressed by a change in strength of the LED.

  For example, when the sound image exists only in the direction of the speaker 16b, all of the corresponding eight LEDs 81b emit light with the luminance corresponding to the added value, or the number of LEDs 81b corresponding to the added value among the eight LEDs 81b is set. What is necessary is just to change light-emission brightness in synchronization with sound, for example by making it emit light. When a strong sound is output from the direction of the speaker 16b, a weak sound is output from the direction of the speaker 16c, and no sound is output from the direction of other speakers, for example, eight LEDs 81b are caused to emit light. The light emission brightness may be changed in synchronization with the sound by causing one LED 81c to emit light.

  Even when such a sound image display method using a plurality of light emitting units is adopted, the sound image display can be changed in accordance with the reproduction of the sound, and the direction of the audible sound image matches the place of the visual effect. , Can give an immersive feeling to the listener. In particular, by changing the emission color according to the sound intensity or changing the emission intensity, for example, when playing music video or playing music, you can feel the feeling at the live venue Can be given to listeners.

  In the above example, the case where the listener visually recognizes the position of the sound image from one direction, that is, the case where the sound image display is performed only in one direction (forward direction) has been described. However, for example, for 5.1 to 7.1ch input audio signals, a virtual sound source is set and audio is output from array speakers prepared in three or four directions, as shown in FIG. 8 in each direction. It is also possible to make the sound image visible by changing the light emitting portion. Such an example will be described below.

  First, processing such as audio signal separation, sound image direction and signal power value calculation in such a configuration will be described with reference to FIGS. 1 and 2 again. Here, points that are fundamentally different from the above-described processing example will be described, and description of portions that perform similar processing will be omitted.

First, the audio signal extraction unit 12 separates and extracts an audio signal from the obtained signal. In this example, the obtained signal is 5.1 ch, of which L, R, C, L _S , R _S The audio signals of the five channels are output to the audio signal processing unit 13. The audio signals of the remaining LFE channels are processed by the time required for the audio signal processing unit 13 to process the signals L, R, C, L _S and R _S described above with a delay processing unit (not shown) provided separately. Delay and output to the D / A converter 14.

  The audio signal processing unit 13 in this example converts the input audio signals of five or more channels of the multi-channel reproduction method for reproduction by the speaker group. Here, in order to be able to express a more appropriate sound image, the plurality of speakers 16 are preferably made up of a number of speakers equal to or greater than the number of channels of the input audio signal. In this case, for example, when 5 channels out of 5.1 channels are used as the input audio signal, the number of channels of the plurality of speakers 16 is decreased by one by a downmix process described later, and is larger than the decreased number of channels. The number of speakers (in other words, the number equal to or greater than the number of channels of the input audio signal).

  More specifically, the audio signal processing unit 13 generates a multi-channel audio signal different from the input audio signal from the obtained 5-channel signal. That is, the audio signal processing unit 13 converts the input audio signal into another multi-channel audio signal. In order to output from the speaker having the number of channels of the input audio signal or more, the number of multi-channel channels after conversion is preferably set to the number of input channels (5 in this example) or more. However, allocation to a plurality of speakers (speaker groups) 16 is possible even if the number of channels is smaller than the number of input channels by using channels for virtual sound sources. In the following description, it is assumed that signals corresponding to the number of virtual sound sources are generated.

  The audio signal processing unit 13 outputs the generated audio signal to the D / A converter 14. The number of virtual sound sources can be determined in advance if there is a certain number or more, but the amount of calculation increases as the number of virtual sound sources increases. Therefore, it is desirable to determine the number in consideration of the performance of the mounted device. In the example described here, the number is assumed to be 16, and a case where virtual sound sources are arranged on the circumference as shown in FIG. 17 described later will be described, but the present invention is not limited to this example.

  The D / A converter 14 converts the obtained signals into analog signals, and outputs the respective signals to the amplifier 15 and a subwoofer amplifier (not shown). Each amplifier 15 amplifies the input analog signal and transmits it to each speaker 16, and this amplified analog signal is output from each speaker 16 as sound into the space. The subwoofer amplifier amplifies the input LFE analog signal and transmits it to a subwoofer (not shown), and the amplified analog signal is output from the subwoofer as sound into the space.

  The conversion unit 21 in this example includes a downmixing unit (not shown) that downmixes one specific channel among five or more channels that are input audio signals to two channels adjacent to the one specific channel. Have. Here, an example in which a C channel signal is employed as the specific channel will be described. The downmix unit adds the gain of the C signal multiplied by the downmix coefficient to each of the R and L channel signals. The downmix coefficient is a real number larger than 0 and does not change with time, and takes a value such as 1/2 or 1 / √2.

The conversion unit 21 reads each of four input signals, which are a combination of the two channels output from the downmix unit and L _S and R _S , as much as ¼ of audio data of one segment.
The read out 256-point audio data is stored in the buffer. This buffer can hold the sound signal waveform for the immediately preceding segment, and the past segments are discarded. Connect the previous 3/4 segment data (768 points) and the latest 1/4 segment data (256 points) to create audio data for one segment, and multiply by the window function of Equation (2) . That is, all sample data is read four times in the window function calculation.

Assuming that the output signals from the downmix unit are x _L (m), x _R (m), x _Ls (m), and x _Rs (m), respectively, the conversion unit 21 performs the audio signal x ′ _L after the window function multiplication. _{(m), x 'R (} m), x' LS (m), x 'RS (m) is,
x ′ _L (m) = w (m) × _L (m),
x ′ _R (m) = w (m) × _R (m),
x ′ _LS (m) = w (m) × _LS (m),
x ′ _RS (m) = w (m) × _RS (m) (36)
Is calculated.

The transform unit 21 performs discrete Fourier transform on the sound data obtained in this way as shown in the following equation (37) to obtain sound data in the frequency domain. That is, the conversion unit 21 performs discrete Fourier transform on the two channels after downmixing and the remaining channels (audio data thereof) excluding the specific channel. Here, DFT represents discrete Fourier transform, k is a natural number, and 0 <k ≦ M / 2. X _L (k), X _R (k), X _LS (k), and X _RS (k) are complex numbers.
X _L (k) = DFT (x ′ _L (m)),
X _R (k) = DFT (x ′ _R (m)),
X _LS (k) = DFT (x ′ _LS (m)),
X _RS (k) = DFT (x ′ _RS (m)) (37)

  For each line spectrum, the separation / extraction unit 22 separates and extracts a correlation signal and a non-correlation signal with respect to the combination of the two adjacent channels in the four or more channels converted by the conversion unit 21. For simplification of explanation, in this example, the processing of the separation / extraction unit 22 will be described without being divided into the processing of the separation unit 22a and the processing of the calculation unit 22b.

  The separation / extraction unit 22 in this example determines, based on the power level of each channel, which combination of the two adjacent channels is focused on for each line spectrum (that is, for each frequency). To do. As described above, the separation / extraction unit 22 may perform separation and extraction for each frequency region (small band) instead of for each line spectrum. In this case, the determination is performed for each small band.

  The specific contents of the separation and extraction process in the separation and extraction unit 22 will be described again with reference to FIG. In the 5.1ch example described here, the discrete Fourier transform is applied to four channels as shown in Equation (37). Therefore, the separation and extraction unit 22 performs the processing of steps S32 to S34 for each line spectrum for the four-channel audio signals after the discrete Fourier transform in the conversion unit 21, but prior to that, the pair determination process of interest for each line spectrum is performed. (Not shown in FIG. 3) is executed (steps S31a and S31b). Specific processing will be described.

  In the focused pair determination process described above, a process of determining which of the two input signals adjacent to each other in the speaker arrangement among the speaker groups assumed to be reproduced by the original multi-channel playback method is executed.

  Here, the definition of the adjacent speaker arrangement will be described with reference to FIGS. 9 and 10. FIG. 9 is a diagram showing an arrangement example of five speakers excluding LFE in the 5.1ch surround system speaker group, and FIG. 10 is an arrangement example of output target speakers after downmixing in the arrangement example of FIG. FIG.

  As shown in FIG. 9, in the 5.1ch surround system, there are a left front speaker 91, a right front speaker 92, a center speaker 93, a left surround speaker 94, and a right surround speaker 95 except for the LFE speaker. When determining whether or not the speaker arrangements are adjacent to each other, the C signal added to each of L and R in the above-described downmix unit is excluded from the target. That is, the four speakers shown in FIG. 10 (the left front speaker 91, the right front speaker 92, the left surround speaker 94, and the center speaker 93 related to the C signal to be added to other signals in the downmix unit). Consider the arrangement of the right surround speaker 95).

  As shown in FIG. 10, the right front speaker 92 and the left surround speaker 94 are adjacent to the left front speaker 91. Similarly, the left surround speaker 94 is adjacent to the left front speaker 91 and the right surround speaker 95, and the right front speaker 92 is adjacent to the left front speaker 91 and the right surround speaker 95, and adjacent to the right surround speaker 95. A right front speaker 92 and a left surround speaker 94 are formed. Therefore, in this case, there are four combinations (pairs) as shown by the double arrows in FIG.

  In the target pair determination process, a determination as to which of these pairs is focused is performed as illustrated in FIG. FIG. 11 is a flowchart for explaining an example of the target pair determination process in the separation / extraction unit 22.

First, for each pair, for each line spectrum, power sums P _{1 to} P ₄ are calculated as in the following formula (38) (step S111). Here, when the gain (amplitude) of the audio signal X of the target line spectrum is G, P (X) represents the power (corresponding to the sound pressure) of the audio signal X and can be expressed by the square value of the gain G. .
P ₁ (k) = P (X _L (k)) + P (X _R (k)),
P ₂ (k) = P (X _LS (k)) + P (X _L (k)),
P ₃ (k) = P (X _RS (k)) + P (X _LS (k)),
P ₄ (k) = P (X _R (k)) + P (X _RS (k)) (38)

Next, with respect to each pair of line spectra, a normalized correlation coefficient between both channels is obtained by the following equation (39) to obtain a correlation coefficient (step S112). Re {BB} and Im {CC} represent a real part of “BB” and an imaginary part of “CC”, respectively.

The normalized correlation coefficient _{^{d (k) 1 ~d (k}} ) 4 are intended to represent how much correlation to both channels of the audio signal takes a real value between 0 and 1. 1 if the signals are exactly the same, and 0 if the signals are completely uncorrelated. Here, when the power of the audio signals of both channels is 0, it is assumed that the correlation signal and the non-correlation signal cannot be extracted with respect to the line spectrum, and the process moves to the next line spectrum without performing the process. If either one of the powers is 0, the calculation cannot be performed using Equation (39), but the normalized correlation coefficient d ^(k) _r = 0 is set, and the processing of the line spectrum is continued. However, r = 1 to 4.

Next, for each pair, σ _r (k) obtained by multiplying the values obtained by Equation (38) and Equation (39) is calculated as follows.
σ ₁ (k) = P ₁ (k) × d ^(k) ₁ ,
σ ₂ (k) = P ₂ (k) × d ^(k) ₂ ,
σ ₃ (k) = P ₃ (k) × d ^(k) ₃ ,
σ ₄ (k) = P ₄ (k) × d ^(k) ₄ (40)

Then, r _max which is r when σ _r (k) is the maximum value is obtained, and a pair to be noted is determined (step S113). For example, when r _max = 1, the left front speaker 91 and the right front speaker 92 in FIG. In the following description, the case where the result of the determination in step S113 (ie, the result of the target pair determination process) is a pair to which the left front speaker 91 and the right front speaker 92 are to be focused will be described as an example. The same applies to a pair to be noted.

  As described above, for each line spectrum, the separation and extraction unit 22 determines which combination of the two adjacent channels is to be focused on, the power magnitude and the correlation coefficient for the two adjacent channel combinations, and It is preferable to make a determination based on the multiplication result. Of course, when the separation and extraction are performed for each small band, this determination is also performed for each small band.

Further, in the equation (40), when calculating σ _r (k), the calculation of the correlation coefficient can be omitted by setting d ^(k) _r = 1. Omitting the calculation of the correlation coefficient with d ^(k) _r = 1 means that, based on the sum of the magnitudes of the powers of two adjacent channels, a pair having a large sum is determined as a pair to be noted. means. As described above, the separation / extraction unit 22 may perform the determination based on the power levels of two adjacent channels for each line spectrum without using the correlation coefficient. However, the accuracy of the determination can be increased by using the correlation coefficient together for the determination.

Examples of other determinations are given. Since it is only necessary to finally obtain the maximum value of σ _r (k), whether or not d ^(k) _r = 1 is omitted or not, P _r (k) is set as a neighbor. The product of the power of the two matching channels can be employed. As yet another example, it can be determined to focus on a pair composed of a channel having the largest power and a channel having the largest power among the two adjacent channels.

  As a result of the target pair determination process, only the pair with the largest power and the large correlation coefficient, that is, only the target pair is assigned to the virtual sound source after being separated into a correlated signal and an uncorrelated signal as described later. Performed (steps S32 to S34). This process means that an assumption is made that there is one sound image belonging to the same line spectrum or small band. In this assumption, steps S32 to S34 are conversion processes for reproducing an accurate sound image position. It can be said that. Similarly, when the above determination is performed only for the magnitude of power, the processing of steps S32 to S34 may be performed only for the target pair.

First, using the normalized correlation coefficient d ^(k) ₁ described above, conversion coefficients for separating and extracting the correlated signal and the uncorrelated signal from the audio signals of both channels are obtained (step S32), and each acquired The correlation signal and the non-correlation signal are separated and extracted from the audio signals of both channels using the conversion coefficient (step S33). What is necessary is just to extract both a correlation signal and a non-correlation signal as the estimated audio | voice signal.

Specifically, by setting d ^(k) ₁ to d ^(k) , the processing in steps S32 and S33 may be performed similarly to the example described with reference to FIGS. Omitted. However, in this example, processing is executed for both channels of the target pair instead of the left and right channels. In other words, in the model used in this example, the signals of both channels are composed of uncorrelated signals and correlated signals, and for correlated signals, the signal waveforms that differ only in gain from both channels (that is, signals consisting of the same frequency components). Waveform) is output. Here, as described above, the gain corresponds to the amplitude of the signal waveform and is a value related to the sound pressure. In this model, the direction of the sound image synthesized by the correlation signals output from both channel signals is determined by the balance of the sound pressures of both of the correlation signals.

  In this example, instead of directly assigning to the reproduction speaker in step S34, assignment to the virtual sound source is first performed. This description is basically omitted because it can be easily understood by replacing “reproducing speaker” with “virtual sound source”. As described above, the direction of the synthesized sound image generated from the correlation signal estimated for each line spectrum is estimated as pre-processing for this assignment.

  In addition, the case where the virtual sound source is arranged on a straight line like the reproduction speaker of FIG. However, as shown in FIG. 12, another example of the positional relationship between the listener, the left and right speakers, and the virtual sound source, the virtual sound sources 121a to 121e are arranged with the speaker arrangement circles in FIG. 9 (circles indicated by broken lines in FIG. 12). ) And the concentric circular arcs, the signal can be assigned to the virtual sound source in the same manner by arranging the listener 122 at the center point, for example.

  In the above, the case where attention is paid to the pair of the left front speaker 91 and the right front speaker 92 in FIG. By this virtual sound source assignment, the k-th line spectrums of the audio signals of the left front speaker 91 and the right front speaker 92 are arranged on a straight line in the same manner as the reproduction speakers 52a to 52e in FIG. Although assigned to the twelve virtual sound sources 121a to 121e, the k-th line spectrum of each of the audio signals of the left surround speaker 94 and the right surround speaker 95 in FIG. 10 has not yet been assigned.

  These assignments will be described with reference to FIG. FIG. 13 is a schematic diagram for explaining an example of a positional relationship between a listener, left and right speakers, and left and right surround speakers and a virtual sound source. As shown in FIG. 13, the k-th line spectrum of the audio signal of the left surround speaker 94 is assigned to the virtual sound source 131 a in the same direction as the left surround speaker 94 when viewed from the listener 132. Similarly, the k-th line spectrum of the audio signal of the right surround speaker 95 is assigned to the virtual sound source 131b in the same direction as the right surround speaker 95 when viewed from the listener 132.

  As described above, in step S34, four channels are assigned to the virtual sound source for the k-th line spectrum. Here, as a result of the target pair determination process, the correlation signal and the uncorrelated signal are separated by focusing on the left front speaker 91 and the right front speaker 92 in FIG. May be different. Such a case will be described with reference to FIGS. 14 to 16 are schematic diagrams for explaining other examples of the positional relationship between the listener, the left and right speakers, and the left and right surround speakers and the virtual sound source.

  As a result of the focused pair determination process, for example, when the result is that the left front speaker 91 and the left surround speaker 94 are focused, the k-th line for the left front speaker 91 and the left surround speaker 94 as shown in FIG. Spectral correlation signals and uncorrelated signals are assigned to the virtual sound sources 141a to 141e, the kth line spectrum of the right front speaker 92 is assigned to the virtual sound source 141f, and the kth line spectrum of the right surround speaker 95 is assigned to the virtual sound source 141g. Assigned to each.

  The same applies when the pair of interest is another pair. When the result of focusing attention on the left surround speaker 94 and the right surround speaker 95 is, as shown in FIG. 15, the correlation signal and the uncorrelated signal of the kth line spectrum for the left surround speaker 94 and the right surround speaker 95 are obtained. The kth line spectrum of the left front speaker 91 is assigned to the virtual sound source 151f, and the kth line spectrum of the right surround speaker 95 is assigned to the virtual sound source 151g. Further, when the result of paying attention to the right surround speaker 95 and the right front speaker 92 is, as shown in FIG. 16, the correlation signal of the kth line spectrum and the non-correlation with respect to the right surround speaker 95 and the right front speaker 92. Signals are assigned to the virtual sound sources 161a to 161e, the kth line spectrum of the left surround speaker 94 is assigned to the virtual sound source 151f, and the kth line spectrum of the left front speaker 91 is assigned to the virtual sound source 151g.

  The process as described above is performed for all line spectra by the loop of steps S31a and S31b. For example, when 256 discrete Fourier transforms are performed, all the points of the segment (1024 points) up to the 1st to 127th line spectrum, and when 512 discrete Fourier transforms are performed, the 1st to 255th line spectrum. When the discrete Fourier transform is performed for, the first to 511th line spectra are obtained.

As a result, if the number of virtual sound sources 171 as shown in FIG. 17 is J (J = 16 in this example), output audio signals Y ₁ (k) in the frequency domain for each virtual sound source (output channel),... , Y _J (k) is obtained. These outputs become the output results of the separation and extraction unit 22 in FIG.

  Here, FIG. 17 is a schematic diagram for explaining an example of the positional relationship between the left and right speakers and the left and right surround speakers and all virtual sound sources, and FIG. 18 is a schematic diagram for explaining an example different from FIG. FIG. In the example of FIG. 17, 16 virtual sound sources 171 are arranged concentrically with a circle connecting the speakers 91 to 95 in FIG. 9. However, as in the example of FIG. In this example, they may be arranged on a trapezoid. The example of FIG. 18 is an arrangement example in which the virtual sound sources 181 are arranged in a straight line on each of the four sides. In FIG. 5, the virtual sound sources arranged at the positions of the reproduction speakers 52a to 52e are displayed for each side. It is a combination.

  Further, here, the conversion processing of the signal of the 5.1ch surround system has been described as an example, but the conversion processing can be similarly performed in the 6.1ch or 7.1ch surround system. This point will be described with reference to FIGS. FIG. 19 is a diagram illustrating an arrangement example of output target speakers after downmixing among the six speakers excluding LFE in the speaker group of the 6.1ch surround system. FIG. 20 is a diagram illustrating an arrangement example of output target speakers after downmixing among seven speakers excluding LFE in the speaker group of the 7.1ch surround system.

Also in the 6.1ch system, the center (C) channel is downmixed into the L channel and the R channel, and each of the 5 channels of L / R / L _S / R _S / C _B as indicated by the arrows in FIG. In consideration of this pair, the same processing as described above may be performed. Also in 7.1ch system, downmixed center (C) channels L and R channels, among the _{L / R / L S / R} S / L B / of R _B 6ch, as indicated by arrows in FIG. 20 In consideration of each channel pair, the same processing as described above may be performed.

  Thus, it is preferable that the input audio signal is a 5.1ch, 6.1ch, or 7.1ch surround sound signal, and one channel to be downmixed is the input audio signal of the front center channel. This is because sound pressure panning is performed between the front center channel and the left and right front channels as described above, but an appropriate sound image can be obtained in this example even in such a case. Since the subwoofer audio signal is usually processed separately, the input audio signal can be regarded as an acoustic signal obtained by removing the subwoofer channel from any of the surround sound signals. Then, after down-mixing the center channel of such a surround sound signal, the pair of interest is determined from the adjacent channel pairs for each line spectrum or for each small band, and the correlation signal is obtained only for the pair of interest. By performing the uncorrelated signal separation, even if sound pressure panning is performed between the left and right front channels, the surround sound signal can be reproduced as an appropriate sound image by the wavefront synthesis reproduction method. Further, not only these surround sound signals but also a surround sound signal such as 9.1ch can be applied as an input sound signal.

As described above, the processing of the separation and extraction unit 22 in this example is performed.
Next, processing of the inverse transform unit 23 is performed. The inverse conversion unit 23 performs the correlation signal (or the correlation signal and the non-correlation signal) on the combination of interest extracted by the separation extraction unit 22 or the audio signal generated from the correlation signal. Alternatively, an inverse discrete Fourier transform is performed on the speech signal generated from the correlated signal and the uncorrelated signal. The inverse transform unit 23 also performs discrete Fourier inverse transform on the audio signal transformed by the transform unit 21 for channels other than the noted combination.

Specifically, the inverse transform unit 23 performs discrete Fourier inverse transform on each output channel output from the separation / extraction unit 22, so that the time-domain output speech signal y ′ _J (m) is obtained according to the above equation (35). Ask for. Here, the discrete Fourier inverse transform is performed on the correlation signal and the non-correlation signal for the noted combination, and the discrete Fourier is applied to the audio signal converted by the conversion unit 21 for the channels other than the noted combination. Although an example of performing the inverse transformation is given, the same applies to other cases.

As described in equations (36) and (37) with respect to equation (35), the signal obtained by inverse transformation is also obtained in this example because the signal subjected to the discrete Fourier transform is a signal after window function multiplication. ′ _J (m) is also multiplied by the window function. Therefore, the signal obtained in this way is multiplied by the window function shown in Equation (2) again, and added to the output buffer while shifting by 1/4 segment length from the head of the previous processed segment. To obtain the converted data.

  Next, generation of audio signals for each speaker will be described with reference to FIGS. FIG. 21 illustrates a speaker that outputs sound corresponding to each virtual sound source when a virtual sound source is provided behind a group of speakers arranged on one straight line in the technique described in Non-Patent Document 1. It is a schematic diagram. 22 to 24 are schematic views for explaining an example of arrangement of speaker groups in the audio signal reproduction device of FIG.

  In the audio output signal generation unit 24 in this example, the audio signal of each virtual sound source generated by the inverse conversion unit 23 is generated according to the technique described in Non-Patent Document 1 described above. However, in the technique described in Non-Patent Document 1, when virtual sound sources 212a to 212e are behind a speaker group (speaker array) 211 arranged on one straight line, as shown in FIG. In order to determine which virtual sound source is to be output ”, a vertical line drawn from the virtual sound source (shown for the virtual sound source 212b) to a straight line indicating the arrangement direction of the speaker array 211, and the virtual sound source 212b and the speaker A method is adopted in which the speaker outputs the sound of the virtual sound source 212b when the angle ψ formed by the connected lines is smaller than a certain value.

  However, the virtual sound source arrangement example applied in FIGS. 12 to 16 and the virtual sound source arrangement example described with reference to FIGS. 17 and 18 require speaker groups arranged so that at least a part thereof is not aligned. Yes, for example, the arrangement of each virtual sound source 222 and each speaker 221 as illustrated in FIG. 22 is required. Therefore, in the method described in Non-Patent Document 1, the sound of a certain virtual sound source (for example, the virtual sound source 222a) is also output from the speaker group 221b facing it, which adversely affects the sound image localization.

  Therefore, for example, to determine which speaker of the speaker group 221 outputs the sound of the virtual sound source 222a in FIG. 22, the center point 224 of the circle in which each virtual sound source 222 is arranged and the virtual sound source 222a are connected. When the angle ψa formed by the straight line and the straight line connecting the speaker to be determined (for example, the speaker 221a in FIG. 22) and the center point 224 is smaller than a certain value, the speaker 221a outputs the sound of the virtual sound source 222a. judge. By performing this for all combinations of speakers and all virtual sound sources, it is possible to determine the audio signal output by each speaker even in the case of the arrangement as shown in FIG.

  In this way, the audio signal processing unit 13 converts the input audio signals of five or more channels of the multi-channel reproduction method to be reproduced by the speaker group as a sound image for a virtual sound source that is a virtually existing sound source. be able to. Here, as illustrated in FIG. 22 and illustrated in FIG. 23 and FIG. 24 described later, the speaker group in this example is arranged so as not to be aligned in a straight line at least partially. The speaker to be output is determined by the angle formed by the straight line connecting the center point.

  Further, as exemplified in FIG. 23, even when the speaker groups 231 are arranged on the circumference, it is possible to determine an audio signal for each virtual sound source 232 by using the same method.

  Further, for example, when the above processing is performed with the restriction that the pair of the left surround speaker 94 and the right surround speaker 95 in FIG. 10 is not focused, the virtual sound source to which the audio signal is assigned is as shown in FIG. The virtual sound sources 242 (in this example, 13 virtual sound sources 242) are limited, and the virtual sound sources 242 may be output by the speaker group 241 arranged not only on the entire periphery but on only three sides as shown in FIG. Is possible.

  As described above, the processing performed by the audio signal processing unit 13 in this example is a process of assigning to a virtual sound source (or a real speaker) after separating only a pair of interest into a correlated signal and an uncorrelated signal. This process means that an assumption is made that there is one sound image belonging to the same line spectrum or small band. Under the assumption, a conversion process for reproducing an accurate sound image position is possible. Therefore, by such conversion processing, it is possible to convert the input audio signals of five or more channels of the multi-channel reproduction method into audio signals that can provide an appropriate sound image when reproduced using the speaker group.

In particular, in the present invention, the pair of interest is determined using the power magnitude, or using the power magnitude and the correlation coefficient (that is, the correlation coefficient magnitude). Therefore, for the frequency component included in the downmix source channel signal (C signal in this example), the power of the downmix destination pair becomes larger than the power of the other pair, and the downmix destination pair is focused. Determined to be a pair. Therefore, even when sound pressure panning is performed between the center channel signal and the left front channel signal, or between the center channel signal and the right front channel signal, the sound image can be converted into an appropriate sound image. Of course, for other frequency components (in this example, frequency components that are not included in the C signal and are included in the original L, R, L _S , and R _S signals), other pairs are selected. As a result, the sound image can be similarly converted into an appropriate sound image.

  Next, the implementation of the present invention will be briefly described. The present invention can be used for an apparatus with an image such as a home theater system or a mini theater system. FIG. 25 is a diagram illustrating a configuration example of a video display system including the audio signal reproduction device of FIG. As in a room 250 shown in FIG. 25, the audio signal reproducing device according to the present invention can be applied to a speaker system in which speaker groups 251 to 253 are arranged side by side on three sides (or four sides) of a room wall. Then, as illustrated in FIG. 25, this speaker system is connected to a video display device 254 such as a television device, and audio corresponding to the video displayed on the video display device 254 may be output from the speaker groups 251 to 253. it can.

Furthermore, also in this example, θ illustrated in FIG. 4 and P _S ^(k) of Expression (14 ⁾ are obtained for each line spectrum (or for each small band). Therefore, also in this example, if a plurality of light emitting units are provided in an array speaker arranged in three or four directions (or circular shape) such that LEDs 255 to 257 are arranged beside each speaker of the speaker groups 251 to 253, As described with reference to FIG. 8, it is possible to change the emission color or change the emission intensity according to the sound intensity in three or four directions (or in a circular shape). Further, only in front of the listener, the bar graph 72 illustrated in FIG. 7 can be displayed on the video display device 254 instead of or in addition to the LED 256.

  As the wavefront synthesis reproduction method applicable in each of the above-described examples, any method may be used as long as it includes a speaker array (a plurality of speakers) and outputs sound images from virtual speakers as described above. In addition to the WFS method described in Non-Patent Document 1, there are various methods such as a method using a preceding sound effect (Haas effect) as a phenomenon related to human sound image perception. Here, the preceding sound effect means that if the same sound is played from multiple sound sources and each sound reaching the listener from each sound source has a small time difference, the sound image is localized in the sound source direction of the sound that has arrived in advance. It points out the effect to do. If this effect is used, a sound image can be perceived at the virtual sound source position. However, it is difficult to clearly perceive the sound image only by the effect. Here, humans also have the property of perceiving a sound image in the direction in which the sound pressure is felt highest. Therefore, in the audio signal reproducing apparatus, the preceding sound effect described above and the effect of perceiving the maximum sound pressure direction are combined, so that a sound image can be perceived in the direction of the virtual sound source even with a small number of speakers.

  As described above, various examples of the audio signal reproduction device according to the present invention have been described on the assumption that the input audio signal is a multi-channel reproduction system. A supplemental description will be given of a case where the input audio signal is a wavefront synthesis reproduction type audio signal. In this case, the signal power value (or sound level) is adjusted to a position that matches the sound source position information indicated by the input sound signal without performing processing such as separation of the correlation signal as in the sound image display processing that is the main feature of the present invention. (Pressure level) may be displayed, and sound may be output from the speaker array by, for example, a wavefront synthesis reproduction method. Thereby, even when the input sound signal is a sound signal of the wavefront synthesis reproduction method, it is possible to display the sound image position so that the sound image position matches between hearing and perception.

  In addition, each component of the audio signal reproduction device according to the present invention such as each component in the audio signal processing unit 13 and the display control unit 18 illustrated in FIG. 1 includes, for example, a microprocessor (or DSP: Digital Signal Processor), This can be realized by hardware such as a memory, a bus, an interface, and a peripheral device, and software that can be executed on these hardware. Except for the display unit 19, a part or all of the hardware can be mounted as an integrated circuit / IC (Integrated Circuit) chip set. In this case, the software may be stored in the memory. In addition, all the components of the present invention may be configured by hardware, and in that case as well, part or all of the hardware can be mounted as an integrated circuit / IC chip set. .

  In addition, a recording medium on which a program code of software for realizing the functions in the various configuration examples described above is recorded is supplied to a device such as a general-purpose computer serving as an audio signal reproduction device, and is then processed by a microprocessor or DSP in the device. The object of the present invention is also achieved by executing the program code. In this case, the software program code itself realizes the functions of the above-described various configuration examples. Even if the program code itself or a recording medium (external recording medium or internal storage device) on which the program code is recorded is used. The present invention can be configured by the control side reading and executing the code. Examples of the external recording medium include various media such as an optical disk such as a CD-ROM or a DVD-ROM and a non-volatile semiconductor memory such as a memory card. Examples of the internal storage device include various devices such as a hard disk and a semiconductor memory. The program code can be downloaded from the Internet and executed, or received from a broadcast wave and executed.

  The audio signal reproduction apparatus according to the present invention has been described above. However, as illustrated in the flowchart of the processing flow, the present invention can also take the form of an audio signal reproduction method for reproducing an audio signal. This audio signal reproduction method includes the following separation step, calculation step, addition step, and sound image display step.

  The separation step is a step in which the separation unit separates the two audio signals into a correlation signal and a non-correlation signal for each frequency or frequency band. The calculation step is a step in which the calculation unit calculates a sound image direction and a signal power value for each correlation signal separated in the separation step. In the adding step, for each correlation signal, the adding unit assigns a signal power value to one of a plurality of predetermined sound image direction groups based on the sound image direction, and adds the signal power value for each sound image direction group. This is a step for obtaining. The sound image display step is a step in which the sound image display unit displays information indicating an addition value for each sound image direction group at a position corresponding to each of the plurality of sound image direction groups. Other application examples are the same as those described for the audio signal reproducing apparatus, and the description thereof is omitted.

  In other words, the program code itself is a program for causing a computer to execute the audio signal reproduction method. That is, this program causes a computer to separate two audio signals into a correlation signal and a non-correlation signal for each frequency or frequency band, and for each correlation signal separated in the separation step, A calculation step for calculating a sound image direction and a signal power value, and for each correlation signal, a signal power value is assigned to one of a plurality of predetermined sound image direction groups based on the sound image direction, and a signal is generated for each sound image direction group. This is a program for executing an addition step for obtaining an addition value of power values, and a sound image display step for displaying information indicating the addition value for each sound image direction group at a position corresponding to each of the plurality of sound image direction groups. . Other application examples are the same as those described for the audio signal reproducing apparatus, and the description thereof is omitted.

  As described above, the audio signal reproduction device according to the present invention separates two audio signals into a correlated signal and a non-correlated signal for each frequency or frequency band, and the separating unit. A calculation unit for calculating a sound image direction and a signal power value for each separated correlation signal, and the signal power for one of a plurality of predetermined sound image direction groups based on the sound image direction for each correlation signal. An adder for assigning a value and obtaining an added value of the signal power value for each sound image direction group, and displaying information indicating the added value for each sound image direction group at a position corresponding to each of the plurality of sound image direction groups And a sound image display unit. As a result, the sound image position can be displayed so that the sound image position matches between hearing and perception, and the listener's immersive feeling can be enhanced.

  The sound image display unit may display information indicating the added value for each sound image direction group at a position corresponding to each of the plurality of sound image direction groups in a bar graph format. Thereby, a listener's immersion feeling can be increased more.

  Alternatively, the sound image display unit includes a plurality of light emitting units, and the light emission color of the light emitting unit at a position corresponding to each of the plurality of sound image direction groups is used as information indicating the added value for each sound image direction group. You may make it change according to it. By changing the emission color according to the intensity of the sound, for example, when playing music video or playing music, it is possible to give the listener the feeling at the live venue.

  Alternatively, the sound image display unit includes a plurality of light emitting units, and the light emission intensity of the light emitting unit at a position corresponding to each of the plurality of sound image direction groups is used as information indicating the added value for each sound image direction group. You may make it change according to it. By changing the light emission intensity in accordance with the intensity of the sound, for example, when playing music video or playing music, it is possible to give the listener the feeling at the live venue.

  In the audio signal reproduction method according to the present invention, the separation unit separates the two audio signals into a correlation signal and a non-correlation signal for each frequency or frequency band, and a calculation unit includes the separation A calculation step for calculating a sound image direction and a signal power value for each correlation signal separated in the step, and an adding unit, for each correlation signal, of a plurality of predetermined sound image direction groups based on the sound image direction An addition step of assigning the signal power value to one and obtaining an addition value of the signal power value for each sound image direction group, and a sound image display unit at each sound image direction at a position corresponding to each of the plurality of sound image direction groups And a sound image display step for displaying information indicating the added value for the group. As a result, the sound image position can be displayed so that the sound image position matches between hearing and perception, and the listener's immersive feeling can be enhanced.

  A program according to the present invention includes a separation step of separating two audio signals into a correlation signal and a non-correlation signal for each frequency or frequency band, and each correlation signal separated in the separation step. Calculating a sound image direction and a signal power value, and assigning the signal power value to one of a plurality of predetermined sound image direction groups based on the sound image direction for each correlation signal, An addition step for obtaining an addition value of the signal power values for each direction group; and a sound image display step for displaying information indicating the addition value for each sound image direction group at a position corresponding to each of the plurality of sound image direction groups; Is a program for executing Thereby, the function of the present invention can be provided as a program.

  A recording medium according to the present invention is a computer-readable recording medium on which the above program is recorded. Thus, the program can be distributed on the recording medium.

DESCRIPTION OF SYMBOLS 10 ... Audio | voice signal reproduction apparatus, 11 ... Decoder, 12 ... Audio signal extraction part, 13 ... Audio signal processing part, 14 ... D / A converter, 15 ... Amplifier, 16, 16a, 16b, 16c ... Speaker, 17 ... Speaker array , 18 ... display control unit, 18a ... addition unit, 19 ... display unit, 21 ... conversion unit, 22 ... separation / extraction unit, 22a ... separation unit, 22b ... calculation unit, 23 ... inverse conversion unit, 24 ... audio output signal generation , 70 ... TV device, 71 ... Display panel, 72 ... Bar graph, 73a, 73b, 73c, ... Information, 81a, 81b, 81c ... LED.

Claims (5)

The two audio signals, and a separation unit for separating the No. phases SekiShin for each frequency or each frequency band,
For each correlation signal separated by the separation unit, a calculation unit for calculating a sound image direction and a signal power value;
Depending on the signal power values assigned to the previous SL plurality of sound directions groups defined Me based-out pre sound image direction about the respective correlation signal, performs a display corresponding to each of the plurality of sound image direction Group A sound image display section;
An audio signal reproducing apparatus comprising: The sound image display unit, the audio signal reproducing apparatus according to claim 1, wherein the table Shimesuru a bar graph format. 3. The audio signal reproduction device according to claim 1, wherein the sound image display unit includes a plurality of light emitting units, and changes a light emission color of the light emitting unit according to the assigned signal power value. 4. . The sound image display section includes a plurality of light emitting portions, the audio signal reproducing apparatus according to claim 1 or 2, characterized in that to change the emission intensity of the light emitting unit in accordance with the assigned signal power value . Separation section, the two audio signals, and a separation step of separating the correlation signal for each frequency or each frequency band,
A calculation unit that calculates a sound image direction and a signal power value for each correlation signal separated in the separation step ;
The sound-image display unit is performed, the display corresponding to each of the plurality of sound image direction groups corresponding to the signal power values assigned to the plurality of sound image direction Group predetermined based on the sound image direction for each correlation signal A sound image display step;
A method of reproducing an audio signal, comprising:

Images