JP6296072B2 - Sound reproduction apparatus and program - Google Patents

Description

  The present invention relates to a sound reproduction device and a program, and can be applied to three-dimensional reproduction of sound signals from stereo speakers.

  Conventionally, there is binaural reproduction using a head-related transfer function (HRTF) as a technique for reproducing a sense of presence as if the sound image is localized arbitrarily and as if it were present. HRTF is a transmission characteristic of sound from the sound source to the ear. A dedicated microphone is attached to the ear of a person or a dummy head, and the sound source is placed in various directions for measurement. In binaural playback, a three-dimensional sound effect can be created by convolving the HRTF in the direction to be localized in the sound source, converting it to a binaural sound source, and playing it from headphones or earphones. However, when a binaural sound source is reproduced as it is from a speaker, a sufficient stereophonic effect cannot be obtained. The binaural sound source for the right ear needs to reach only the right ear, but when played from the speaker, the binaural sound source for the right ear enters not only the right ear but also the left ear. Similarly, the left speaker The binaural sound source for the left ear reproduced from the above will enter the right ear as well as the left ear. Such a phenomenon is called crosstalk and is a cause of hindering the stereophonic effect.

  Conventionally, there is a technique called trans-oral reproduction that can achieve the same effect as binaural reproduction even with reproduction by a speaker (see Non-Patent Document 1 and Patent Document 1). In the conventional binaural reproduction method, after measuring the room transfer function from each speaker to both ears, the transfer function is convolved with the binaural sound source, and a process of designing a filter that cancels only the crosstalk component therein is performed. In the conventional binaural reproduction method, this filter is applied to a sound source for sound image localization and reproduced from a speaker. Thereby, in the conventional binaural reproduction method, the crosstalk component is canceled at the listener's ear, and only the left and right binaural sound sources reach the ear, and the same stereophonic effect as in binaural reproduction can be obtained. In the transoral reproduction technique described in Non-Patent Document 1, with respect to sound reproduction by a speaker, only a crosstalk component can be canceled and a three-dimensional sound effect can be obtained similarly to binaural reproduction.

  However, the conventional transoral reproduction technique has a problem that the listening position (sweet spot) where the three-dimensional sound effect can be obtained is narrow. Therefore, if the listener changes his / her head, front / rear / left / right, and the direction of the face, the stereophonic effect will be lost. To deal with this problem, Patent Document 1 prepares three or more speakers, uses a camera to constantly analyze the position and orientation of the listener's face, and selects a speaker and transfer function suitable for the listener at that time. A method for transoral reproduction is proposed.

JP 2013-110633 A

WG Gardner, “3-D Audio Using Loudspeakers”, Springer, 1997, [Online], INTERNET, [searched January 28, 2016], <URL: sound.media.mit.edu/Papers/gardner_thesis.pdf>

  However, the method described in Patent Document 1 requires a separate device for performing processing for detecting the position and orientation of the listener's face in real time. In addition, since three or more speakers and a camera are required, there is a problem that the place where the 3D sound effect can be experienced becomes limited because of a large-scale system.

  Therefore, there is a demand for a sound reproducing device and a program that use a plurality of speakers as sound sources and perform processing for localizing sound sources in a predetermined direction with high accuracy.

According to a first aspect of the present invention, there is provided a sound reproducing apparatus for generating a stereophonic signal to be supplied to each of a plurality of speakers by performing stereophonic processing on an input acoustic signal. Using a head-related transfer function held by the head-related transfer function held by the head-related transfer function held by the head-related transfer function; A first stereophonic signal generator that generates a first stereoacoustic signal in which the sound source is localized in the sound image localization direction; and (4) crossing from the first stereoacoustic signal to each of the speakers. a holds the crosstalk cancellation filter to remove talk component, click to hold the crosstalk cancellation filter based on the parameters the information acquisition unit corresponding to the acquired sound image localization direction (5) For each of the speakers, a crosstalk component is removed from the first stereophonic sound signal by using a crosstalk cancellation filter held by the crosstalk cancellation filter holding unit. And a second stereophonic signal generator for generating two stereoacoustic signals.

The sound reproduction program according to the second aspect of the present invention provides a computer mounted on an audio reproduction device that generates a three-dimensional sound signal to be supplied to each of a plurality of speakers by performing three-dimensional sound processing on an input sound signal. A head-related transfer function holding unit that holds a head-related transfer function corresponding to the direction; (2) an information acquisition unit that acquires information on a sound image localization direction that localizes at least a sound source; and (3) the head-related transfer function holding unit. A first stereophonic signal generation unit that generates a first stereoacoustic signal in which the sound source is localized in the sound image localization direction using the head-related transfer function held by (1), and (4) for each of the speakers, a holds the crosstalk cancellation filter to remove crosstalk components from the first stereophonic signal, based on a parameter corresponding to the sound image localization direction in which the information obtaining unit has obtained black A crosstalk cancellation filter holding unit for holding a talk cancellation filter; and (5) using the crosstalk cancellation filter held by the crosstalk cancellation filter holding unit for each of the speakers, from the first stereophonic sound signal. It is made to function as a 2nd stereophonic signal generation part which removes a crosstalk component and generates the 2nd stereophonic signal.

  ADVANTAGE OF THE INVENTION According to this invention, the sound reproducing apparatus and program which perform accurately the process which uses a several speaker as a sound source and localizes a sound source in a predetermined | prescribed direction can be provided.

It is the block diagram shown about the functional structure of the sound reproduction apparatus which concerns on 1st Embodiment. It is explanatory drawing shown about the positional relationship between the speaker and user in 1st Embodiment, and the structure of the transfer function. It is explanatory drawing shown about the structure of the experimental environment which verified the effect of the sound reproduction apparatus which concerns on 1st Embodiment. It is explanatory drawing (the 1) shown about the experimental result which verified the effect of the sound reproduction apparatus which concerns on 1st Embodiment. It is explanatory drawing (the 2) shown about the experimental result which verified the effect of the sound reproduction apparatus which concerns on 1st Embodiment. It is the block diagram shown about the functional structure of the sound reproduction apparatus which concerns on 2nd Embodiment. It is the block diagram shown about the functional structure of the sound reproduction apparatus which concerns on 3rd Embodiment. It is explanatory drawing shown about the example of the process which replaces a head-related transfer function with the transfer function between a speaker and a user in the audio | voice reproduction apparatus which concerns on 3rd Embodiment.

(A) First Embodiment Hereinafter, a first embodiment of a sound reproducing device and a program according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of the First Embodiment FIG. 1 is a block diagram showing the overall configuration of the sound reproducing device 10 of this embodiment.

  The sound reproduction apparatus 10 of this embodiment includes a data input unit 11, a position information acquisition unit 12, a filter formation unit 13, an HRTF holding unit 14, a transfer function holding unit 15, a stereophonic sound processing unit 16, an output unit 17, and two It has speakers (right speaker 18R, left speaker 18L).

  The sound reproducing device 10 may be configured entirely using hardware (for example, a dedicated semiconductor chip), or may be configured in software for a part or all of arithmetic processing (data processing and signal processing). You may do it. For example, the sound reproduction device 10 may be realized by installing the sound reproduction program according to the embodiment in the execution configuration (computer) of a program having a processor and a memory for each component other than the speaker.

  The sound reproducing device 10 is a device that generates a signal (hereinafter referred to as “stereoscopic sound signal”) obtained by performing stereophonic sound processing on an input acoustic signal (hereinafter referred to as “input acoustic signal”). In this embodiment, it is assumed that the sound reproduction device 10 is configured to output the generated stereophonic signal from two speakers (the right speaker 18R and the left speaker 18L) to the user U who is a listener. The method by which the sound reproduction device 10 outputs a stereophonic sound signal is not limited. For example, it is output as digital sound data (sound data) (for example, output by writing to a predetermined data recording medium or transmitting data by communication). You may make it do.

  FIG. 2 shows the positional relationship between the speakers (the right speaker 18R and the left speaker 18L) and the user U (the positional relationship when viewed from above) and between the speakers (the right speaker 18R and the left speaker 18L) and the user U. It is explanatory drawing shown about the transfer function on the path | route.

  As shown in FIG. 2, the right speaker 18 </ b> R is a speaker disposed on the right side when viewed from the user U, and the left speaker 18 </ b> L is a speaker disposed on the left side when viewed from the user U.

  For example, in a conference terminal such as a video conference system, the sound reproduction device 10 performs stereophonic sound processing on an audio signal (far-end speaker's audio signal) captured by a remote microphone (not shown) and outputs the sound signal. is there. For example, as shown in FIG. 2, a sound (for example, displayed on the display D) subjected to sound image localization according to an image (for example, an image photographed by a remote camera) displayed on a display arranged in front of the user U. Sound that is localized according to the position of the far-end speaker is output), so that a realistic sound (stereoscopic sound) can be output to the user U.

  In this embodiment, the sound reproducing device 10 is configured to localize (sound image localization) the sound source of the input acoustic signal (hereinafter referred to as “sound image localization direction”) and the position information of the user U (hereinafter referred to as “user position information”). 3) and position information of each speaker (hereinafter also referred to as “speaker position information”), and performing stereophonic sound processing to localize the sound source of the input sound signal in the specified sound image localization direction, It is assumed that the device emits (outputs) toward the user U. Note that the specific method, input timing, and data format of the sound image localization direction, user position information, and speaker position information held by the sound reproduction device 10 are not limited. For example, the sound reproduction device 10 may be a device that receives input of each of the above-described information on a conference terminal of a conference system (not shown) and outputs a sound based on the input sound signal. In addition, the use of the sound reproducing device 10 is not limited to the conference terminal.

  The data input unit 11 has a function of converting an input acoustic signal from an analog signal to a digital signal (A / D conversion). In this embodiment, the input acoustic signal is described as an analog signal, but may be a digital signal. When the input acoustic signal is in a digital format, the data input unit 11 performs data conversion of the digital signal or the digital signal and outputs it to the subsequent stage. For example, when the input acoustic signal is input in a packet format according to a protocol such as RTP (Real-Time Transport Protocol), the data input unit 11 buffers the input packet to continuously input audio data (for example, You may make it output as audio | voice data (PCM (Pulse Code Modulation) format etc.). Here, it is assumed that the audio data output from the data input unit 11 is monaural (one channel) audio data.

  Further, the input acoustic signal supplied to the data input unit 11 is not limited to real-time data, and data (offline data) recorded in advance on a data recording medium (not shown) (for example, a hard disk drive or a flash memory) is used. It may be read and output to the subsequent stage as a continuous digital sound signal.

  The position information acquisition unit 12 has a function of acquiring information necessary for the stereophonic sound processing of the input sound signal and supplying the information to the filter forming unit 13. The position information acquisition unit 12 acquires a sound image localization direction, user position information, and speaker position information. The method by which the position information acquisition unit 12 acquires information and the timing at which information is acquired are not limited. For example, the position information acquisition unit 12 may acquire information in real time, may acquire preset data, or may store information (for example, information recorded in a data recording medium (not shown) in advance). , Information paired with data of the input acoustic signal) may be obtained (updated) at a predetermined timing.

  Next, an example of each piece of information acquired by the position information acquisition unit 12 will be described with reference to FIG.

In this embodiment, the position information acquisition unit 12 holds at least position information of the position P _eR of the right ear e _R of the user U and position information of the position P _eL of the left ear e _L as the user position information. To do. Note that the position information acquisition unit 12 does not directly acquire the positions P _eR and P _eL , for example, the position information on the center position P _U of the head of the user U and the information on the direction in which the user U is facing. The position P _eR and P _eL of the user's ears may be acquired and estimated (calculated). As described above, the method by which the position information acquisition unit 12 acquires the information on the positions P _eR and P _eL of each ear as the user position information is not limited.

Further, in this embodiment, the position information acquisition unit 12, a speaker position information, the position of the right speaker 18R (positions when viewed from above as in FIG. 2) and the position information of the P _SR, the left speaker 18L _Assume that the position _PSL is acquired. In this embodiment, the positions P _SR and P _SL are the center positions of the respective speakers (the right speaker 18R and the left speaker 18L) (center positions when viewed from above as shown in FIG. 2).

In the following, as shown in FIG. 2, the sound image localization direction is a two-dimensional plane (on a plane when viewed from above; on a horizontal plane viewed from the user U), and the direction in which the user U is facing. And the angle formed by the sound image localization direction is represented by θ _S. Note that the sound image localization direction input to the sound reproducing device 10 may include a vertical component viewed from the user U.

The HRTF holding unit 14 has a function of holding the HRTF corresponding to the direction of each sound source (sound image localization direction θ _S ) and supplying the HRTF to the filter forming unit 13. The HRTF holding unit 14 may hold the HRTF for each predetermined grid width (for example, 1 °) in advance, or calculate and acquire the HRTF whenever requested by the filter forming unit 13. It may be. Data acquired by various HRTF calculation methods can be applied to the HRTF data itself held by the HRTF holding unit 14. In this embodiment, it is assumed that the HRTF holding unit 14 holds the HRTF for the right ear and the HRTF for the left ear in the direction of each sound source. The HRTF data held by the HRTF holding unit 14 may be the same as the HRTF used in various binaural playback and transoral playback processes.

  The transfer function holding unit 15 has a function of holding a transfer function corresponding to the positional relationship between the user U and each speaker (the right speaker 18R and the left speaker 18L) and supplying the transfer function to the filter forming unit 13.

  Next, each transfer function held by the transfer function holding unit 15 will be described with reference to FIG.

Hereinafter, the transfer function between the position P _eR of the right ear position P _SR and the user U of the right speaker 18R - (right speaker transfer function of the right ear path), intended to represent the G _RR. In the following, the transfer function between the position P _eL the left ear position P _SR and the user U of the right speaker 18R - (right speaker transfer function of the left ear path), intended to represent the G _RL. Furthermore, hereinafter, the transfer function between the position P _SL of the left speaker 18L and the position P _eR of the right ear of the user U (left speaker-right ear path transfer function) is represented as G _LR . Furthermore, hereinafter, the transfer function between the position P _SL of the left speaker 18L and the position P _eL of the left ear of the user U (left speaker-left ear path transfer function) is expressed as G _LL .

In this embodiment, the transfer function holding unit 15 holds the transfer functions G _RR , G _RL , G _LR , and G _LL and supplies them to the filter forming unit 13. The transfer function holding unit 15 may hold the transfer functions G _RR , G _RL , G _LR , G _LL in advance, or calculate (for example, user position information) in response to a request from the filter forming unit 13. And may be calculated based on the speaker position information. The calculation method of the transfer functions G _RR , G _RL , G _LR , and G _LL held by the transfer function holding unit 15 is not limited, and various transfer functions can be applied. As the transfer functions held by the transfer function holding unit 15, the same transfer functions used in various transoral processes can be applied.

  The filter forming unit 13 has a function of holding each filter that is required when the stereophonic sound processing unit 16 in the subsequent stage performs stereophonic sound processing (transoral processing).

Filter forming section 13 obtains the HRTF (filter required for binaural processing of the input audio signal C _i) corresponding to the sound image localization direction from HRTF holding unit 14, and supplies the stereophonic sound processing section 16. Also, the filter forming unit 13 is a filter that cancels the crosstalk component from the binaural sound source (the acoustic signal that the stereophonic sound processing unit 16 has performed binaural processing based on the HRTF data) based on the user position information and the speaker position information. (Hereinafter referred to as “crosstalk cancellation filter”) is generated. Details of the crosstalk cancellation filter generated by the filter forming unit 13 will be described later.

Stereophonic sound processing unit 16, using each filter supplied from the filter forming portion 13 (HRTF and crosstalk cancellation filters) is subjected to a trans-oral treatment with the input audio signal C _i, the acoustic signal to be Transaural source Generated and supplied to the output unit 17.

Stereophonic sound processing unit 16 first performs binaural processing on the input audio signals C _i based on the HRTF, generates an acoustic signal as a binaural sound. Then, the stereophonic sound processing unit 16 multiplies the generated binaural sound source (acoustic signal) by a crosstalk cancellation filter (convolution), generates an acoustic signal as a transaural sound source, and supplies it to the output unit 17.

Hereinafter, the binaural sound source for the right ear is referred to as “B _R ”, and the binaural sound source for the left ear is referred to as “B _L ”. In the following, the stereophonic sound process unit 16, the right speaker 18R obtained as a result of performing a transaural processing sound sources (acoustic signal) T _R, for the left speaker 18L obtained as a result of subjecting the Transaural processing source Let (acoustic signal) be called _TL .

When the input sound signal C _i is composed of sound signals of a plurality of sound sources, the stereophonic sound processing unit 16 generates a trans-oral sound source for each sound source (acoustic signal constituting the input sound signal C _i ), Each trans-oral sound source may be mixed by performing gain adjustment (for example, gain adjustment at a preset ratio) to generate a single trans-oral sound source.

  The output unit 17 distributes and outputs the acoustic signal of the transoral sound source generated by the stereophonic sound processing unit 16 to the speakers 18R and 18L. Note that the output unit 17 may perform signal conversion (for example, conversion from a digital signal to an analog signal) according to the input format of the speakers 18R and 18L and output the converted signals to the speakers 18R and 18L.

  Next, details of the process in which the filter forming unit 13 forms the crosstalk cancellation filter will be described.

First, the filter forming unit 13 acquires HRTF data corresponding to the sound image localization direction from the HRTF holding unit 14. Further, the filter forming unit 13 acquires transfer functions G _RR , G _RL , G _LR , and G _LL corresponding to the user position information / speaker position information from the transfer function holding unit 15.

Then, the filter forming portion 13 is designed transfer function _{G RR,} using a _{G RL, G LR, G LL} , the crosstalk cancellation filter for canceling a crosstalk component from the binaural sound source. The crosstalk cancellation filter convolves the binaural sound sources B _R and B _{L with a} binaural sound source and the room transfer functions G _RR , G _RL , G _LR , and G _LL from the speakers 18R and 18L to both ears of the user U. A filter design that cancels only the crosstalk components (components that reach the left ear of the user U from the right speaker 18R and components that reach the right ear of the user U from the left speaker 18L) is performed.

Hereinafter, the crosstalk cancellation filter for the right speaker 18R is referred to as “C _R (ω)” (“ω” is a frequency, the same applies hereinafter), and the crosstalk cancellation filter for the left speaker 18L is referred to as “C _L (ω)”. Shall. In other words, the crosstalk cancellation filter C _{R (omega)} is subjected to transaural processing binaural sound source B _R, a filter for generating a transaural sound T _R for the right speaker 18R. In addition, the crosstalk cancellation filter C _L (ω) is a filter for generating a trans-oral sound source T _L for the left speaker 18L by performing trans-oral processing on the binaural sound source B _L.

Hereinafter, an example of processing for forming (holding) the crosstalk cancellation filters C _R (ω) and C _L (ω) will be described.

First, the filter forming unit 13 performs the left speaker-left ear path filter C _LL (ω) and the right speaker-right ear path filter C _RR (ω) as in the following formulas (1) to (4). The left speaker-right ear path filter C _LR (ω) and the right speaker-left ear path filter C _RL (ω) are generated (designed). In the following formulas (1) to (4), H _L (ω) and H _R (ω) are held by the HRTF (HRTF holding unit 14) corresponding to the sound image localization directions for the left and right ears, respectively. HRTF). In the following formulas (1) to (4), G _RR , G _RL , G _LR , and G _LL are transfer functions held by the transfer function holding unit 15, respectively. In the following formulas (1) to (4), ω represents a frequency. The common term G ₀ (ω) in the following formulas (1) to (4) is expressed as the following formula (5) when the following formulas (1) to (4) are combined into an equation. Can do.

Here, if the crosstalk cancellation filters C _R (ω) and C _L (ω) for each speaker are obtained by a method similar to that of the conventional transoral reproduction technique (Non-patent Document 1), an arithmetic expression thereof is as follows: (6) and (7).

In this embodiment, the filter forming unit 13 adds a weight according to the positional relationship between the user U and each speaker to some of the elements of the conventional arithmetic expression (the above expressions (6) and (7)). These are calculated as crosstalk cancellation filters C _R (ω) and C _L (ω).

Specifically, in the example of this embodiment, the filter forming unit 13 obtains the crosstalk cancellation filters C _R (ω) and C _L (ω) by the following equations (8) and (9). .

In the following equations (8) and (9), α is a parameter (a parameter that changes the balance of the left and right cancellation amount) that varies according to the sound image localization direction θ _S (the direction of the virtual sound source), and the following (10) It can be shown as:

In the following equation (10), x is a parameter obtained by converting the unit of the sound image localization direction θ _S (virtual sound source direction) from degrees (degrees) to radians (rad). Here, when the front direction of the user is 0 (radian), it is π / 2 if it is 90 ° clockwise (90 ° clockwise), and 90 ° counterclockwise (90 ° left). Then, 3π / 2 is assumed. Therefore, when the sound image localization direction θ _S is 0 °, α = ½, when the sound image localization direction θ _S is 90 degrees to the right, α = 0, and when the sound image localization direction θ _S is 90 degrees to the left, α = −1. Become.

Accordingly, the sound image localization direction θ _S is determined to be 90 ° to the left and 90 ° to the right by obtaining the crosstalk cancellation filters C _R (ω) and C _L (ω) as in the above (8) and (9) using α. Then, it changes like the following (11)-(14) Formula.

The following equations (11) and (12) show the crosstalk cancellation filters C _R (ω) and C _L (ω) when the sound image localization direction θ _S is 90 ° to the left. The following equations (13) and (14) indicate the crosstalk cancellation filters C _R (ω) and C _L (ω) when the sound image localization direction θ _S is 90 ° to the right.

When the crosstalk cancellation filters C _R (ω) and C _L (ω) are obtained in consideration of α that varies according to the sound image localization direction θ _{S as} in the above (8) and (9), the sound image localization is obtained. When the direction θ _S is the left direction (for example, left 90 °), as shown in the following equations (11) and (12), the crosstalk cancellation filter C _R (ω) for the right speaker 18R. The amount of cancellation due to increases. In other words, when the above (8) and (9) are applied, when the sound image localization direction θ _S is the left direction, the crosstalk for the right speaker 18R increases as the angle represented by θ _S increases. Since the amount of cancellation by the cancel filter C _R (ω) increases, the acoustic signal _TL output from the left speaker 18L is more emphasized.

Further, when the above (8) and (9) are applied, when the sound image localization direction θ _S is the right direction (for example, 90 ° to the right), the following expressions (13) and (14) are shown. As described above, the amount of cancellation of the crosstalk cancellation filter C _L (ω) for the left speaker 18L increases. In other words, when the above (8) and (9) are applied and the sound image localization direction θ _S is rightward, the crosstalk for the left speaker 18L increases as the angle represented by θ _S increases. since the cancellation amount by canceling filter C _{L (omega)} increases, so that the acoustic signal T _R output from the right speaker 18R can be more emphasized.

As described above, in the above (8) and (9), when the sound image localization direction θ _S is the left direction (for example, 0 <x <π), the crosstalk cancellation filter for the right speaker 18R. The amount of cancellation by C _R (ω) is increased to emphasize the acoustic signal _TL output from the left speaker 18L. In the above (8) and (9), when the sound image localization direction θ _S is the right direction (for example, when π <x <2π), the crosstalk cancellation filter C _L (for the left speaker 18L) by increasing the cancel amount of omega), highlighting the acoustic signal T _R output from the right speaker 18R. In other words, in the above (8) and (9), α is set so as to increase the amount of crosstalk cancellation for the speaker on the side opposite to the sound image localization direction θ _S side (right side or left side as viewed from the user U). Is set.

In other words, in the above (8) and (9), in addition to the transfer functions G _RR , G _RL , G _LR , and G _LL , the sound image localization direction θ _S is used to adjust the amount of left and right crosstalk cancellation. ing. Thereby, for the user U, since the sound of the sound source on the sound image localization direction θ _S side is more emphasized, even when the head of the user U moves (the position of the ear of the user U moves) It is possible to stabilize the sense of localization of the sound heard by the user U.

In this embodiment, in (8) and (9) above, α (x) is used to adjust the balance (amount) of the left and right crosstalk cancellations, but the same adjustment as described above is possible. For example, the adjustment method of the balance (amount) of the left and right crosstalk cancellation is not limited.

Similarly to the HRTF holding unit 14, the filter forming unit 13 holds the crosstalk cancellation filters C _R (ω) and C _L (ω) corresponding to each sound image localization direction θ _{S in} advance, and the sound image localization direction θ. _The crosstalk cancellation filters C _R (ω) and C _L (ω) to be applied every time _S changes may be switched (selected), or the crosstalk to be applied every time the sound image localization direction θ _S changes. Cancel filters C _R (ω) and C _L (ω) may be calculated.

(A-2) Operation of the First Embodiment Next, the operation of the sound reproduction device 10 of the first embodiment having the above configuration will be described with reference to FIG.

The data input unit 11 converts the input analog signal into a digital signal and outputs it as an input acoustic signal Ci. In this embodiment, the input acoustic signal C _i is supplied to the stereophonic sound processing unit 16 via the filter forming unit 13.

The position information acquisition unit 12 acquires, for example, the latest sound image localization direction, user position information (P _eR , P _eL ), and speaker position information P _SR , P _SL for each predetermined timing and sends them to the filter formation unit 13. Supply.

Each time the information supplied from the position information acquisition unit 12 is updated, the filter forming unit 13 receives the HRTF (H _L (ω), corresponding to the updated information from the HRTF holding unit 14 and the transfer function holding unit 15. H _R (ω)) and transfer functions G _RR , G _RL , G _LR , G _LL are acquired. Then, the filter forming unit 13 uses the acquired HRTF (H _L (ω), H _R (ω)) and the transfer functions G _RR , G _RL , G _LR , G _LL to cancel crosstalk used for transoral processing. The filters C _R (ω) and C _L (ω) are held (held by calculation or selection). Then, the filter forming unit 13 supplies the most recently acquired HRTF (H _L (ω), H _R (ω)) and the crosstalk cancellation filters C _R (ω), C _L (ω) to the stereophonic sound processing unit 16. Supply. At this time, for example, the filter forming unit 13 calculates the parameter α corresponding to the sound image localization direction θ _S (calculated by the above equation (10)), and substitutes α into the above equations (8) and (9). The crosstalk cancellation filters C _R (ω) and C _L (ω) are calculated.

The stereophonic sound processing unit 16 uses the latest supplied HRTF (H _L (ω), H _R (ω)) and the crosstalk cancellation filters C _R (ω), C _L (ω) from the filter forming unit 13. Then, transoral processing is performed on the input acoustic signal C _i . Specifically, first, the three-dimensional sound processing unit 16 generates a binaural sound source B _R for the right ear subjected to binaural processing using the H _{R (ω)} for the right ear to the input audio signal C _i, input A binaural sound source _BL for left ear is generated by binaural processing using H _L (ω) for left ear on the acoustic signal C _i . Then, stereophonic sound processing unit 16 multiplies the crosstalk cancellation filter C _R for the right speaker 18R _(omega) to binaural sound B _R for the right ear to generate a Transaural sound T _R for the right speaker 18R, This is supplied to the output unit 17. Further, the stereophonic sound processing unit 16 multiplies the binaural sound source _BL for the left ear by the crosstalk cancellation filter C _L (ω) for the left speaker 18L to generate a transoral sound source T _L for the left speaker 18L, This is supplied to the output unit 17.

The output unit 17 converts the transoral sound sources T _R and T _L supplied from the stereophonic sound processing unit 16 into analog signals and outputs (supplies) to the speakers 18R and 18L, respectively.

As described above, an audio reproducing device 10, an input audio signal C _i by transaural processing, and outputs a sound signal as Transaural sound source to the user U.

(A-3) Effects of First Embodiment According to the first embodiment, the following effects can be achieved.

In the sound reproducing apparatus 10, when the transaural processing, and applying the cross-talk cancellation filters corresponding to the sound image localization direction theta _S. Thereby, in the sound reproduction apparatus 10, the sweet spot which can obtain a three-dimensional sound effect can be widened. Thereby, in the first embodiment, for example, even if the user U turns his face in the sound source localization direction, it is possible to experience a sound with a sense of reality while maintaining a sense of localization and naturalness.

Further, in the sound reproducing apparatus 10, it is possible to hold the crosstalk cancellation filters for each sound image localization direction theta _S, a configuration for selecting the crosstalk cancellation filter in accordance with the sound image localization direction theta _S. In this case, in the sound reproducing device 10, even if the process of changing the crosstalk cancellation filter is performed for each sound image localization direction θ _S , the increase in the processing amount (the increase in the processing amount compared to the conventional technology) is small. Therefore, it is possible to efficiently perform high-quality stereophonic processing.

  Next, the contents and results of an experiment (hereinafter referred to as “main experiment”) when the sound reproducing apparatus 10 is actually constructed and listened to by the user will be described.

  FIG. 3 is an explanatory diagram showing the environment of this experiment. FIG. 3 shows the positional relationship between the user U and the speaker in this experiment.

  FIG. 3A shows the positional relationship when the user U and each speaker are viewed from the direction. FIG. 3B shows the position of each speaker when viewed from the user U.

  In this experiment, as shown in FIG. 3, a small speaker of 4 cm square was used, and a total of 16 speaker arrays SA were constructed, 8 in a row and 2 in upper and lower rows. In the speaker array SA, the speakers other than the two speakers at the lower center are dummy speakers SD that do not emit sound. As shown in FIG. 3B, in the speaker array SA, the right speaker 18R and the left speaker 18L are the right speaker and the left speaker 18L of the two speakers at the lower center.

As shown in FIG. 3, in the speaker array SA, the distance L1 between the left and right directions of each speaker is 22 cm, and the distance L2 between the top and bottom is 15 cm. Further, the distance L3 from the end in the left-right direction of the speaker array SA is 1.7 m. Furthermore, it is assumed that the height L4 from the ground to the lower speaker of the speaker array SA is 85 cm. Furthermore, as shown in FIG. 3A, the distance L5 from the center of the speaker array to the center position P _U of the head of the user U who is the subject (listener) is 1 m.

In this experiment, the female voice subjected to the stereophonic sound processing was reproduced from the two central speakers 18R and 18L (transoral reproduction in this embodiment). At this time, the direction of the sound source to be localized (sound image localization direction θ _S ) is four types of 60 ° to the right, 60 ° to the left, 90 ° to the right, and 90 ° to the left, and reproduction was also performed in this order. At this time, one sound source is reproduced at a time. In this experiment, the sound is reproduced while switching the direction of the sound source to be localized (sound image localization direction θ _S ) as described above, and the user U who is the subject selects a speaker that feels sounding, etc. A questionnaire was conducted. In this experiment, the test subject was informed that sound was sounding from the central two speakers in advance, but the speaker at the top and bottom was sounding down and listened.

  And in this experiment, after the reproduction | regeneration of the above-mentioned 4 types of audio | voice was complete | finished, the subjective evaluation questionnaire was implemented by the average opinion score (Mean Opinion Score MOS). In this experiment, the items in the questionnaire were localization (whether the position of the sound image was stable without fluctuation), sound quality (whether you felt distortion or abnormal noise), naturalness (actually sounded from the speaker) Whether or not you feel it).

FIG. 4 is a graph showing the results of counting the direction (position) of the sound source felt by the subject. 4 (a) to 4 (d) show the total results when the direction of the sound source to be localized (sound image localization direction θ _S ) is 60 ° left, 60 ° right, 90 ° left, and 90 ° right. It is the shown graph. In each graph shown in FIG. 4, the vertical axis (height of each bar graph) is the number of people, and the horizontal axis is the position of the speaker. In the horizontal axis of each graph shown in FIG. 4, 1 indicates the leftmost speaker, and 8 indicates the rightmost speaker. In each graph shown in FIG. 4, the bar graph in the front row corresponds to the upper speaker of the speaker array SA, and the rear row corresponds to the lower speaker in the speaker array SA. The speakers 18R and 18L that are actually sounding correspond to the lower and fourth bar graphs.

  As shown in FIG. 4, in this experiment, when the sound source position is 60 ° to the right, there are many subjects who have selected one speaker inside the right end, but at the left 60 °, the left end is almost selected. . Furthermore, as shown in FIG. 4, in this experiment, when the sound source position was 90 °, all left and right speakers were selected.

  Next, the result of the subjective evaluation in this experiment is shown in FIG. In this experiment, the MOS values were 3.83 for localization, 3.67 for sound quality, and 4.33 for naturalness, all exceeding 3. In particular, in this experiment, the naturalness exceeds 4, and it can be seen that the subject felt as if the sound was actually sounding from the dummy speaker. Further, in this experiment, the sense of localization is a high value close to 4. As described above, the result of this experiment showed that the sound reproducing apparatus 10 according to this embodiment can obtain a stereophonic effect that is stable in localization and can be heard naturally.

(B) Second Embodiment Hereinafter, a second embodiment of the sound reproducing device and the program according to the present invention will be described in detail with reference to the drawings.

(B-1) Configuration of Second Embodiment FIG. 6 is a block diagram showing an overall configuration of a sound reproducing device 10A of the second embodiment. In FIG. 6, the same or corresponding parts as those in FIG.

  Hereinafter, differences of the third embodiment from the first embodiment will be described.

  The sound reproducing device 10A of the second embodiment is different from the first embodiment in that a filter holding unit 19 is added and the filter forming unit 13 is replaced with a filter forming unit 13A.

In the first embodiment, the example in which the filter forming unit 13 performs the process of forming the crosstalk cancellation filter every time the sound image localization direction θ _S is acquired has been described. On the other hand, the filter forming unit 13A of the second embodiment forms a crosstalk cancellation filter of each sound image localization direction θ _S and holds it in the filter holding unit 19. Thereafter, the filter forming unit 13 selects a crosstalk cancellation filter corresponding to the input sound image localization direction θ _S and supplies it to the stereophonic sound processing unit 16.

Further, when position information (user position information and speaker position information) is supplied from the position information acquisition unit 12, the filter forming unit 13A holds the crosstalk cancellation filter corresponding to the position information in the filter holding unit 19. Check if it exists. When the crosstalk cancellation filter corresponding to the position information supplied from the position information acquisition unit 12 is not held in the filter holding unit 19, the filter forming unit 13 </ b> A A sound image localization direction θ _S (crosstalk cancellation filter) is formed, and the filter holding unit 19 holds it. That is, in the second embodiment, the filter holding unit 19 functions as a cache for the crosstalk cancellation filter.

(B-2) Effects of Second Embodiment According to the second embodiment, the following effects can be obtained in addition to the effects of the first embodiment.

In the sound reproducing device 10A of the second embodiment, a crosstalk cancellation filter for each sound image localization direction θ _S is formed and held (cached) in the filter holding unit 19, so that the processing amount ( The amount of processing that forms the crosstalk cancellation filter) can be reduced.

(C) Third Embodiment Hereinafter, a third embodiment of the sound reproducing device and the program according to the present invention will be described in detail with reference to the drawings.

(C-1) Configuration and Operation of the Third Embodiment FIG. 7 is a block diagram showing the overall configuration of the sound reproducing device 10B of the third embodiment. In FIG. 7, the same or corresponding parts as those in FIG.

  Hereinafter, differences of the third embodiment from the first embodiment will be described.

  The sound reproducing device 10B according to the third embodiment is different from the first embodiment in that the transfer function holding unit 15 is deleted and the filter forming unit 13 is replaced with a filter forming unit 13B. Further, the sound reproducing device 10B of the third embodiment is different from the first embodiment in that an angle calculation unit 20 is added.

The filter forming unit 13 according to the first embodiment acquires the HRTF and the transfer functions G _RR , G _RL , G _LR , and G _LL to form a crosstalk cancellation filter. Therefore, the filter forming unit 13 of the first embodiment cannot form a crosstalk cancellation filter in a space where the transfer function is not measured. Therefore, the filter forming unit 13B of the third embodiment uses HRTF (HRTF held by the HRTF holding unit 14) instead of the transfer function when forming the crosstalk cancellation filter.

  When the user position information and the speaker position information are supplied from the position information acquisition unit 12, the angle calculation unit 20 receives each speaker 18R from the user U (each ear of the user U) based on the user position information and the speaker position information. The direction (angle) to 18L is calculated and supplied to the filter forming unit 13B.

  FIG. 8 is an explanatory diagram showing an example of each direction (angle) acquired by the angle calculation unit 20.

Specifically, the angle calculation unit 20 is based on the user position information and the speaker position information, and θ _RR indicating the direction (angle) from the right ear position P _eR of the user U to the position P _SR of the right speaker 18R. , Θ _RL indicating the direction (angle) from the position P _eR of the right ear of the user U to the position P _SL of the left speaker 18L, and the direction from the position P _eL of the left ear of the user U to the position P _{SR of the} right speaker 18R Θ _LR indicating (angle) and θ _LL indicating the direction (angle) from the position P _eL of the left ear of the user U to the position P _SL of the left speaker 18L are calculated.

As shown in FIG. 8, θ _RR and θ _RL are directions (angles) starting from the position _PeR of the right ear of the user U. θ _RR and θ _RL indicate directions in which the speakers 18R and 18L exist, assuming that the direction in which the user U is facing from the position P _eR of the right ear of the user U is 0 °.

Further, as shown in FIG. 8, θ _LR and θ _LL are directions (angles) starting from the position P _eL of the right ear of the user U. θ _LR and θ _LL indicate directions in which the speakers 18R and 18L exist, assuming that the direction in which the user U is facing from the right ear position _PeR of the user U is 0 °.

The filter forming unit 13B then obtains the directions θ _RR , θ _RL , θ _LR , and θ _LL indicating the directions (directions from the user's U ears to the speakers 18R and 18L) acquired from the angle calculation unit 20. HRTF is acquired (HRTF holding unit 14). Hereinafter, HRTFs corresponding to θ _RR , θ _RL , θ _LR , and θ _LL are represented as H _RR (ω), H _RL (ω), H _LR (ω), and H _LL (ω), respectively.

Then, the filter forming unit 13B uses H _RR (ω), H _RL (ω), H _LR (ω), and H _LL (ω) as transfer functions G _RR , G _RL , G _LR , and G _LL , respectively. Generate a crosstalk cancellation filter.

(C-2) Effects of Third Embodiment According to the third embodiment, the following effects can be achieved in addition to the effects of the first embodiment.

  In the sound reproducing device 10B of the third embodiment, since the HRTF is used as a transfer function, a crosstalk cancellation filter is formed with a certain degree of accuracy even in a space that does not hold the corresponding transfer function data, Acoustic processing (trans-oral processing) can be performed.

(D) Other Embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments as exemplified below.

  (D-1) In each of the above embodiments, the sound reproducing device of the present invention has been described as a configuration that performs real-time update of three parameters of the sound image localization direction, the user position information, and the speaker position information as the position information. A parameter that does not need to be updated in real time may be configured to keep a preset value without updating. For example, in the sound reproducing apparatus, the user position information and the speaker position information may be held as fixed values and acquired as parameters that vary only in the sound image localization direction.

In addition, the sound reproduction device of the present invention may be configured to directly hold the transfer functions G _RR , G _RL , G _LR , G _LL without holding the user position information and the speaker position information, for example.

  DESCRIPTION OF SYMBOLS 10 ... Sound reproduction apparatus, 11 ... Data input part, 12 ... Position information acquisition part, 13 ... Filter formation part, 14 ... HRTF holding part, 15 ... Transfer function holding part, 16 ... Stereophonic sound processing part, 17 ... Output part, 18R ... right speaker, 18L ... left speaker.

Claims (7)

In an audio reproduction device that generates stereophonic signals to be supplied to each of a plurality of speakers by performing stereophonic processing on an input acoustic signal,
A head-related transfer function holding unit that holds a head-related transfer function corresponding to the direction of each sound source;
An information acquisition unit that acquires information on a sound image localization direction that localizes at least a sound source;
A first stereophonic signal generation unit that generates a first stereoacoustic signal in which a sound source is localized in the sound image localization direction using the head-related transfer function held by the head-related transfer function holding unit;
For each of the speakers, a crosstalk component is removed from the first stereophonic sound signal and a crosstalk cancellation filter is retained, and a parameter corresponding to the sound image localization direction acquired by the information acquisition unit is used. A crosstalk cancellation filter holding unit for holding a crosstalk cancellation filter based on ,
For each of the speakers, a second stereophonic signal is generated by removing a crosstalk component from the first stereophonic signal using a crosstalk cancellation filter held by the crosstalk cancellation filter holding unit. And a stereophonic sound signal generator.   The crosstalk cancellation filter holding unit adjusts the balance of the amount of crosstalk components removed by the crosstalk cancellation filter corresponding to each speaker according to the sound image localization direction acquired by the information acquisition unit. The stereophonic sound reproducing device according to claim 1.   The crosstalk cancellation filter holding unit balances the amount of cancellation of crosstalk components by the crosstalk cancellation filter corresponding to each speaker, using a parameter that varies according to the sound image localization direction acquired by the information acquisition unit. The three-dimensional sound reproducing apparatus according to claim 2, wherein adjustment is performed.   The crosstalk cancellation filter holding unit holds a crosstalk cancellation filter for each sound image localization direction in advance, selects a crosstalk cancellation filter corresponding to the sound image localization direction acquired by the information acquisition unit, and selects the second stereoscopic The stereophonic sound reproducing device according to any one of claims 1 to 3, wherein the sound signal generating unit is supplied to the sound signal generating unit. The information acquisition unit further acquires the position information of the listener and the position information of each speaker,
A transfer function holding unit for acquiring a transfer function between each speaker and the listener according to the position information of the listener and the position information of each speaker;
The stereophonic sound reproducing device according to claim 1, wherein the crosstalk cancellation filter holding unit generates a crosstalk cancellation filter using the transfer function acquired by the transfer function holding unit. .   The transfer function holding unit recognizes the direction from the listener to each speaker based on the listener's position information and the position information of each speaker, and from the listener to each speaker. The stereophonic sound reproducing device according to claim 5, wherein a head-related transfer function corresponding to the direction of the sound is acquired as a transfer function between each of the speakers and the listener. A computer mounted on a sound reproduction device that generates a three-dimensional sound signal that is supplied to each of a plurality of speakers by performing a three-dimensional sound process on the input sound signal,
A head-related transfer function holding unit that holds a head-related transfer function corresponding to the direction of each sound source;
An information acquisition unit that acquires information on a sound image localization direction that localizes at least a sound source;
A first stereophonic signal generation unit that generates a first stereoacoustic signal in which a sound source is localized in the sound image localization direction using the head-related transfer function held by the head-related transfer function holding unit;
For each of the speakers, a crosstalk component is removed from the first stereophonic sound signal and a crosstalk cancellation filter is retained, and a parameter corresponding to the sound image localization direction acquired by the information acquisition unit is used. A crosstalk cancellation filter holding unit for holding a crosstalk cancellation filter based on ,
For each of the speakers, a second stereophonic signal is generated by removing a crosstalk component from the first stereophonic signal using a crosstalk cancellation filter held by the crosstalk cancellation filter holding unit. A stereophonic sound reproduction program that functions as a stereophonic sound signal generation unit.

Images