JPH08107600A - Sound image localization device - Google Patents

Abstract

PURPOSE: To stably localize a sound image at a designated position in an absolute coordinate system at all times even if the head of a listener is rotated. CONSTITUTION: This device is equipped with an angle of rotation sensor 5 which detects the rotation of the head of the listener 2, and the position information r, θ and ϕ of a virtual sound source are corrected by the output α, β and γof the angle of rotation sensor 5. A virtual sound source position controller 12 performs various kinds of processing such as delay control, filtering, amplitude control, etc., on an audio input signal SI based on corrected position information r', θ' and ϕ', and generates audio output signals SOR, SOL so as to localize the virtual sound source at the designated position in the absolute coordinate system at all times irrespective of the position of the head of the listener.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound image localization apparatus suitable for a three-dimensional virtual reality system or the like.

[0002]

2. Description of the Related Art Heretofore, a sound image localization apparatus for locating a virtual sound source at an arbitrary position in a sound field space to generate a three-dimensional sound field has been known in sound reproduction systems, electronic musical instruments, games and the like. Further, even in the three-dimensional virtual reality system, this kind of sound image localization device is used as a means for improving the realism in the virtual experience. In this case, the sound image localization device generates a plurality of channels of signals having a time difference, an amplitude difference, and a frequency characteristic difference from a monaural sound source based on a binaural method so as to give a sense of direction, a sense of distance, and a sense of distance. A three-dimensional sound field is generated, and an acoustic signal is generated as if a sound is emitted from each part in the three-dimensional virtual space.

[0003]

By the way, in this type of sound image localization apparatus, various control parameters necessary for delay control, filtering, amplitude control, etc. are provided on the assumption that the listener's head is always facing the front. The audio output signal of each channel is calculated and controlled based on these control parameters. Therefore, there is a problem that the sound image is not properly localized at the designated virtual sound source position when the listener's head is displaced from the front. For example, in a sound image localization device that uses headphones, the virtual sound source position is localized at a position different from the desired position to be localized according to the change (rotation) of the position of the listener's head. In other words, for example, if you want to localize the sound behind the listener, there is no problem if the normal listener's head is facing the front, but if the listener's head has rotated 90 ° to the right, Of course, since the headphones are also rotated by 90 °, the listener feels that the sound is localized on the left side of the listener (localized behind the listener's head). In addition, a sound image localization device that uses a fixedly installed speaker system does not cause problems like headphones, but the transfer characteristics from the left and right speakers to both ears of the listener change due to changes in the position of the listener's head. Therefore, it is particularly difficult to process the crosstalk component, and a sound that is not preferable for hearing may be generated. Such a change hinders stable localization of the virtual sound source. Particularly in a high-level virtual reality system in which a three-dimensional image is panned according to the movement of the viewer's head or line of sight, such movement of the virtual sound source position hinders matching between the image and the virtual sound source position, and Cause a drop in.

Although a two-dimensional sound image localization is an example, a method for solving the above problem has been proposed (Japanese Patent Laid-Open No. 4-17500). In this example, the sound image localization control is performed based on the left-right rotation of the entire listener, that is, the left-right rotation of the chair. However, even in this method, the movement of the listener's head is not taken into consideration, and the problem similar to that of the above-described conventional example occurs in the end. The present invention has been made in view of such a problem, and a sound image that can always stably position a sound image at a designated position in an absolute coordinate system even if the position (rotation) of the listener's head changes. It is intended to provide a localization device.

[0005]

A sound image localization apparatus according to the present invention calculates a control parameter based on position information indicating a prespecified virtual sound source position in a sound field space, and outputs an audio input signal supplied from the sound source. A virtual sound source position control means for generating a plurality of channels of audio output signals for locating a sound image at the virtual sound source position by performing delay control, filtering and / or amplitude control on the basis of the calculated control parameter. In a sound image localization device including a plurality of electroacoustic conversion means for respectively electroacoustic converting audio output signals of a plurality of channels output from the virtual sound source position control means, a rotation angle detection means for detecting a rotation angle of a listener's head. The virtual sound source position control means is configured to rotate the head of the listener detected by the rotation angle detection means. Modifying the control parameter based on the angle to control the audio output signal so that the virtual sound source is always localized at the specified position in the absolute coordinate system regardless of the position of the listener's head. Characterize.

When the electro-acoustic converting means is headphones attached to both ears of the listener, the virtual sound source position control means determines the rotation angle of the listener's head detected by the rotation angle detecting means. Based on the virtual sound source position of the virtual sound source position of the position information of the virtual sound source position in the coordinate system after the coordinate conversion, the position on the neck of the listener vertically lowered from the center position of both ears of the listener It is desirable to calculate the control parameter as the position information.

Further, when the electroacoustic conversion means is a speaker fixedly installed in the sound field space, the virtual sound source position control means rotates the head of the listener detected by the rotation angle detection means. Based on the angle, coordinate conversion is performed centering on the position on the neck of the listener vertically lowered from the center position of both ears of the listener, and the virtual sound source in which the virtual sound source position in the coordinate system after conversion is corrected It is preferable that the control parameter is adjusted based on the change in the position of the speaker with respect to the binaural position of the listener.

[0008]

According to the present invention, the control parameter for sound image localization is modified based on the rotation angle of the listener's head detected by the rotation angle detecting means, and the audio input signal is modified using this modified control parameter. Since the delay control, the filtering, the amplitude control, etc. are performed on the audio output signal, the audio output signal can be controlled so that the virtual sound source is always localized at the designated position in the absolute coordinate system.

Since the listener's head normally rotates in each direction (x-axis, y-axis, z-axis) about the center position of the neck, when correcting the position information indicating the virtual sound source position,
By setting the center position of the coordinate conversion to the position above the listener's neck, the virtual sound source position can be corrected accurately.

When the electroacoustic converting means is headphones, the headphones always maintain a fixed position with respect to the listener's head, and therefore the virtual sound source position is always moved in the opposite direction to the head moving direction by the same amount. With such control, the virtual sound source position in the absolute coordinate system is fixed at the set position. Therefore, if the control parameter is calculated by using the position information of the virtual sound source position in the coordinate system after the coordinate conversion based on the rotation of the head as the corrected position information, the sound image is always localized at a fixed position regardless of the rotation of the listener's head. Will be done.

Further, when the electroacoustic conversion means is a speaker fixedly installed in the sound field space, it is not sufficient to set the virtual sound source position in the coordinate system after coordinate conversion to the corrected virtual sound source position. Absent. This is because the position of the speaker changes with respect to the positions of both ears of the listener. This change
It affects control parameters and the like that give the transfer function of the crosstalk component from each speaker. In one aspect of the present invention, the virtual sound source can be localized at a fixed position by modifying these control parameters based on the change in the position of the speaker with respect to the binaural position of the listener.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration of a three-dimensional virtual reality system to which a sound image localization device according to an embodiment of the present invention is applied. The headgear 1 is mounted on the head of the listener 2, and the speakers 3 and 4 are embedded at positions corresponding to both ears of the listener 2. A rotation angle sensor 5 that detects a rotation angle of the headgear 1 in a three-dimensional direction from a reference position is attached to a part of the headgear 1. The speakers 3 and 4 and the rotation angle sensor 5 are connected to the controller 6 via a signal line 7. The controller 6 is provided with a display device 8 so that the listener 2 can watch a three-dimensional graphics screen displayed on the display device 8 and operate a joystick or the like (not shown) to perform a game or a simulation. There is. And the display device 8
The controller 6 executes the sound image localization control so that the positions corresponding to the respective parts of the screen displayed in FIG.

FIG. 2 is a block diagram showing a more detailed structure of the system. An audio source 11 and a virtual sound source position control device 12 are provided inside the controller 6. The audio source 11 is a sound source of a computer, a disc reproducing device, a video reproducing device, or the like, and in this embodiment, a monaural audio signal SI.
And position information r, θ, φ in the three-dimensional absolute coordinate system indicating the virtual sound source position. Virtual sound source position control device 1
2 is an audio input signal SI and position information r, θ,
φ and the angle information α, β, γ from the rotation angle sensor 5 are input, and the audio input signal SI is subjected to various processes such as delay control, filtering and amplitude control based on the position information and the angle information, and the two channels are processed. Audio output signal S
Generates OR and SOL. The two-channel audio output signals are supplied to the speakers 3 and 4 to supply acoustic waves to both ears of the listener 2.

FIG. 3 is a block diagram showing a concrete example of the configuration of the virtual sound source position control device 12. The monaural audio input signal SI is supplied to the notch filter 22 via the amplifier 21. The notch filter 22 attenuates a specific frequency component of the audio signal SI on the basis of human auditory characteristics to give a vertical feeling to the input signal SI.
The output of the notch filter 22 is subjected to delay processing by a delay circuit 23 that gives a difference in sound propagation time from the virtual sound source position to both ears, and is converted into a two-channel signal having a time difference. These signals are respectively FIR (finite impulse response)
It is supplied to the filters 24 and 25. FIR filter 2
Numerals 4 and 25 measure the impulse response (head related transfer function) when the sound image is localized in the front, rear, left and right directions of the listener 2 in advance by, for example, a dummy head, and store the result as an FIR coefficient. The signal for each direction is generated from the input signal. FIR filter 2
The outputs of 4 and 25 are amplitude-controlled by the amplifiers 26 and 27 so that the signals of the respective directional components are controlled so that the sound comes from the designated virtual sound source position. The outputs of the amplifiers 26 and 27 are added by adders 28 and 29, respectively, and then the left and right amplitude balances based on the directions of the virtual sound sources are adjusted by the amplifiers 30 and 31, and the distances to the virtual sound sources are adjusted by the amplifiers 32 and 33. Based on the amplitude control.
The signal of each channel is output as audio output signals SOR and SOL through these processes.

On the other hand, the position information r, θ, φ given from the audio source 11 is corrected by the position information correction section 35 by the sensor outputs α, β, γ. The parameter calculation unit 36 uses the corrected position information α ′, β ′,
Based on γ ′, control parameters Nt, T, VRx, V
Lx, VR, VL, Vr are generated and supplied to each part.

Next, the operating principle of this system will be described. FIG. 4 shows position information r, θ, which indicates the virtual sound source position.
It is a figure for demonstrating (phi). Now, in the figure (a)
As shown in, the middle point between the ears of the listener 2 is the center point P0 of the three-dimensional coordinates, and the head of the listener 2 is in the reference direction (front direction).
The right direction, the front side, and the upper side of the listener 2 in the case of being directed to are set as the X axis, Y axis, and Z axis of the absolute coordinate system, respectively. The position information of the virtual sound source Ps is, as shown in FIG. 7B, the distance r from the center point P0 to the virtual sound source Ps and the horizontal angle of the virtual sound source Ps as viewed from the front of the listener 2 (Y-axis direction). (Azimuth) θ and the angle (elevation) φ in the vertical direction when viewed from the angle θ with respect to the front surface of the listener 2.
Given by and.

On the other hand, as shown in FIG. 5, the rotation angle associated with the movement of the head of the listener 2 is the rotation angle α about the Z axis and the rotation angle α.
Rotation angle β around the axis and rotation angle γ around the Y axis
Can be represented by three types. These rotation angles α, β,
γ is detected by the rotation angle sensor 5. The rotation angle sensor 5 includes a gyro sensor, a magnet type sensor, and a right angle 3
A known direction detector such as an electromagnetic wave reception type sensor using a direction receiving unit can be applied, but if a gyro sensor is taken as an example, then a device as shown in FIG. 6 can be used, for example. That is, the flywheel 42 is rotated by the motor 41 so that the inertia of the flywheel 42 stabilizes the shaft 43 in a certain direction, and the arm 44 supporting the shaft 43 and the left and right directions HR and HL of the headgear 1 rotate about the shaft. The encoder 46 detects a rotation angle α between the arm 45 and the arm 45 which is supported. The rotation angle β of the headgear 1 about the left-right direction HR and HL is detected by the encoder 47 that is interlocked with the rotation axis of the arm 45. Further, the arm 4 rotatably supported about the front-rear direction HF, HB of the headgear 1 as an axis.
8 is provided with a weight 49 slidably, and an encoder 50 that interlocks with the rotation axis of the arm 48 causes front and rear directions HF and HB generated by the movement of the weight 49 in the gravity direction.
The rotation angle γ about the axis is detected.

In the case of the headphone type sound image localization apparatus as in this embodiment, with respect to the rotational movement of the listener's head, if the virtual sound source position is rotationally moved in the opposite direction,
The position of the virtual sound source Ps in the absolute coordinate system can be kept constant. This means that the position information r, θ, φ of the virtual sound source Ps given in the absolute coordinate system X, Y, Z is converted in the transformed coordinate system X ′, Y ′, Z ′ with the listener's head as a reference. This is equivalent to converting the information into r ', θ', φ'and using this as given position information.

Now, the coordinate value X of each axis of the position Ps of the virtual sound source
s, Ys, and Zs can be expressed as in Equation 1 by using the given positional information r, θ, and φ.

[0020]

## EQU1 ## Xs = r sinθ cosφ Ys = r cosθ cosφ Zs = r sinφ

On the other hand, as shown in FIG. 7, since the movement of the head of the listener 2 is centered on the neck, the center of coordinate rotation is the coordinate origin P0 described above and the distance m along the neck of the listener 2. Set it to the position where it was moved downward. m is set to about 15 cm as an average value. Transformation coordinate system X ',
Coordinate value X of each axis of virtual sound source Ps 'in Y'and Z'
s', Ys', and Zs' can be expressed by the following equation 2.

[0022]

[Equation 2]

Here, M is a transformation matrix, and transformation matrices M (α), M based on the rotation angles α, β, γ in the respective axial directions.
It is defined by Equation (3) by (β) and M (γ).

[0024]

## EQU3 ## M = M (α) M (β) M (γ)

The coordinate value X of each axis of the obtained virtual sound source Ps'
From s', Ys', Zs', conversion position information r ', θ',
φ ′ can be obtained by the equation 4.

[0026]

Equation 4] ^{r '= √ (Xs' 2} + Ys' 2 + Zs' 2) θ' = tan -1 (Xs' / Ys') φ '= tan -1 [Zs' / √ (Xs' 2 + Ys' 2 )]

The position information correction unit 35 shown in FIG. 3 executes the above-mentioned processing, and the position information r, θ, φ and sensor outputs α, β,
From γ by coordinate conversion processing, conversion position information r ′,
Generates and outputs θ'and φ '. FIG. 8 shows the position information correction unit 3.
6 is a flowchart showing the contents of coordinate conversion processing in No. 5. When the position information r, θ, φ is given to the position information correction unit 35, first, the coordinate value Xs, Ys of each axis in the absolute coordinate system is given.
, Zs are obtained (S1). However, for the Zs axis,
It is shifted (+ m) in advance to the movement coordinate system of the head. Next, the outputs α, β, γ of the rotation angle sensor 5 are sampled (S
2) The coordinate conversion process is performed by the conversion matrix M (S
3), shift the Zs' axis to match the original coordinates (-
m) (S4). Then, the converted coordinate Xs obtained
From ′, Ys ′, Zs ′, conversion position information r ′, θ ′,
φ'is calculated (S5) and output (S6). The conversion position information can be obtained in real time by executing the sampling of the sensor output and the conversion process at regular intervals.

These converted position information r ', θ', φ '
Is supplied to the parameter calculation unit 36, and the following control parameters are calculated. First, the attenuation frequency Nt of the notch filter 22 is obtained. As shown in FIG. 9, it is known that in the human ear, the dead band frequency shifts to a high frequency region as the angle (elevation) φ of the sound source in the height direction increases, that is, as the sound source position is located higher. ing. Therefore, the parameter calculator 36 determines the attenuation frequency Nt by the elevation φ ′ and controls the notch filter 22.

The propagation time difference T between the left and right channels in the delay circuit 23 is obtained from the difference in the distance from the virtual sound source Ps' to each ear. That is, as shown in FIG. 10, the coordinate center point P
If the distance from 0'to both ears of the listener 2 is h, the distance DR from the position Ps '(Xs', Ys ', Zs') of the virtual sound source to the right ear (h, 0, 0) and the left ear. (-H, 0,
The distance DL to 0) can be expressed as in Equation 5.

[0030]

DR = √ (r ′ ² −2r′h sin θ ′ sin φ ′ + h ² ) DL = √ (r ′ ² + 2r′h sin θ ′ sin φ ′ + h ² )

Therefore, if the speed of sound is Vs,

[0032]

[Equation 6] T = (DR-DL) / Vs

Is the propagation time difference when the acoustic signal arrives from the virtual sound source Ps' to both ears.

Parameters VRx and VLx for controlling the amplitude of each direction component output of the FIR filters 26 and 27 [however,
R is for the right ear, L is for the left ear, x is F (front), R (right), B
(Rear) and L (left). ] Indicates the left and right ears and the sound source Ps ′
And azimuth θR, θL and elevation φ. The azimuths θR and θL of each ear are obtained as follows.

[0035]

## EQU7 ## θR = tan ^-1 [(r'sinφ'sinθ'-h) /
r'sin φ'cos θ '] θL = tan ^-1 [(r'sin φ'sin θ' + h) / r'sin φ'c
osθ ']

FIG. 11 is a diagram showing the relationship between the azimuth angles θR and θL of each ear and the control parameters VRx and VLx. In FIG. 11A, elevation φ ′ is 0 ° (horizontal), and FIG. Indicates the values when the elevation φ ′ is ± 90 ° (vertical). At this time, although each gain is set to 1/4 in the figure, it is not necessarily limited to this. The parameter calculation unit 36 determines the control parameters VRx and VLx based on these relationships and controls the amplifiers 26 and 27.

Further, the control parameters VR and VL of the amplifiers 30 and 31 for controlling the amplitudes of the left and right channels are determined by the difference between the above-mentioned virtual sound source Ps' and both ears. Further, the control parameter Vr of the amplifiers 32 and 33 for controlling the overall amplitude of both channels is the virtual sound source P.
It is determined by the distance r'from s'.

The above is a sound image localization apparatus using headphones, but FIG. 12 shows the configuration of a system in which the present invention is applied to a sound image localization apparatus using a fixedly arranged speaker. Also in this embodiment, since the rotation angle sensor 5 is installed on the head of the listener 2, some kind of head gear is required. A point to be considered in the case of the fixed type speaker is that the position of the virtual sound source with respect to the listener 2 changes due to the rotation of the head of the listener 2, and the relative positions of the speakers 3 and 4 with respect to the listener 2 also change. . Therefore, the converted position information r ′, θ ′, φ ′
Needs to be re-corrected to the vicinity of the position of the first position information r, θ, φ again based on the change in the relative positions of the speakers 3 and 4. Further, as a simpler method, the control parameters may be obtained by directly using the initial position information r, θ, φ. However, the problem of crosstalk peculiar to fixed speakers requires some measures.

That is, as shown in FIG. 12, in the case of a sound image localization apparatus using a fixed type speaker, the sound of the right speaker 3 is supplied to the left ear, and the sound of the left speaker 4 is supplied to the right ear. Occurs. Therefore, as shown in FIG. 13, in the virtual sound source position control device 12, the crosstalk canceller 61 includes, for example, the amplifiers 30, 31 and the amplifiers 32,
It is arranged between 33 and.

Crosstalk can be represented by, for example, the model shown in FIG. Now, let us define the path through which the acoustic wave propagates from the right (left) speaker to the right (left) ear as the main path, and the path through which the acoustic wave propagates from the right (left) speaker to the left (right) ear as the crosstalk path. , D is the time difference between the time when the acoustic wave propagates in the main path and the time when the acoustic wave propagates in the crosstalk path, and k is the time difference when propagating in the crosstalk path with respect to the attenuation amount when the acoustic wave propagates in the main path. It is the ratio of the amount of attenuation.

The details of the crosstalk canceller 61 based on this model are shown in FIG. Here, the propagation time difference d and the attenuation ratio k are calculated assuming that the listener 2 faces the front of the speakers 3 and 4, but when the head of the listener 2 rotates, the values of d and k are also calculated. It changes symmetrically. Therefore, the parameter calculation unit 62 calculates the propagation time differences dr, dl and the attenuation ratios kr, kl from the sensor outputs α, β, γ. The propagation time differences dr and dl are
Perform the above-mentioned coordinate conversion process with the position of as the sound source position,
The distance from both ears to each speaker is calculated using the coordinate system of the head of the listener 2, and the distance is divided by the speed of sound to obtain the difference in propagation time. Also, the attenuation ratios kr and kl are calculated in advance by the listener 2
The ratio k of attenuation from several points around the head of the listener is calculated in advance, and according to the direction of the speaker in the coordinate system of the head of the listener 2,
It suffices to obtain these values by appropriately performing interpolation processing. The propagation time differences dr, dl, kr, and kl thus obtained are control parameters that determine the operation of the crosstalk canceller 61.

[0042]

As described above, according to the present invention,
Based on the rotation angle of the listener's head detected by the rotation angle detecting means, a control parameter for sound image localization is modified, and the modified control parameter is used to delay control, filter and amplitude control the audio input signal. Therefore, the audio output signal can be controlled so that the virtual sound source is always localized at the designated position in the absolute coordinate system.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a system to which a headphone type sound image localization device of the present invention is applied.

FIG. 2 is a more detailed block diagram of the system.

FIG. 3 is a block diagram of a virtual sound source position control device in the system.

FIG. 4 is a diagram for explaining position information.

FIG. 5 is a diagram for explaining a rotation direction of a listener's head.

FIG. 6 is a perspective view showing an example of a rotation angle sensor.

FIG. 7 is a diagram showing a movement of a listener's head and a transformation coordinate system.

FIG. 8 is a flowchart showing a process of a position information correction unit.

FIG. 9 is a diagram showing characteristics of a notch filter.

FIG. 10 is a diagram for explaining a procedure for obtaining a time difference in the delay circuit.

FIG. 11 is a diagram for explaining amplitude control of the FIR filter output.

FIG. 12 is a diagram showing a configuration of a system to which a fixed speaker type sound image localization device of the present invention is applied.

FIG. 13 is a block diagram of a virtual sound source position control device in the same system.

FIG. 14 is a block diagram showing a crosstalk model in the same system.

FIG. 15 is a block diagram showing details of a crosstalk canceller in the system.

[Explanation of symbols]

1 ... Headgear, 2 ... Listener, 3, 4 ... Speaker, 5 ...
Rotation angle sensor, 6 ... Controller, 7 ... Signal line, 8 ... Display device, 11 ... Audio source, 12 ... Virtual sound source position control device 21, 26, 27, 30-33 ... Amplifier, 22 ... Notch filter, 23 ... Delay circuit, 24,
25 ... FIR filter, 28, 29 ... Adder, 35 ... Position information correction unit, 36, 62 ... Parameter calculation unit, 61 ...
Crosstalk canceller.

Claims (3)

[Claims] 1. A control parameter is calculated based on position information indicating a prespecified virtual sound source position in a sound field space, and is delayed based on the calculated control parameter with respect to an audio input signal supplied from a sound source. Virtual sound source position control means for generating audio output signals of a plurality of channels for localizing a sound image at the virtual sound source position by performing control, filtering, and / or amplitude control, and a plurality of channels output from the virtual sound source position control means. A sound image localization apparatus including a plurality of electroacoustic conversion means for converting respective audio output signals into electroacoustic signals, comprising a rotation angle detection means for detecting a rotation angle of a listener's head, wherein the virtual sound source position control means is adapted to Correcting the control parameter based on the rotation angle of the listener's head detected by the angle detection means, A sound image localization apparatus that controls the audio output signal so that the virtual sound source is always localized at a specified position in an absolute coordinate system regardless of the position of the listener's head. 2. The electroacoustic conversion means is headphones attached to both ears of the listener, and the virtual sound source position control means is based on the rotation angle of the listener's head detected by the rotation angle detection means. By performing coordinate conversion centered on the position on the neck of the listener vertically lowered from the center position of both ears of the listener, the position information of the virtual sound source position in the coordinate system after conversion is the position of the virtual sound source position. The sound image localization apparatus according to claim 1, wherein the control parameter is calculated as information. 3. The electroacoustic conversion means is a speaker fixedly installed in the sound field space, and the virtual sound source position control means is a rotation angle of a listener's head detected by the rotation angle detection means. Based on the above, coordinate conversion is performed centering on the position on the neck of the listener vertically lowered from the center position of both ears of the listener, and the virtual sound source position in which the virtual sound source position in the coordinate system after conversion is corrected. The sound image localization apparatus according to claim 1, wherein the control parameter is corrected based on a change in the position of the speaker with respect to the binaural positions of the listener.