JP2006222801A - Moving sound image presenting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving sound image presenting device which obtains rendering effects including a Doppler effect when being given only position information changing momentarily and presents a sound image of high reality/localization. <P>SOLUTION: A head related transfer function is separated into an initial delay and a main response, and the Doppler effect is added so that the delay is processed together with the Doppler effect and spectrum components are rendered later in the main response part, whereby a rendering method is closer to physical phenomena (easy for an application creator to use) and an effect of high reality can be obtained. Head related transfer functions matching individuals are selected and used to obtain an affect of presenting a moving sound image of high reality/localization to many and unspecified persons. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention calculates a sound image whose position changes every moment in a three-dimensional space from parameters (sound source / listener position information, monaural sound source) and head-related impulse response (HRIR: HRIR). The present invention relates to a configuration of a moving sound image presentation device that performs processing and presents a moving sound image virtually to a listener.

  The sound emitted in the three-dimensional space finally reaches both ears through various physical phenomena such as reflection and diffraction. In order for the audience to perceive such a phenomenon in a pseudo manner, it is necessary to produce (render) the original sound.

  Possible types of rendering include sound source location information, early reflections, rear reverberation, diffracted sounds, and Doppler effects, but location information and Doppler effects are particularly important, especially when the sound changes its position dynamically. is there.

  The head related transfer function synthesis method is mainly used to render the position of the sound source. Such a technique is disclosed in Patent Document 1, for example. In Patent Document 1, there is a method in which a sound image is localized at an arbitrary position by using a head acoustic transfer function observed in advance at a plurality of positions and adding distance attenuation to a digital filter approximating the head acoustic transfer function. It is shown. However, since this method cannot obtain the Doppler effect, the sound image is not sufficiently localized and realistic.

  In the position movement rendering with the Doppler effect in the conventional moving sound image presentation device, the frequency is shifted in advance up and down as necessary and then rendered in two stages so that the position is added.

JP-A-8-265900

  In this way, the conventional technology performs sound image localization with sound pressure attenuation according to distance, so the Doppler effect due to the relative position movement of the sound source / listener present in the real world is not taken into account, and the sense of reality is sufficient There was no problem. In addition, the method of shifting the frequency in advance has a problem that rendering is in two stages, so that the consistency with the physical phenomenon is low, and it is difficult to use because it requires complicated work from the user and application developer.

  In addition, since the head-related transfer function differs for each listener, if the head-related transfer function measured in advance differs from the listener's head-related transfer function, the sense of reality and localization of the sound image will be lost, and an unspecified number of listeners There is a problem that it is difficult to use.

  In the present invention, if only position information is given momentarily, both the position information and the Doppler effect are added, and the head-related transfer functions that are different among individuals are made more consistent to move the sound image with a sense of reality and localization. The object is to provide a sound source presentation device.

  According to the present invention, HRIR storage means for storing HRIR measured at a number of points in space, delay-free HRTF calculation means for removing delay from the HRIR, and HRIR calculated by the delay-free HRTF calculation means are stored. HRIR storage means, binaural arrival wavefront calculation means for calculating the Doppler effect generated in both ears, convolution calculation means, and parameter interpolation means for adjusting the time difference between the application / listener and the sound source / listener A moving sound image presentation device that presents a virtual realistic three-dimensional sound image to a listener by accurately simulating the sound wave surface reaching both ears by the position of the head, the angle of the head, and the Doppler effect can get.

  The delay-free HRIR computing means of the present invention computes a cross-correlation coefficient for two pairs of HRIR storage means, upsampling means, and HRIR for storing HRIR measured at a number of points in space, HRIR delay means that delays one HRIR so that its value is maximized, and a series of delay processing is applied to all HRIRs stored in the HRIR storage means, and then the same amount of delay is removed from all HRIRs The present invention provides an HRIR calculation method and HRIT calculation unit from which a delay component has been removed, characterized by comprising a delay removal unit that performs the above and a downsampling unit that returns the sampling frequency to be used.

  Further, the binaural arrival wavefront calculating means of the present invention performs parameter interpolation until the parameter is updated by the application with respect to the distance calculated by the transmission distance calculating means considering the diffraction of the head and the left and right transmission distance calculating means. Relative speed calculation means for calculating the relative speed of sound reaching the left and right ears by dividing the distance difference for each interpolation by the interpolation interval, and the delay time growth rate for determining the sound sample readout position of the delay line from the calculated relative speed Calculating means, pointer interpolation means for reading sound samples from the delay line, and distance attenuation adding means for expressing sound attenuation according to the transmission distance, and the sound sample reading position from the delay line as the delay time By changing every moment according to the growth rate, it was possible to simulate the wavefront reaching both ears and to add the time difference between both ears and the Doppler effect of each ear. To provide a mobile sound image presentation apparatus according to claim.

  Further, the HRIR storage means of the present invention stores HRIR information collected from a plurality of humans, and selects a HRIR suitable for the individual listener by the HRIR selection means for use. provide.

  Further, the HRIR selection means of the present invention creates a virtual moving sound image having a specific trajectory from a plurality of HRTFs, presents two moving sound sources after teaching the moving sound image trajectory created in advance to the listener, By selecting which HRTF matches the trajectory taught by the tournament method one after the other and selecting the HRTF that best matches the individual listener, the HRTF is personalized and a more localized 3D sound image is presented. A moving sound image presentation device characterized by the above is provided.

  According to the present invention, the head-related transfer function is separated into the initial delay and the main response, the delay is processed together with the Doppler effect, and the Doppler effect is added so that the spectral component is rendered later in the main response portion. As a result, the rendering method is closer to a physical phenomenon (easy to use for application creators), and the realism is good. In addition, by selecting and using a head-related transfer function that matches each individual, it is possible to improve the sense of reality and localization of the moving sound image presented to an unspecified number of listeners.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of a moving sound source presentation apparatus according to an embodiment of the present invention. Referring to FIG. 1, the moving sound source presentation device 300 receives information (sound source position 101, listener position 102, monaural sound source data 103) from the application 100, performs rendering (rendering) based on the information, and attaches it to the listener. By transmitting it to the headphones 200, a virtual realistic sound image is presented to the listener.

  The moving sound source presentation device 300 in FIG. 1 includes an HRIR storage unit 1 that stores HRIR measured at a number of points in space, and a personalized HRIR selection unit 2 that selects a personalized HRIR that best suits the viewer. The non-delayed HRTF calculating means 3 for removing the delay from the HRIR, the non-delayed HRIR storage means 4 for storing the HRIR calculated by the non-delayed HRTF calculating means 3, and the binaural for calculating the Doppler effect generated in each ear. The arrival wavefront calculation means 5, the convolution calculation means 7, and the parameter interpolation means 6 that adjusts the time difference between the application and the listener.

  FIG. 2 is a block diagram for explaining details of the delay-free HRIR computing means 3 in the present invention.

  In FIG. 2, the personalized HRIR selection means 2 in the present invention is omitted as being already selected.

  The HRIR storage means 1 measures and stores in advance a number of points in the space, and the HRIR selected by the HRIR selection means 2 is subjected to upsampling 8 times in the upsampling means 3a. In this state, the cross-correlation function is calculated by the cross-correlation coefficient inspection unit 3b for the two HRIRs, and one HRIR is delayed by the HRIR delay unit 3c so that the value becomes maximum. After performing a series of delay processes on all HRIRs stored in the HRIR storage means, the initial delay removal means 3d removes the same amount of delay from all HRIRs.

  In this embodiment, the upsampling is 8 times, but other numerical values can be selected.

The transmission distance calculation means 5a in the binaural arrival wavefront calculation means 5 in FIG. 1 is a calculation means that considers the path that the sound wave emitted from the sound source travels through both ears of the listener. In the present invention, the calculation is divided into two cases as shown in FIG. As shown in FIG. 4 (a), when the contact point of the tangent drawn from the sound source to the head assuming a spherical shape exists at a position that does not cross the straight line connecting both ears (hereinafter referred to as the ear axis), the sound source The straight line distance from the contact point to the contact point and the curved distance from the contact point to the ear are defined as the transmission distance. In this case, the transmission distance to the right ear is R ₁ + R ₂ in the figure, and the transmission distance to the left ear is L ₁ + L ₂ . On the other hand, as shown in FIG. 4 (b), when the contact exists at a position across the ear axis, the transmission distance to one ear is calculated by the linear distance to the sound source, and the other ear is the same as described above. The calculation is based on the linear distance from the sound source to the contact point and the curved distance from the contact point to the ear. In this case, the reach distance of the ear on the sound source side is D _{1 in} the figure, and the reach distance to the ear on the shadow side with respect to the sound source is S ₁ + S ₂ . As described above, the transmission distance calculation means of the moving sound image presentation apparatus according to the embodiment of the present invention is not only a linear distance between the sound source and the ear, but also a calculation method considering diffraction.

As shown in the transmission distance calculation means, the relative speed calculation means 5b needs to take the diffraction path into consideration. As the simplest calculation means, there is a method of decomposing the velocity vector of the sound source and the listener in the direction of both ears. This method depends on the linear velocity from the sound source to both ears. Therefore, in the moving sound image presentation device of the present invention, the relative velocity is calculated based on the change in the relative position in a very short time. When the relative speed is ν _r , the path distance between one ear and the sound source at a certain time is γ _n , and the distance at the next time is γ _{n + 1} , the relative speed at the interpolation interval Δt time is calculated by the following equation (1). To do.

  With this method, the relative velocity takes into account the diffraction path.

  The delay line 5d1 is prepared in order to read out sound samples reaching both ears independently for each ear. The sound source sample 103 obtained from the application 100 is put into the left and right delay lines 5d1 one sample per unit time. On the other hand, when reading the sound sample from the delay line 5d1, the relative speed calculation means 5b calculates the relative speed in consideration of the transmission path, calculates the delay time growth rate, and dynamically changes the sound sample reading position. And a sound sample reaching both ears at a certain time is calculated.

As described above, the delay time growth rate is used to control the sound sample readout position from the delay line. The delay time growth rate is expressed by the following equation, where ν _sl is the speed of the sound source and ν _ls is the speed of the listener.
Calculated by Using this delay time growth rate, the readout position per unit time is expressed by the following equation 2-1.
By incrementing each one, it is possible to simulate binaural sound waves.

The pointer interpolation means 5d2 is used when the read position on the delay line 5d1 does not indicate an integer delay line index. When the reading position indicates an integer index, the sound sample value stored in the index is output. In the interpolation when the integer index is not indicated, the sound sample value corresponding to the readout position is calculated by Lagrange interpolation. Lagrangian interpolation is a method of interpolating points on a function of a curve. When coordinates (x ₀ , y ₀ ) to (x _n , y _n ) of n + 1 points are given, an nth order polynomial passing through this n + 1 is used. Is calculated by the following equation (3).

  In this apparatus, a desired sound sample value is calculated from the data of two samples before and after using a cubic interpolation polynomial.

The distance attenuation adding means 5e refers to means for adding sound attenuation according to the transmission distance to the output sample obtained using the above means. In this device, assuming a vibrating body in an infinite baffle, ρ is the distance between the sound source and each ear, and σ is the radius of the sound source.
Attenuation by is added to the sound sample.

The moving sound image presenting apparatus is configured by the above components. The result of having performed simulation using this device is shown. The simulation conditions are
・ Sound source: Octave band noise with a center frequency of 4 kHz ・ Orbit: Linear movement backward from the direction of 50 m forward and 1.5 m left ・ Speed: 1 cm / ms (about 36 km / h)
・ HRIR: Recorded using a dummy head. As the relationship between time and frequency, the spectrogram of the sound source is shown in FIG. 4, and the spectrogram of the left output of the apparatus is shown in FIG.

  In FIG. 5, the components in the octave band band centering on 4 kHz are steadily dominant, whereas in FIG. 6, the dominant band changes with time. Thereby, it turns out that the Doppler effect is added.

  In order to investigate the relationship between time and sound source position, the relationship between time and sound source angle was examined with the horizontal axis representing time and the vertical axis representing the sound source angle from the head center. The sound source angle was calculated by examining the correspondence between the interaural level difference (ILD) of the output sound and the ILD in HRIR. The result is shown in FIG.

  The sound source angle increases counterclockwise with the front direction being 0 degrees. It can be seen from FIG. 7 that the sound source angle changes according to the trajectory moving from the left front to the left rear.

  From the above results, it can be seen that the moving sound image presentation device in the embodiment of the present invention realizes simultaneous simulation of the position and the Doppler effect.

  FIG. 3 is a conceptual diagram showing specific sound image presentation positions of the HRTF personalization means 2 in one embodiment of the present invention.

HRTF personalization is done by the following procedure.
(1) HRTFs are measured and prepared in advance from a plurality of persons (16 persons in this embodiment).
(2) A virtual moving sound image having a sagittal trajectory shown in FIG. 3B is created using the plurality of HRTFs.
(3) Teach the listener the trajectory of the moving sound image created.
(4) Presenting two moving sound images to the listener, and comparing and selecting which sound image matches the image of the sound image trajectory taught in advance.
(5) Perform this comparison in the tournament method, and finally select and select the HRTF that performed well.

  If the trajectory of the moving sound image is presented on a horizontal plane as shown in FIG. 3 (a), it is difficult to accurately reflect the result of HRTF with only the time difference between the left and right, so a sagittal plane as shown in FIG. 3 (b) is desirable. .

  The tournament used in the procedure of (5) can be personalized more strictly by using a Swiss tournament for narrowing down, as well as personalized HRTFs for the second and third candidate auditory displays. Can be requested.

  The more the number of sound images prepared in advance, the better. However, a multiple of 2 or a multiplier of 2 is preferable because of the nature of the tournament. In this embodiment, 32 HRTFs are first divided into 16 sets, and each HRTF is used to select which of the localization feelings is good. The one with a good sense of orientation will win one win and the one with a bad one will lose one. Next, create and select 16 pairs of HRTFs with the same number of wins.

  Repeat this until there is one undefeated HRTF.

  Using this Swiss tournament, you can avoid the fact that a good (more personalized) HRTF will be eliminated and not selected early.

  FIG. 8 is an example in which the moving sound image presentation device according to the present invention is applied to a sound space training device.

  It consists of a headphone with a position sensor attached to the listener, a hammer-type controller similarly equipped with a position sensor, and a moving sound image presentation device.

  In the moving sound image presenting device, a virtual moving sound image is presented from the position specified by the sound space training application and the head posture information by the position sensor provided in the headphones.

  The listener points the location of the presented sound image with the controller.

  The pointed position is detected by a position sensor provided in the controller, and if the pointed position and the position of the presentation sound image are the same, a score is given.

  By using such an application, it is possible to perform training for accurately grasping and pointing the presented moving sound image.

  The sound source presentation apparatus according to the present invention is a training apparatus for performing training for recognizing a position of a sound by presenting a three-dimensional sound image to a visually impaired person, an entertainment device for manipulating a sound image presented three-dimensionally, and the like It can be applied to various fields.

  Moreover, the moving sound image presentation apparatus manufactured by the present invention can be widely used as a high-precision auditory display.

It is a block diagram which shows one Example of the moving sound image presentation apparatus by this invention. It is a block diagram which shows the detailed structure of the delay HRIR calculation means of the moving sound image presentation apparatus in one Example of this invention. It is a figure which shows the track | orbit of the sound image shown to a viewer in the personalized HRIR selection means of the moving sound image presentation apparatus in one Example of this invention. It is a conceptual diagram explaining the calculation means of the transmission distance to both ears in the moving sound image presentation apparatus in one Example of this invention. It is a figure which shows the spectrogram of the sound source for demonstrating the effect of the moving sound image presentation apparatus of this invention. It is a figure which shows the spectrogram of the apparatus output sound for demonstrating the effect of the moving sound image presentation apparatus of this invention. It is a figure which shows the relationship between time for demonstrating the effect of the moving sound image presentation apparatus of this invention, and a sound source angle. It is a figure which shows one Example when one Example of the moving sound image presentation apparatus of this invention is applied to a training apparatus.

Explanation of symbols

1 HRIR storage means 2 Personalized HRIR selection means 3 Delayless HRIR calculation means 4 Delayless HRIR storage means 5 Binaural arrival difference wavefront calculation means 5a Transmission distance calculation means 5b Relative velocity calculation means 5c Delay time growth rate calculation means 5d Time-varying Delay circuit 5d1 Delay line 5d2 Pointer interpolation means 5e Distance attenuation addition means 6a Parameter interpolation means 6b Parameter interpolation means 7 Convolution calculation means 100 Application 101 Sound source position 102 Listener position 103 Monaural sound source data 200 Headphone 300 Mobile sound image presentation device

Claims (5)

Listening to sound images whose position changes every moment in a three-dimensional space using parameters (sound source / listener position information, monaural sound source) and head-related impulse response (HRIR: HRIR). In a moving sound image presentation device presented to a person,
HRIR storage means for storing HRIR measured at a number of points in space, delay-free HRTF calculation means for removing delay from the HRIR, HRIR storage means for storing HRIR calculated by the delay-free HRTF calculation means, It has binaural arrival wavefront calculation means for calculating the Doppler effect generated in each of the ears, convolution calculation means, and parameter interpolation means for adjusting the time difference between the application and the listener. An apparatus for presenting a moving sound image, characterized by presenting a virtual three-dimensional sound image to a listener by precisely simulating sound waves that reach both ears by an angle and a Doppler effect.   The delay-free HRIR computing means according to claim 1 computes cross-correlation coefficients for two pairs of HRIR storage means for storing HRIR measured at a number of points in space, upsampling means, and HRIR. HRIR delay means for delaying one HRIR so that the maximum value is obtained, and a series of delay processing is applied to all HRIRs stored in the HRIR storage means, and then the same amount of delay is applied from all HRIRs. A HRIR calculation method and an HRIR calculation unit from which a delay component has been removed, characterized by comprising: a delay removal unit that removes the delay component; and a downsampling unit that returns the sampling frequency to be used.   The binaural arrival wavefront calculating means according to claim 1 performs parameter interpolation until a parameter is updated by an application with respect to the distance calculated by the transmission distance calculating means considering the diffraction of the head and the left and right transmission distance calculating means, Relative speed calculation means for calculating the relative speed of sound reaching the left and right ears by dividing the distance difference for each interpolation by the interpolation interval, and delay time growth for determining the sound sample readout position of the delay line from the calculated relative speed It is composed of rate calculation means, pointer interpolation means when reading sound samples from the delay line, and distance attenuation adding means that expresses sound attenuation according to the transmission distance, and delays the sound sample reading position from the delay line By changing every moment according to the time growth rate, the wavefront reaching both ears is simulated, and the time difference between both ears and the Doppler effect of each ear can be added. Moving sound image presentation apparatus according to claim and.   The HRIR storage means according to claim 1 stores HRIR information collected from a plurality of persons, and selects and uses an HRIR that is more suitable for the individual listener by the HRIR selection means. . The HRIR selection means according to claim 4 creates a virtual moving sound source having a specific trajectory from a plurality of HRTFs, teaches the trajectory of the moving sound image created in advance to the listener, and presents the two moving sound sources. , Which one of the moving sound source matches the trajectory taught in the tournament method one after the other, by selecting the HRTF that best matches the individual listener, personalize the HRTF, and produce a more three-dimensional sound image with a better sense of localization A moving sound image presentation device with a feature that can be presented.