JP5826582B2 - Sound source direction estimation method, sound source direction estimation device, and sound source estimation image creation device - Google Patents

Description

  The present invention relates to a method and apparatus for estimating a sound source direction from sound information collected by a plurality of microphones, and to estimate a sound source using sound information collected by a microphone and video information photographed by a photographing means. The present invention relates to an apparatus for creating an image to be used.

Conventionally, as a method of estimating the sound source direction that is the direction of sound arrival, a microphone array in which a large number of microphones are arranged at equal intervals is configured, and a sound pressure signal collected by a reference microphone and each microphone are collected. A so-called acoustic technique has been devised that estimates the sound source direction from the phase difference with the sound pressure signal (see, for example, Non-Patent Document 1).
On the other hand, not a phase difference between output signals of a plurality of microphones constituting a microphone array, but a plurality of microphone pairs arranged in a straight line intersecting each other by a plurality of microphones, and a phase difference between two paired microphones There has been proposed a method for estimating the direction of a sound source from the ratio of the arrival time difference corresponding to the above and the arrival time difference between two other paired microphones (see, for example, Patent Documents 1 to 3).

Specifically, as shown in FIG. 6, two microphone pairs (M1, M3) and microphone pairs (four microphones M1 to M4) are arranged at predetermined intervals on two straight lines orthogonal to each other. M2, M4) arranged so as to constitute a microphone constituting the arrival time difference D ₁₃ of the sound input to the microphone M1, M3 constituting the microphone pair (M1, M3), said microphone pairs (M2, M4) The horizontal angle θ between the measurement point and the position of the sound source is estimated from the ratio with the arrival time difference D ₂₄ of the sound input to M2 and M4, and the fifth microphone is located at a position not on the plane formed by the microphones M1 to M4. M5 is arranged to constitute four pairs of microphones (M5, M1), (M5, M2), (M5, M3), (M5, M4). The elevation angle φ formed by the measurement point and the position of the sound source is estimated from the arrival time differences D ₁₃ , D ₂₄ and D _5j (j = 1 to 4) of the sounds collected by the microphones constituting the pair.
The arrival time difference D _ij is obtained by subjecting sound pressure waveform data obtained by A / D conversion of signals input to the two microphone pairs (M _i , M _j ) to fast Fourier transform, respectively. The cross spectrum of the waveform data is obtained and further calculated using the phase angle information of the target frequency f.
The sound source direction measured from the measurement point can be expressed by the horizontal angle θ and the elevation angle φ.

Thereby, compared with the case where the sound source direction is estimated using the microphone array, the sound source direction can be accurately estimated with a small number of microphones.
Further, at this time, a video sampling means such as a CCD camera is provided to take an image of the estimated sound source direction, and the image data and the sound source direction data are combined to generate the estimated sound source direction ( If the sound source estimation image in which θ, φ) and the sound pressure level are graphically displayed is displayed on a display screen such as a display, the sound source can be visually grasped.
Simultaneously with the sound collection, the image collection means continuously shoots the image, and the sound pressure waveform data that is the sound information and the image data that is the image information are stored in the hard disk of the computer. After collecting information and video information, the sound pressure waveform data is extracted from the hard disk to estimate the sound source direction, and the image data corresponding to the sound pressure waveform data used for the calculation of the sound source direction is extracted from the hard disk. A method of displaying a sound source estimation image by combining image data and sound source direction data is also performed.

Japanese Patent Laid-Open No. 2002-181913 JP 2006-324895 A JP 2008-224259 A

Toshiro Oga, Yoshio Yamazaki, Yutaka Kaneda; Acoustic system and digital processing, Corona, 1995

In the conventional method, the direction of the sound source and the magnitude of the incoming sound can be measured for each frequency, so that the information on the sound source can be reliably grasped. It is necessary to perform arithmetic processing to distinguish the
Also, the sound source direction analysis interval is 0.1 to 1.0 sec. For this reason, it was difficult to accurately capture impact sounds with a short period.

  The present invention has been made in view of the conventional problems, and can easily and accurately estimate the direction of the sound source of a direct sound even when the reflected sound is large, and can accurately extract the impact sound. It is an object of the present invention to provide a method and an apparatus that can be used.

As a result of intensive studies, the inventors of the present application have obtained a very short time fast Fourier in which the frequency resolution is reduced by shortening the length of the analysis section (the width of the window of the window function applied to the input signal) when obtaining the cross spectrum. If the cross spectrum is obtained by performing the conversion many times, and the centroidal phase difference (arrival time difference) is calculated from the weighted average cross spectrum obtained by weighted averaging of the obtained multiple cross spectra, the direct sound The present inventors have found that the direction of a sound source can be estimated with high accuracy and have arrived at the present invention.
That is, the invention according to claim 1 of the present application is a method for estimating the direction of a sound source from sound pressure signals of sounds collected by a plurality of microphones, and is arranged on each of two straight lines intersecting each other at predetermined intervals. A step of collecting the sound pressure signal of the incoming sound using the first and second microphone pairs, and the sound pressure signal and the second microphone collected by the microphones M1 and M3 constituting the first microphone pair. A / D conversion is performed on the sound pressure signals collected by the microphones M2 and M4 constituting the pair to obtain sound pressure waveform data of the sounds collected by the four microphones M1 to M4, respectively, A step of fast Fourier transforming the pressure waveform data; and a cross-slice of the sound pressure waveform data of the microphones M1 and M3 subjected to the fast Fourier transform. Step wherein a vector seeking cross spectrum of the sound pressure waveform data of the microphone M2, M4 calculates microphones M1, M3 between the arrival time difference D ₁₃ sounds the microphone M2, M4 between the arrival time difference D ₂₄ sounds respectively And estimating the sound source direction of the incoming sound from the calculated arrival time difference D ₁₃ in the first microphone pair and the arrival time difference D ₂₄ in the second microphone pair, and performing the fast Fourier transform In the step, the length of the analysis section is 0.1 msec. -10 msec. The step of calculating the arrival time difference by performing the very short time fast Fourier transform a number of times in succession or performing a number of times by overlapping a part of the analysis section, A step of obtaining an amplitude value of a cross spectrum obtained for each operation, and a step of obtaining a weighted average cross spectrum obtained by performing a weighted average of the cross spectrum obtained for each operation of the ultrashort-time fast Fourier transform from the amplitude value. And calculating the sound arrival time differences D ₁₃ and D ₂₄ between the microphones from the weighted average cross spectrum.
As described above, the analysis of the short time analysis was performed many times, and the average of these multiple cross spectra was obtained by the weighted average based on the amplitude value to calculate the arrival time difference, thereby reducing the reflected sound and noise components. Even in a highly reflective field, a direct sound such as an impact sound can be reliably captured, and a sound source direction can be accurately estimated for a continuous sound.

The invention according to claim 2 is the sound source direction estimating method according to claim 1, wherein, in addition to the four microphones M1 to M4, a fifth microphone that is not on a plane formed by the two pairs of microphones. The step of collecting the sound pressure signal of the incoming sound by providing M5 and calculating the arrival time difference includes the arrival time difference D ₁₃ between the microphones M1, M3 and the microphones M2, M4 constituting the two pairs of microphones. , and D _24, and calculates an arrival time difference D ₅₁ to D ₅₄ between microphones constituting the four sets of microphones pair composed of the respective said fifth microphone M5 of the four microphones M1 to M4, the sound source in the step of estimating a direction, a sound source direction of the incoming sound with the calculated arrival time difference _{D 13, D 24, D 51} ~D 54 Characterized in that it estimated.
Thus, since the elevation angle φ can be estimated in addition to the horizontal angle θ of the sound source direction viewed from the measurement point, the estimation accuracy of the sound source direction can be improved.

According to a third aspect of the present invention, there are provided a first microphone pair and a second microphone pair disposed at predetermined intervals on two straight lines that intersect with each other, and a fifth microphone that is not on a plane formed by the two microphone pairs. A sound source direction estimating device for estimating the direction of the sound source from the sound pressure signal of the sound collected by the sound collecting means, wherein each sound pressure signal collected by each microphone is converted into a digital signal. An A / D converter for converting, a fast Fourier transformer for fast Fourier transforming sound pressure waveform data, which is a sound pressure signal converted into the digital signal,
Among the sound pressure waveform data subjected to the fast Fourier transform, a cross spectrum of sound pressure waveform data of sound collected by two microphones constituting the first microphone pair and 2 constituting the second microphone pair Cross spectrum of sound pressure waveform data of sound collected by one microphone, and sound pressure waveform data of sound collected by each of the fifth microphone and the four microphones constituting the first and second microphone pairs cross spectrum and the cross-spectrum calculation means for calculating a time of arrival difference D _13, D ₂₄ of the sound between the microphones respectively constituting the first and second microphone pair from the cross-spectral and the fifth microphone and The arrival of sound between the four microphones constituting the two microphone pairs And the arrival time difference calculating means for calculating a reach time difference D ₅₁ to D _54, and a sound source direction estimating means for estimating the sound source direction using the arrival time the calculated difference _{D 13, D 24, D 51} ~D 54, The fast Fourier transformer has an analysis section length of 0.1 msec. -10 msec. The ultra-short-time fast Fourier transform is continuously performed many times, or a part of the analysis section is overlapped many times, and the cross spectrum calculation means is operated for each operation of the ultra-short-time fast Fourier transform. And calculating a weighted average cross spectrum obtained by weighted averaging the cross spectrum obtained for each operation of the extremely short time fast Fourier transform from the amplitude value. The calculating means calculates the arrival time differences D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ between the microphones from the weighted average cross spectrum.
By adopting such a configuration, a weighted average cross spectrum of sound pressure waveform data subjected to short-time fast Fourier transform can be obtained reliably, so that a sound source direction estimating device that can accurately estimate the sound source direction of a direct sound Can be obtained.

According to a fourth aspect of the present invention, there is provided a fifth embodiment in which the first and second microphone pairs disposed at predetermined intervals on two straight lines that intersect with each other and the plane formed by the two microphone pairs are not on the fifth plane. A sound collecting means including a microphone and a photographing means for photographing a sound source direction image, and a sound pressure signal of the sound propagated from the sound source collected by the sound collecting means, the sound pressure signal, and the photographing means. A sound source estimation image creating apparatus that creates a sound source estimation image that is an image in which a graphic showing a direction of a sound source is drawn from a video signal in a direction of the sound source, and the sound pressure collected by each microphone An A / D converter that converts a signal and a video signal captured by a photographing means into digital signals, and a fast Fourier transform of sound pressure waveform data that is a sound pressure signal converted into the digital signals, respectively A fast Fourier transformer, a cross spectrum of sound pressure waveform data of sound collected by two microphones constituting the first microphone pair of the sound pressure waveform data subjected to the fast Fourier transform, and the second Of the sound pressure waveform data of the sound collected by the two microphones constituting the microphone pair, and each of the four microphones constituting the fifth microphone and the first and second microphone pairs. The cross spectrum calculation means for calculating the cross spectrum of the sound pressure waveform data of the sound and the sound arrival time differences D ₁₃ and D ₂₄ between the microphones constituting the first and second microphone pairs from the cross spectrum, respectively. And the fifth microphone and the four microphones constituting the two pairs of microphones And the arrival time difference calculating means for calculating an arrival time difference D ₅₁ to D ₅₄ of the sound between the microphones, the sound source direction estimation for estimating the sound source direction using the arrival time the calculated difference _{D 13, D 24, D 51} ~D 54 Means for synthesizing the estimated sound source direction data and the image data which is the video signal converted into the digital signal, and is a sound source estimation image which is an image in which a figure showing the estimated sound source direction is drawn Sound source estimation image creating means for creating an image, wherein the fast Fourier transformer has an analysis section length of 0.1 msec. -10 msec. The ultra-short-time fast Fourier transform is continuously performed many times, or a part of the analysis section is overlapped many times, and the cross spectrum calculation means is operated for each operation of the ultra-short-time fast Fourier transform. And calculating a weighted average cross spectrum obtained by weighted averaging the cross spectrum obtained for each operation of the extremely short time fast Fourier transform from the amplitude value. The calculating means calculates the arrival time differences D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ between the microphones from the weighted average cross spectrum.
By adopting such a configuration, the sound source direction of the direct sound can be estimated with high accuracy, and a sound source estimation image for estimating the sound source can be easily created.
Further, the invention described in claim 5 is the sound source estimation image creating apparatus according to claim 4, further comprising display means having a display screen for displaying the created sound source estimation image. To do.
Thereby, since the sound source estimation image can be displayed on the display screen of the display means, the operator can easily visually recognize the sound source.

  The summary of the invention does not list all necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

It is a functional block diagram which shows the structure of the image display system for sound source estimation which concerns on embodiment of this invention. It is a flowchart which shows the display method of the image for sound source estimation using the image display system for sound source estimation which concerns on this Embodiment. It is a figure for demonstrating very short time fast Fourier transform. It is a figure which shows an example of the image for sound source estimation by this invention. It is a figure which shows an example of the conventional image for sound source estimation. It is a figure which shows the arrangement | sequence of the microphone in the sound source search method using the conventional microphone pair.

  Hereinafter, the present invention will be described in detail through embodiments, but the following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

FIG. 1 is a functional block diagram showing a configuration of a sound source estimation image display system.
The sound source estimation image display system includes a sound / video sampling unit 10, a data processing device 20, a calculation device 30, a display device 40, and a storage device 50.
The data processing device 20 includes an amplifier 21, an A / D converter 22, and a video input / output unit 23.
The calculation device 30 includes a buffer 31, sound pressure waveform data extraction means 32, cross spectrum calculation means 33, arrival time difference calculation means 34, sound source direction estimation means 35, image data extraction means 36, and sound source estimation image. Creating means 37. The arithmetic device 30 is configured by software of a personal computer, for example.
The display device 40 includes a display screen 40M that displays a sound source position estimation image that is an image for estimating a sound source position, which will be described later.
The storage device 50 is a memory composed of, for example, a hard disk of a personal computer.

The sound / video collection unit 10 includes a sound collection unit 11, a CCD camera (hereinafter referred to as a camera) 12, a microphone fixing unit 13, a camera support base 14, a support column 15, and a base 16 as a video collection unit. Is provided.
The sound collection means 11 includes a plurality of microphones M1 to M5.
The arrangement of the microphones M1 to M5 is the same as that shown in FIG. 6, and two microphone pairs (M1, M1) are arranged with four microphones M1 to M4 arranged at predetermined intervals on two straight lines orthogonal to each other. M5) and the microphone pair (M2, M4) are arranged so that the fifth microphone M5 is not on the plane formed by the microphones M1 to M4. Specifically, a square formed by the microphones M1 to M4 is formed. It arranges at the position of the apex of the quadrangular pyramid as the bottom. Thereby, four pairs of microphones (M5, M1) to (M5, M4) are further configured.
In this example, the shooting direction of the camera 12 is set to a direction that passes through the intersection of the two orthogonal lines and forms approximately 45 ° with the two lines. Therefore, the direction of the sound / image collection unit 10 is the direction of the white arrow D in FIG. The camera 12 collects an image corresponding to the direction of the sound / image collection unit 10.

The microphones M1 to M5 are installed on the microphone fixing unit 13, the camera 12 is installed on the camera support base 14, and the microphone fixing unit 13 and the camera support base 14 are connected by three support columns 15. That is, the sound collection means 11 and the camera 12 are integrated. The microphones M1 to M5 are disposed on the upper part of the camera 12.
The base 16 includes a rotary support 16a and a support 16b including a rotation mechanism (not shown) that rotates the rotary support 16a, and the camera support 14 is mounted on the rotary support 16a. Therefore, the sound collection means 11 and the camera 12 can be rotated together by rotating the rotary support 16a. The rotation mechanism may be omitted, and the operator may change the direction of the sound / video sampling unit 10 by rotating the base 16.
The microphones M1 to M5 each measure a sound pressure level that is a magnitude of a sound pressure signal of sound coming from a sound source (not shown).

The amplifier 21 includes a low-pass filter, removes high frequency noise components from the sound pressure signals of the sounds collected by the microphones M1 to M5, amplifies the sound pressure signals, and outputs them to the A / D converter 22.
The A / D converter 22 generates sound pressure waveform data obtained by A / D converting the sound pressure signal, and sends the sound pressure waveform data to the sound pressure waveform data storage area 31 a of the buffer 31. The sound pressure waveform data storage area 31a is divided into small areas 311 to 315, and the sound pressure waveform data of the microphones M1 to M5 are stored in the small areas 311 to 315, respectively.
The video input / output means 23 inputs video signals continuously shot by the camera 12 and outputs image data in the shooting direction every preset screen switching time T _p (eg, T _p = 1/30 seconds). The data is sent to the image data storage area 31b of the buffer 31.
The image data output at each predetermined time T _p is image data that constitutes one screen displayed on the display screen 40M of the display device 40, that is, an image for “one frame” of a so-called moving image.

The sound pressure waveform data extraction means 32 sequentially extracts sound pressure waveform data of the length _TF of the analysis section of a preset fast Fourier transform (hereinafter referred to as FFT) from the sound pressure waveform data storage area 31a of the buffer 31. And sequentially output to the fast Fourier transformers 331 to 335 of the cross spectrum calculation means 33. Specifically, the sound pressure waveform data of the microphones M1 to M5 extracted from the small regions 311 to 315 is output to the fast Fourier transformer 33k.
The sound pressure waveform data may be output directly from the A / D converter 22 to the fast Fourier transformer 33k. The sound pressure waveform data may be stored in the storage device 50 from the A / D converter 22 and output from the storage device 50 to the fast Fourier transformer 33k. However, considering the processing speed, the sound pressure waveform data is It is preferable to output the data directly from the A / D converter 22 or through the buffer 31 to the fast Fourier transformer 33k.

The cross spectrum calculation means 33 includes a fast Fourier transformer 33k, a cross spectrum calculator 33m, and a weighted average cross spectrum generator 33M.
The fast Fourier transformer 33k includes five fast Fourier transformers 331 to 335, and for each of the sound pressure waveform data of the microphone Mk (k = 1 to 5), the length _TF of the analysis section is, for example, 2 msec. The extremely short time fast Fourier transform is performed N times within a preset measurement time _Tc , and the results are sequentially output to the cross spectrum calculator 33m.
Note that the extremely short-time fast Fourier transform is continuously performed using a window function whose length is equal to the length of the analysis section. However, in this example, since the length of the analysis section is short, the time is around. It is preferable to overlap a part of the analysis interval.

The cross spectrum calculator 33m includes six cross spectrum calculators 33x, 33y, and 33a to 33d, and six preset groups output from the fast Fourier transformers 331 to 335 for each extremely short time FFT processing. The cross spectrum p _n (f) and the amplitude w _n (f) of the microphone pair are sequentially obtained (n = 1 to N).
Specifically, the cross spectrum calculator 33x and X _n1 (f) which are sound pressure waveform data of the microphones M1 and M3 constituting the microphone pair (M1, M3) output from the fast Fourier transformers 331 and 333, and A cross spectrum p _n13 (f) with X _n3 (f) and its amplitude w _n13 (f) are sequentially obtained for each extremely short time FFT processing.
Cross-spectral calculator 33y includes a microphone pair outputted from the fast Fourier transformer 332, 334 (M2, M4) is a sound pressure waveform data of the microphone M2, M4 constituting the X _n2 (f) and X _n4 (f) The cross spectrum p _n24 (f) and its amplitude w _n24 (f) are obtained.
The cross spectrum calculators 33a to 33d are X _n5 (f) which is the sound pressure waveform data of the microphone M5 output from the fast Fourier transformer 335 and the microphones M1 to M4 output from the fast Fourier transformers 331 to 334, respectively. The cross spectrum p _n5j (f) with the sound pressure waveform data X _ni (f) and the amplitude w _n5j (f) (j = 1 to 4) are respectively obtained.
The cross spectrum p _n (f) is calculated for each frequency f.

The weighted average cross spectrum generator 33M includes six weighted average cross spectrum generators 33X, 33Y, and 33A to 33D, and N cross spectra p obtained by the cross spectrum calculators 33x, 33y, and 33a to 33d, respectively. _n Find the weighted average cross spectrum of (f).
The weighted average cross spectrum generator 33X temporarily stores n = 1 to N cross spectra p _n13 (f) and their amplitudes w _n13 (f) sequentially output from the cross spectrum calculator _33x in a memory (not shown). The cross spectrum p _n13 (f) is weighted and averaged by the amplitude w _n13 (f), and the weighted average cross spectrum P ₁₃ (f) between the sound _pressure signal sampled by the microphone M1 and the sound _pressure signal sampled by the microphone M3. Ask for.
The weighted average cross spectral generator 33Y were harvested cross spectral p _n24 determined by cross-spectral calculator 33y to (f) in its amplitude w _n24 (f) and the sound pressure signals collected by a microphone M2 with microphone M4 A weighted average cross spectrum P ₂₄ (f) with the sound pressure signal is obtained.
The weighted average cross spectrum generators 33A to 33D perform the weighted average of the cross spectra p _n5j (f) obtained by the cross spectrum calculators 33a to 33d, respectively, using the amplitude w _n5j (f), and the sound pressure collected by the microphone M5. A weighted average cross spectrum P _5j (f) between the signal and the sound pressure signal collected by the microphone Mj is obtained (j = 1 to 4).

Ｄ₁₃はマイクロフォン対（Ｍ１，Ｍ３）を構成するマイクロフォンＭ１，Ｍ３に入力する音の到達時間差、Ｄ₂₄はマイクロフォン対（Ｍ２，Ｍ４）を構成するマイクロフォンＭ２，Ｍ４に入力する音の到達時間差、Ｄ_5j（ｊ＝１〜４）は第５のマイクロフォンＭ５に入力する音圧信号とマイクロフォンＭ１〜Ｍ４のそれぞれに入力する音圧信号との到達時間差である。
到達時間差Ｄ_ijは周波数ｆ毎に算出する。
音源方向推定手段３５では、前記求められた到達時間差Ｄ₁₃，Ｄ₂₄及び到達時間差Ｄ_5j（ｊ＝１〜４）から、下記の式（２），（３）を用いて、計測点から見た到来した音の方向である水平角θと仰角φとを算出することで、音源方向を推定する。

The arrival time difference calculation means 34 configures each microphone pair (M _i , M _j ) using the following equation (1) from the weighted average cross spectrum P _ij (f) obtained by the weighted average cross spectrum generator 33M. A sound arrival time difference D _ij between the microphones M _i and M _j is calculated.

D ₁₃ is a difference in arrival time of sounds input to the microphones M 1 and M 3 constituting the microphone pair (M 1, M 3), D ₂₄ is a difference in arrival time of sounds input to the microphones M 2 and M 4 constituting the microphone pair (M 2, M 4), D _5j (j = 1 to 4) is a difference in arrival time between the sound pressure signal input to the fifth microphone M5 and the sound pressure signal input to each of the microphones M1 to M4.
The arrival time difference D _ij is calculated for each frequency f.
The sound source direction estimating means 35 uses the following expressions (2) and (3) from the obtained arrival time differences D ₁₃ and D ₂₄ and the arrival time difference D _5j (j = 1 to 4) to see from the measurement point. The sound source direction is estimated by calculating the horizontal angle θ and the elevation angle φ, which are directions of the incoming sound.

The image data extracting means 36 is the time closest to the time corresponding to the center of the measurement time T _c described above, that is, the time when the N / 2th extremely short time FFT processing is performed from the image data storage area 31b of the buffer 31. Then, the image data taken is extracted and output to the sound source estimation image creating means 37.
The sound source estimation image creating means 37 synthesizes the horizontal angle θ and elevation angle φ data estimated by the sound source direction estimating means 35 and the image data extracted by the image data extracting means 36, and the direction of the sound source in the image. A sound source direction estimation image in which a graphic indicating the size is drawn is created and output to the display device 40.
The storage device 50 stores the horizontal angle θ and elevation angle φ data and the image data used for the sound source direction estimation image together with the measurement time. The measurement time is the shooting time of the image data used for the sound source direction estimation image.

Next, a sound source direction estimation method and a sound source estimation image display method using the sound source estimation image display system of this example will be described with reference to the flowchart of FIG.
First, after connecting the sound / video sampling unit 10, the data processing device 20, the arithmetic device 30, and the display device 40, the sound / video sampling unit 10 is set at a measurement point (step S10).
The operator directs the shooting direction of the camera 12 to the planned measurement location, sees the display screen 40M and confirms that the camera 12 is shooting the planned measurement location, and then simultaneously collects sound with the microphones M1 to M5. Then, an image of the measurement planned place is collected by the camera 12 (step S11).
Next, the sound pressure signal of the sound collected by the microphones M1 to M5 is amplified and A / D converted, and this A / D converted digital signal (hereinafter referred to as sound pressure waveform data) is stored in the sound file storage area 31a of the buffer 31. In addition, the video signal of the camera 12 is A / D converted, and the A / D converted digital signal (hereinafter referred to as image data) is stored in the moving image file storage area 31b of the buffer 31 (step S12).

Next, sound pressure waveform data having a preset length _TF is sequentially extracted from the sound pressure waveform data storage area 31a of the buffer 31 and subjected to extremely short-time fast Fourier transform (step S13). The sound pressure waveform data of the microphone Mi and the sound pressure waveform data of the microphone Mj constituting the microphone pair (Mi, Mj) set in advance are extracted from the sound pressure waveform data subjected to the fast Fourier transform for a short time, and the cross spectrum p is obtained. _Nij is calculated and the amplitude (amplitude value) w _nij of the cross spectrum is calculated (step S14). Note that p _nij is a cross spectrum of the sound pressure waveform data of the microphone Mi and the sound pressure waveform data of the microphone Mj that have been subjected to the fast Fourier transform for the n-th time (n = 1 to N).
The calculation of the cross spectrum p _nij and the amplitude value w _nij is performed for each frequency band determined according to the length T _F of the analysis section and the sampling period. In this example, the cross spectrum p _ij (f) is obtained by dividing the frequency band into three frequency bands of 10 to 500 Hz, 500 to 1000 Hz, and 1000 to 7500 Hz.
As described above, the extremely short time fast Fourier transform has an analysis interval length _TF of 2 msec. In this example, this extremely short-time fast Fourier transform is performed many times within a preset measurement time _Tc .
Specifically, as shown in FIG. 3A, in contrast to the length (about 1.0 sec.) Of the conventional FFT analysis section T ₀ , in this example, as shown in FIG. The length _TF of the FFT analysis interval is extremely shortened, and the extremely short time fast Fourier transform is performed N times (N ≧ 100) continuously over the length of the analysis interval T ₀ . The analysis section length _TF is 0.1 msec. -10 msec. Is preferably in the range of 1 msec. ~ 2 msec. More preferably.
Note that the extremely short-time fast Fourier transform may be continuously performed using a window function whose length is equal to the length of the analysis section. However, since the length of the analysis section is short, FIG. As shown, it is preferable to carry out by overlapping a part of analysis sections that are temporally mixed.

In step S15, it is determined whether or not the calculation of the cross spectrum has been completed.
If the calculation of the cross spectrum has not been completed, the process returns to step S13, the sound pressure waveform data to be analyzed next is taken out from the sound pressure waveform data storage area 31a, and the extremely short time fast Fourier transform is performed to perform the cross spectrum. The operation of calculating is repeated. When the calculation of the cross spectrum is completed, the process proceeds to step S16, and weighting is performed from the N cross spectra p _n (f) and the amplitudes w _n (n = 1 to N) obtained by N operations. An average cross spectrum P (f) is obtained.
The weighted average cross spectrum P ₁₃ (f) can be expressed by the following equation.
P ₁₃ (f) = {Σw _n13 (f) · p _n13 (f)} / {Σw _n13 } …… Σ is the sum of n = 1 to _N.
Then, weighted average from the cross spectrum P _ij (f), calculates the microphone M _i, the arrival time difference D _ij sound between M _j (step S17), the formula (2) described above from these arrival time differences D _ij, ( The horizontal angle θ and the elevation angle φ are calculated using 3), and the sound source direction of the incoming sound is estimated (step S18).
The weighted average cross spectrum P _ij (f), since by using the weighted averages of the cross-spectrum p _n (f) in its amplitude w _n, components of large reflection sound amplitude is small and the amplitude of variation than direct sound conventional Therefore, the sound arrival time difference D _ij between the microphones M _i and M _j is calculated by using the above-described equation (1) because it is much smaller than the reflected sound component obtained from the cross spectrum P _ij (f). Only the arrival time difference D _ij of the direct sound can be extracted.
In addition, in the conventional FFT, when an impact sound is generated, the impact sound is not a periodic sound and the duration is short, so the sound source of the impact sound cannot be accurately grasped. In the example, since the cross spectrum p _n (f) of the sound pressure waveform data subjected to extremely short-time fast Fourier transform is weighted and averaged with the amplitude w _n , the impact sound is accurately obtained even when the duration of the impact sound is short. I can grasp it.

After the estimation of the sound source direction is completed, the image data obtained by photographing the sound source direction and the estimated horizontal angle θ and elevation angle φ data are combined, and for example, the radius indicates the size of the arrival sound. A sound source direction estimation image in which a graphic indicating the direction of the sound source and the size of the sound, such as a circle indicating the frequency, is created and displayed on the display screen 40M of the display means 40 (step S18).
4 is a diagram showing a sound source direction estimation image in a vehicle interior as an example of a sound source direction estimation image, FIG. 5 is a diagram showing a sound source direction estimation image created using a conventional sound source estimation method, and the horizontal axis is a horizontal angle. θ, the vertical axis is the elevation angle φ.
In FIG. 4, a circle with a left-slanted diagonal line indicates a sound source with a frequency band of 10 to 500 Hz, a circle with a diagonally downward-sloping line indicates a sound source with a frequency band of 500 to 1000 Hz, and a circle with a mesh has a frequency band of 1000 It is a sound source of ˜1500 Hz.
On the other hand, in FIG. 5, the sound source direction is obtained by fast Fourier transform by the method shown in FIG. For comparison, all the bands from 31.5 to 500 Hz are circles with a slanting left slope, all the bands from 500 to 1000 Hz are circles with a slanting right slope, and all the bands from 1000 to 7500 Hz are meshed. It was made a circle.
As is clear from comparison between FIG. 4 and FIG. 5, in the conventional method, not only the reflected sound is large, but also the direct sound and the reflected sound vary in frequency, whereas in the method of the present embodiment, Although the accuracy of the information about the frequency band is low, there is no reflected sound and there is little variation in the position of the sound source. Therefore, by using the method of the present embodiment, it was confirmed that the sound source direction of the direct sound can be estimated easily and accurately even in a field where the reflected sound is large.

In the above-described embodiment, the sound source direction of the incoming sound is estimated from the sound pressure signals collected by the first and second microphone pairs arranged at predetermined intervals on two straight lines intersecting each other. The invention is not limited to this, and can be applied to a sound source estimation direction using a microphone array.
In the above example, the weighted average cross spectrum is obtained by weighting and averaging the N cross spectra with the amplitude value. However, the weighted average may be obtained by the square of the amplitude value.
In the above example, the horizontal angle θ and the elevation angle φ formed by the measurement point and the sound source position are estimated using the five microphones M1 to M5. However, when the sound source position is sufficient, the horizontal angle θ is sufficient. The microphone M5 may be omitted, and only two pairs of microphones (M1, M3) and (M2, M4) arranged at predetermined intervals on two straight lines that intersect with each other may be used.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the embodiment. It is apparent from the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  As described above, according to the present invention, even when there is a reflected sound, only the sound source direction of the direct sound can be estimated easily and accurately, and the impact sound can also be accurately extracted. A highly accurate sound source direction estimating apparatus can be provided.

10 sound / video sampling unit, 11 sound sampling means, 12 CCD camera,
13 microphone fixing part, 14 camera support base, 15 struts, 16 base,
20 data processing devices, 21 amplifiers, 22 A / D converters, 23 video input / output means,
30 arithmetic unit 31, buffer, 31a sound data storage area,
31b Image data storage area, 32 sound pressure waveform data extraction means,
33 cross spectrum calculation means, 33k fast Fourier transform,
33m cross spectrum calculator, 33M weighted average cross spectrum generator,
34 arrival time difference calculating means, 35 sound source direction estimating means, 36 image data extracting means,
37 sound source estimation image creation means,
40 display device, 40M display screen, 50 storage device,
M1-M5 microphones.

Claims (5)

A method for estimating the direction of a sound source from sound pressure signals of sounds collected by a plurality of microphones,
Collecting a sound pressure signal of an incoming sound using first and second microphone pairs disposed at predetermined intervals on two straight lines intersecting each other;
The sound pressure signals collected by the microphones M1 and M3 constituting the first microphone pair and the sound pressure signals collected by the microphones M2 and M4 constituting the second microphone pair are A / D converted, respectively. Obtaining respective sound pressure waveform data of sounds collected by the four microphones M1 to M4;
Fast Fourier transform each sound pressure waveform data;
And the fast Fourier transformed the microphone M1, M3 arrival time difference D ₁₃ of the sound between the microphones M1, M3 and a cross spectrum of the sound pressure waveform data determined and a cross spectrum of the sound pressure waveform data of the microphone M2, M4 of Calculating a sound arrival time difference D ₂₄ between the microphones M2 and M4;
Estimating the sound source direction of the incoming sound from the calculated arrival time difference D ₁₃ in the first microphone pair and the arrival time difference D ₂₄ in the second microphone pair;
With
In the fast Fourier transform step, the length of the analysis section is 0.1 msec. -10 msec. Or performing a very short time fast Fourier transform a number of times continuously, or a number of times by overlapping a part of the analysis interval,
The step of calculating the arrival time difference includes:
Obtaining an amplitude value of a cross spectrum obtained for each operation of the ultrashort-time fast Fourier transform;
Obtaining a weighted average cross spectrum obtained by performing a weighted average of the cross spectrum obtained for each operation of the very short time fast Fourier transform from the amplitude value;
Calculating sound arrival time differences D ₁₃ and D ₂₄ between the microphones from the weighted average cross spectrum;
A sound source direction estimation method comprising: In addition to the four microphones M1 to M4, a fifth microphone M5 that is not on the plane formed by the two pairs of microphones is provided to collect a sound pressure signal of the incoming sound and calculate the arrival time difference Then
It is composed of arrival time differences D ₁₃ and D ₂₄ between the microphones M1 and M3 and the microphones M2 and M4 constituting the two pairs of microphones, the fifth microphone M5, and the four microphones M1 to M4. Calculating arrival time differences D _{51 to} D ₅₄ between the microphones constituting the four microphone pairs;
In the step of estimating the sound source direction,
DOA estimation method according to claim 1, characterized in that for estimating the sound source direction of the incoming sound with the calculated arrival time difference _{D 13, D 24, D 51} ~D 54. Sound collecting means comprising first and second microphone pairs disposed on two straight lines intersecting each other at a predetermined interval and a fifth microphone not on a plane formed by the two microphone pairs; A sound source direction estimating device for estimating the direction of a sound source from the sound pressure signal of the sound collected by the sound collecting means,
An A / D converter that converts a sound pressure signal collected by each microphone into a digital signal;
A fast Fourier transformer for fast Fourier transforming sound pressure waveform data that is a sound pressure signal converted into the digital signal;
Among the sound pressure waveform data subjected to the fast Fourier transform, a cross spectrum of sound pressure waveform data of sound collected by two microphones constituting the first microphone pair and 2 constituting the second microphone pair Cross spectrum of sound pressure waveform data of sound collected by one microphone, and sound pressure waveform data of sound collected by each of the fifth microphone and the four microphones constituting the first and second microphone pairs Cross spectrum calculation means for calculating the cross spectrum with
Differences in sound arrival times D ₁₃ and D ₂₄ between the microphones constituting the first and second microphone pairs from the cross spectrum, and between the fifth microphone and the four microphones constituting the two microphone pairs. Arrival time difference calculating means for calculating the arrival time differences D _{51 to} D ₅₄ of the sound of
Sound source direction estimating means for estimating a sound source direction using the calculated arrival time differences D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ ,
The fast Fourier transformer has an analysis section length of 0.1 msec. -10 msec. Or performing a very short time fast Fourier transform a number of times continuously, or a number of times by overlapping a part of the analysis interval,
The cross spectrum calculation means obtains the amplitude value of the cross spectrum obtained for each operation of the extremely short time fast Fourier transform, and obtains the cross spectrum obtained for each operation of the extremely short time fast Fourier transform, Obtain a weighted average cross spectrum that is weighted average from the amplitude value,
The arrival time difference calculating means calculates a sound arrival time difference D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ between the microphones from the weighted average cross spectrum. Sound collecting means comprising first and second microphone pairs arranged at predetermined intervals on two intersecting straight lines and a fifth microphone not on a plane formed by the two microphone pairs, and a sound source direction A photographing means for photographing the image of, and from the sound pressure signal of the sound propagated from the sound source collected by the sound collecting means, the sound pressure signal, and the video signal in the direction of the sound source photographed by the photographing means, A sound source estimation image creating apparatus that creates a sound source estimation image that is an image in which a graphic showing a direction of a sound source is drawn,
An A / D converter that converts the sound pressure signal collected by each microphone and the video signal photographed by the photographing means into digital signals,
A fast Fourier transformer that performs fast Fourier transform on the sound pressure waveform data that is the sound pressure signal converted into the digital signal;
Among the sound pressure waveform data subjected to the fast Fourier transform, a cross spectrum of sound pressure waveform data of sound collected by two microphones constituting the first microphone pair and 2 constituting the second microphone pair Cross spectrum of sound pressure waveform data of sound collected by one microphone, and sound pressure waveform data of sound collected by each of the fifth microphone and the four microphones constituting the first and second microphone pairs Cross spectrum calculation means for calculating the cross spectrum with
Differences in sound arrival times D ₁₃ and D ₂₄ between the microphones constituting the first and second microphone pairs from the cross spectrum, and between the fifth microphone and the four microphones constituting the two microphone pairs. Arrival time difference calculating means for calculating the arrival time differences D _{51 to} D ₅₄ of the sound of
Sound source direction estimating means for estimating a sound source direction using the calculated arrival time differences D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ ;
The estimated sound source direction data and the image data that is the video signal converted into the digital signal are combined to create a sound source estimation image that is an image in which a figure showing the estimated sound source direction is drawn. Sound source estimation image creation means for
The fast Fourier transformer has an analysis section length of 0.1 msec. -10 msec. Or performing a very short time fast Fourier transform a number of times continuously, or a number of times by overlapping a part of the analysis interval,
The cross spectrum calculation means obtains the amplitude value of the cross spectrum obtained for each operation of the extremely short time fast Fourier transform, and obtains the cross spectrum obtained for each operation of the extremely short time fast Fourier transform, Obtain a weighted average cross spectrum that is weighted average from the amplitude value,
The arrival time difference calculating means calculates a sound arrival time difference D ₁₃ , D ₂₄ , D _{51 to} D ₅₄ between the microphones from the weighted average cross spectrum.   5. The sound source estimation image creating apparatus according to claim 4, further comprising display means having a display screen for displaying the created sound source estimation image.

Images