JP2010206419A - Method and device for estimating sound source - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and device for estimating sound source positions by extracting arbitrary portions of measurement data at site. <P>SOLUTION: Sound information and video information are simultaneously sampled by a sound/video sampling unit 10, where a plurality of microphones M1-M5 are integrated with a camera 12. Audio waveform data, namely sound information, and image data, namely video information, are stored in a storage means 24. Only sound pressure waveform data is extracted from the storage means 24 to display a graph of a time series waveform of a sound pressure level on a display screen 25M of a display means 25. Time t<SB>z</SB>for analysis of a sound source direction is designated on the graph. Then, the sound pressure waveform data of analysis time length T<SB>w</SB>with the designated time t<SB>z</SB>as the center are used, and the phase difference between sound pressure signals of the sound sampled by the plurality of microphones M1-M5 is calculated to estimate the direction of the sound source. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a method and apparatus for estimating a sound source using sound information collected by a plurality of microphones and video information photographed by a photographing means.

Conventionally, as a method of estimating the direction of sound arrival, a microphone array in which a large number of microphones are arranged at equal intervals is constructed, and the sound source direction that is the direction of sound wave arrival is estimated from the phase difference of each microphone relative to the reference microphone. So-called acoustic techniques have been devised (for example, see Non-Patent Document 1).
On the other hand, not a phase difference between output signals of a plurality of microphones arranged at a measurement point, but a plurality of microphone pairs arranged in a straight line intersecting each other from the plurality of microphones, and the position between the two microphones constituting the pair There has been proposed a method of estimating a sound source direction from a ratio between an arrival time difference corresponding to a phase difference and an arrival time difference between two paired two microphones (see, for example, Patent Documents 1 to 3).

  Specifically, as shown in FIG. 5, two microphone pairs (M1, M3) and microphone pairs (four microphones M1 to M4) are arranged at predetermined intervals on two straight lines orthogonal to each other. M2, M4) are arranged so as to constitute the microphone pair (M1, M3) and the microphones constituting the microphone pair (M2, M4) and the arrival time difference of the sound pressure signals input to the microphones M1, M3. The horizontal angle θ formed by the measurement point and the sound source position is estimated from the ratio with the arrival time difference between the sound pressure signals input to M2 and M4, and the fifth microphone M5 is not on the plane formed by the microphones M1 to M4. Further, four microphone pairs (M5, M1), (M5, M2), (M5, M3), (M5, M4) are configured and arranged in the positions. The elevation angle φ formed by the measurement point and the sound source position is estimated from the arrival time difference between the microphones constituting the pair of microphones.

Thereby, compared with the case where the sound source direction is estimated using the microphone array, the sound source direction can be accurately estimated with a small number of microphones.
Further, at this time, a video sampling means such as a CCD camera is provided to shoot a video of the estimated sound source direction, and then the video data and the data of the sound source direction are combined to generate the estimated sound source in the video. If the direction and the sound pressure level are displayed graphically, the sound source can be grasped visually.
At the same time as the sound is collected, the video is continuously captured by the video sampling means, and the video information is stored in the computer together with the sound information, and then the direction of the sound source is calculated and estimated in the video. There is also a method of estimating a sound source by displaying a figure of the sound source direction and sound pressure level.

Japanese Patent Laid-Open No. 2002-181913 JP 2006-324895 A JP 2008-224259 A

Toshiro Oga, Yoshio Yamazaki, Yutaka Kaneda; Acoustic system and digital processing, Corona, 1995

  By the way, in the conventional method, after the sound information and the video information are taken into the computer, the direction of the sound source is calculated using all the taken-in information. For this reason, it took time to analyze the sound source position, and it was impossible to immediately know the estimation result of the sound source position at the site. In addition, there was a problem that the validity of the measurement could not be verified.

  The present invention has been made in view of the conventional problems, and an object of the present invention is to provide a method and apparatus for estimating a sound source position by extracting an arbitrary part of measurement data in the field.

  The invention according to claim 1 of the present application collects sound information and video information at the same time using a sound / video sampling unit in which a plurality of microphones and photographing means are integrated, and A sound source estimation method for estimating a sound source using sound pressure waveform data that is data of a sound pressure signal and video image data, and storing the collected sound and video data in a storage means, respectively, After extracting only the sound pressure waveform data from the storage means and displaying a graph of the time series waveform of the sound pressure level, after specifying the time position to analyze the sound source direction on this graph, the specified time position is included Using sound pressure waveform data of a predetermined analysis time length, a phase difference between sound pressure signals of sounds collected by the plurality of microphones is calculated to estimate a sound source direction, and the estimated sound source direction and the analysis time Within A sound source position estimation image in which a figure showing the estimated sound source direction is drawn is synthesized by combining with the shadowed image data, and the sound source is estimated using the sound source position estimation image. .

  The invention according to claim 2 is the sound source estimation method according to claim 1, wherein sound information and video information are obtained using a sound / video sampling unit in which a plurality of microphones and photographing means are integrated. The sound pressure signal of the collected sound is A / D converted and stored in the storage means as sound pressure waveform data, and the video signal is A / D converted to image data. And a third step of extracting the sound pressure waveform data from the storage means, creating a graph of a time series waveform of the sound pressure level, and displaying it on the display screen of the display means; A fourth step of designating a time position for analyzing the direction of the sound source by designating a specific point on the graph on the display screen, and a preset including the designated time position from the storage means Analysis time And calculating the phase difference between the sound pressure signals of the sounds collected by the plurality of microphones using the extracted sound pressure waveform data and estimating the sound source direction. A sixth step of extracting image data at a time between a start time and an end time of the analysis time from the storage unit; and a sound source direction estimated in the sixth step. An eighth step of synthesizing the data and the image data extracted in the seventh step to create and display a sound source position estimated image in which a graphic showing the estimated sound source direction is drawn; And a ninth step of estimating the sound source using a sound source position estimation image.

According to a third aspect of the present invention, in the sound source estimation method according to the second aspect, the time-series waveform of the sound pressure level extracted and displayed in the third step is replaced with the time-series waveform of the magnitude of the collected sound. It is characterized by that.
According to a fourth aspect of the present invention, in the sound source estimation method according to the second aspect, the step of designating a frequency band of the sound pressure signal to be estimated is extracted from the storage means in the third step. Analyzing the frequency of the time-series waveform of the sound pressure level to obtain a time-series waveform of the sound pressure level in the designated frequency band, and providing a time-series waveform of the sound pressure level of the obtained designated frequency band. The graph is displayed on the display screen of the display means.

  The invention according to claim 5 includes a microphone group having two pairs of microphones arranged at predetermined intervals on two straight lines that intersect with each other, an imaging unit that captures an image of a sound source direction, and a display unit, From the sound pressure signal of the sound propagated from the sound source collected by the microphone group and the video signal obtained by photographing the sound source direction, an image displaying the sound source position data is created and displayed on the display means to estimate the sound source An estimation device, an A / D converter for converting a sound pressure signal collected by each microphone and a video signal photographed by a photographing means into a digital signal, and the A / D-converted sound pressure signal on time Storage means for storing sound pressure waveform data arranged in series as a sound file and storing moving picture data in which the A / D converted video signals are arranged in time series as a moving picture file Extracting sound pressure waveform data having a predetermined analysis time length from the sound file to generate a sound pressure level time-series waveform graph; and a specific point of the time-series waveform graph. By designating, the analysis position specifying means for specifying the time position for analyzing the sound source direction, and the sound pressure waveform data of the preset analysis time length including the specified time position are extracted from the sound file. Then, the extracted sound pressure waveform data is subjected to frequency analysis to obtain respective phase differences between the microphones constituting the two sets of microphone pairs, and the sound source direction is determined from the ratio of the phase differences between the two obtained microphone pairs. A sound source direction estimating means for estimating the image data extracting means for extracting image data at a time between a start time and an end time of the analysis time from the moving image file; Sound source position estimation image creating means for synthesizing the data of the determined sound source direction and the extracted image data, and creating a sound source direction estimation image in which a figure indicating the estimated sound source direction is drawn; The plurality of microphones and the photographing unit are integrally incorporated, and simultaneously collect sound information and video information, and the display unit includes a sound pressure waveform graph created by the sound pressure waveform creating unit, The sound source direction estimated image created by the sound source position estimated image creating means is displayed.

  According to a sixth aspect of the present invention, in the sound source estimating apparatus according to the fifth aspect, a fifth microphone not on a plane formed by the two pairs of microphones is added to the microphone group, and the sound source direction is added. In the estimation means, the four sets of microphones constituted by the phase difference between the microphones constituting the two sets of microphone pairs, and each of the fifth microphone and the four microphones constituting the two sets of microphone pairs. A sound source direction is estimated using a phase difference between microphones constituting a pair.

According to the present invention, sound information and video information are simultaneously collected using a sound / video sampling unit in which a plurality of microphones and photographing means are integrated, and the sound pressure signal data of the collected sound is collected. The sound pressure waveform data and the video image data are respectively stored in the storage means, and only the sound pressure waveform data is called from the storage means and displayed using the time-series waveform graph of the sound pressure level. Since the time position for analysis is specified and the sound source direction is estimated using the sound pressure waveform data for the predetermined analysis time length including this specified time position, any part of the measurement data can be The sound source position can be estimated by extraction. Therefore, the estimation time of the sound source position can be greatly shortened.
Also, when estimating the sound source, the estimated sound source direction and the image data taken within the analysis time are combined to create a sound source position estimation image in which a figure showing the estimated sound source direction is drawn. Since the sound source is estimated using the sound source position estimation image, the sound source can be reliably estimated.
In addition, since the validity of the measurement can be determined at an early stage such that the position of the measurement point is not appropriate, the sound source position can be estimated efficiently. In addition, since data at an arbitrary time can be taken out, it is possible to grasp the change in the sound source position, and the history of the propagated sound can be verified on site.

In addition, if the sound source is estimated according to the steps described in claim 2, it is possible to reliably extract an arbitrary portion of the measurement data and estimate the sound source position at the site.
If the time series waveform of the sound pressure level to be displayed is the time series waveform of the magnitude of the collected sound, the sound source position can be estimated according to the magnitude of the propagation sound.
Moreover, if the time-series waveform of the sound pressure level to be displayed is a time-series waveform of the sound pressure level in a predetermined frequency band, for example, a sound from a specific device such as a sound from a power box transformer or a vibration sound of an air conditioner is used. It is possible to estimate the position of the sound source for the propagation sound.

In addition, by using the sound source estimation apparatus according to the fifth aspect, it is possible to reliably extract an arbitrary portion of the measurement data, so that the sound source position can be estimated efficiently.
At this time, the first to fourth microphones constituting the two microphone pairs arranged on the two straight lines intersecting each other at a predetermined interval and the fifth microphone not on the plane formed by the two microphone pairs And the sound source direction is estimated using the phase difference ratio between the microphones constituting the two microphone pairs and the phase difference between the first to fifth microphones. Then, the horizontal angle θ and the elevation angle φ can be estimated efficiently and accurately with a small number of microphones.

  The summary of the invention does not enumerate all necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

It is a functional block diagram which shows the structure of the sound source estimation system which concerns on this Embodiment. 3 is a flowchart illustrating a sound source estimation method according to the present invention. It is a figure which shows an example of the time-sequential waveform of a sound pressure level. It is a figure which shows an example of the display screen on which the sound source position estimation image was displayed. It is a figure which shows the arrangement | sequence of the microphone in the estimation method of the sound source using the conventional microphone pair.

  Hereinafter, the present invention will be described in detail through embodiments, but the following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

FIG. 1 is a functional block diagram showing the configuration of the sound source estimation system.
The sound source estimation system includes a sound / video sampling unit 10 and a sound source position estimation device 20.
The sound / image collection unit 10 includes a sound collection unit 11, a CCD camera (hereinafter referred to as a camera) 12 as a video collection unit, a microphone fixing unit 13, a camera support 14, a column 15, and a turntable 16. The base 17 is provided. The sound collection means 11 includes a plurality of microphones M1 to M5.
The microphones M1 to M5 are installed on the microphone fixing unit 13, the camera 12 is installed on the camera support base 14, and the microphone fixing unit 13 and the camera support base 14 are connected by three support columns 15. That is, the sound collection means 11 and the camera 12 are integrated. The microphones M1 to M5 are disposed on the upper part of the camera 12.
The base 17 is a support member composed of three legs, and the turntable 16 is installed on the base 17. The camera support 14 is mounted on the rotation member 16r of the turntable 16.
Therefore, the sound collecting means 11 and the camera 12 can be rotated together by rotating the rotating member 16r.
Microphones M1 to M5 each measure the sound pressure level of sound propagated from a sound source (not shown).

The arrangement of the microphones M1 to M5 is the same as that shown in FIG. 5, and two microphone pairs (M1) in which four microphones M1 to M4 are arranged at predetermined intervals on two straight lines orthogonal to each other. , M3) and the microphone pair (M2, M4), and the fifth microphone M5 is not located on the plane formed by the microphones M1 to M4, more specifically, the square formed by the microphones M1 to M4. It is arranged at the position of the apex of the quadrangular pyramid with the bottom face. Thereby, four pairs of microphones (M5, M1) to (M5, M4) are further configured.
In this example, the shooting direction of the camera 12 is set to a direction that passes through the intersection of the two orthogonal lines and forms approximately 45 ° with the two lines. Therefore, the direction of the sound / image collection unit 10 is the direction of the white arrow D in FIG. The camera 12 collects an image corresponding to the direction of the sound / image collection unit 10.

The sound source position estimating apparatus 20 includes an amplifier 21, an A / D converter 22, a video input / output unit 23, a storage unit 24, a display unit 25, an analysis time length setting unit 26, and an analysis frequency specifying unit 27. A sound pressure waveform generating means 28, an analysis position specifying means 29, a sound pressure waveform data extracting means 30, a sound source direction estimating means 31, an image data extracting means 32, and a data synthesizing means 33.
The amplifier 21 includes a low-pass filter, removes high-frequency noise components from the sound pressure signal of the sound collected by the microphones M1 to M5, amplifies the sound pressure signal, and outputs the amplified signal to the A / D converter 22. The A / D converter 22 creates sound pressure waveform data obtained by A / D converting the sound pressure signal, and outputs the sound pressure waveform data to the storage unit 24.
The video input / output unit 23 inputs video signals continuously shot by the camera 12 and outputs image data in the shooting direction to the storage unit 24 every predetermined time (for example, 1/30 seconds).
The storage unit 24 arranges and stores the sound pressure waveform data and the image data in time series.
As a method of storing sound pressure waveform data and image data in time series, the sound pressure waveform data and the image data are stored in synchronization, or the sound pressure waveform data and the image data are respectively stored in time. A well-known method such as storing data can be used.
Hereinafter, a file storing sound pressure waveform data is called a sound file, and a file storing image data is called a moving image file.

The display means 25 includes a display screen 25M including a sound pressure level display unit 25a that displays a graph of sound pressure waveform data, which will be described later, and an image display unit 25b that displays a sound source position estimation image for estimating the sound source position. . The sound pressure waveform data graph is created by the sound pressure waveform creation means 28, and the sound source position estimation image is created by the data synthesis means 33.
The analysis time length setting means 26 sets an analysis time length that is the size of the sound pressure waveform data used for estimating the sound source direction.
The analysis frequency designation means 27 designates the frequency band of the time series waveform of the sound pressure level created by the sound pressure waveform creation means 28.
The sound pressure waveform creation means 28 extracts sound pressure waveform data from the sound file in the storage means 24 and creates data for displaying a time series waveform graph of the sound pressure level. This sound pressure waveform creation means 28 includes frequency analysis means 28F. When the frequency band is designated by the analysis frequency designation means 27, a time series of sound pressure levels called from the sound file in the storage means 24 is obtained. The waveform is subjected to frequency analysis, and data for displaying a time-series waveform graph of sound pressure levels in a predetermined frequency band is created.
The analysis position designating unit 29 designates a time point for analyzing the sound source direction by designating a specific point on the graph displayed on the sound pressure level display unit 25a of the display screen 25M.
The sound pressure waveform data extracting means 30 stores the sound pressure waveform data for analyzing the sound source direction, that is, the sound pressure waveform data whose time width centered on the designated time position is the analysis time length. And output to the sound source direction estimating means 31.
The sound source direction estimating means 31 obtains the phase difference between the microphones M1 to M5 from the extracted sound pressure waveform data, estimates the sound source direction from the obtained phase difference, and outputs the estimation result to the data synthesizing means 33. To do. Details of the estimation of the sound source direction will be described later.

The image data extraction means 32 extracts the image data at the middle time between the start time and the end time of the analysis time from the storage means 24 and outputs this to the data synthesis means 33.
The data synthesizing unit 33 synthesizes the sound source direction data estimated by the sound source direction estimating unit 31 and the image data output from the image data extracting unit 32, and is an image in which a graphic showing the sound source direction is drawn in the image. A sound source position estimation image is created and output to the display means 25.

Next, a method for estimating the sound source direction using the sound source estimation system will be described with reference to the flowchart of FIG.
First, after connecting the sound / video sampling unit 10 and the sound source position estimating device 20, the sound / video sampling unit 10 is set at a measurement point. Then, the shooting direction of the camera 12 is directed to the planned measurement location, the sound is collected by the microphones M1 to M5, and the image of the planned measurement location is collected by the camera 12 (step S11). At this time, sound and video are sampled while the revolving table 16 is reciprocated around the planned measurement location at a slow speed of, for example, about 3 ° / sec. A rotation range of about ± 60 ° is appropriate.
Next, the output signals of the microphones M1 to M5 and the video signal of the camera 12 are A / D converted, respectively, the sound pressure waveform data is stored in the sound file of the storage means 24, and the image data is stored in the moving image file (step). S12).

Next, the sound pressure waveform data is called from the sound file to create a time-series waveform graph of the sound pressure level, and this graph is displayed on the sound pressure level display section 25a provided on the display screen 25M of the display means 25 ( Step S13).
FIG. 3 is a diagram showing an example of this. The horizontal axis of the graph is time (seconds), the vertical axis is the sound pressure level (dB), and the solid line P (t) indicates the magnitude of the collected sound, that is, the total frequency. It is a graph which shows the time change of a sound pressure level.
Although it is not always necessary to display an image on the image display unit 25b, in FIG. 3, the currently captured image is displayed as a moving image.
The measurer inputs whether the sound source direction is estimated from the beginning or the measurement timing is designated (step S14). However, if the time to collect the sound and video is short, it is difficult to specify the measurement timing for the work, so it is the same as when specifying the measurement timing after sampling the sound and video. Preferably, the method estimates the sound source.

When performing sound source direction estimated from the first, from the beginning of the sound file, extract the sound pressure waveform data analysis time length T _w that is set in advance, performs DOA estimation (Step S15).
The sound source direction is estimated by frequency-analyzing the sound pressure waveform data by FFT, obtaining the respective phase differences between the microphones M1 to M5 for each frequency, and estimating the sound source direction for each frequency from the obtained phase difference. To do. In this example, instead of the phase difference, the horizontal angle θ and the elevation angle φ are obtained using the arrival time difference which is a physical quantity proportional to the phase difference.
The calculation method of the horizontal angle θ and the elevation angle φ in step S15 will be described later.

Next, central time t _m of the analysis time, namely, the image data located half the time has passed by a length _{t m = t 0 + (T} w / 2) of the analysis time length T _w from the measurement starting time t ₀ G _m is extracted from the moving image file (step S16).
Then, the calculated sound source direction data (θ _f , φ _f ) for each frequency and the image data G _m are combined (step S 17), and the sound source position estimation image which is the combined image is displayed on the display means 25. and displays on the image display unit 25b provided on the display screen 25M, the sound pressure level display section 25a displays the taken time of the image data G _m is the data of the sound source position estimation image (step S18).
Figure 4 is a diagram showing an example, in the image display unit 25b, on the image data G _m, the sound source position estimation screen 35 is displayed graphic (circles tracery) 34 is drawn representing the sound source direction Is done. The horizontal axis of the sound source position estimation screen 35 is the horizontal angle θ _f , and the vertical axis is the elevation angle φ _f . The size of the circle represents the sound pressure level.
It is also possible to display the estimated sound source direction for each preset frequency band. In this case, the color of the mesh-shaped circle 34 may be set for each frequency band.
Further, instead of the graph of the time-series waveform of the sound pressure level, the sound pressure level display screen 36 displaying the sound pressure level (dB) when the horizontal axis is the horizontal angle θ is displayed on the sound pressure level display unit 25a. It is also possible to do.
Finally, a sound source is estimated from the sound source position estimation screen 35 (step S19). On the sound source position estimation screen 35, the image of the sound source that is estimated at the location where the graphic 34 representing the sound source direction is drawn.

When estimating the sound source direction by designating the measurement timing, the process proceeds to step S21. In step S21, a time point for analyzing the sound source direction is designated by designating a specific point on the time-series waveform graph of the sound pressure level.
For example, when the storage means 24 to the data synthesizing means 33 of the sound source position estimation apparatus 20 are configured by a computer, the mouse is displayed on the graph of the time-series waveform of the sound pressure level displayed on the screen of the display (display means 25). The time position where the sound source direction is analyzed is designated by acquiring the time position on the graph.
If the specified time position and time t _z, because this time t _z is the center time of the analysis time, from the sound file, only half the length of the analysis time length T _w than the time t _z from the previous time extracts sound pressure waveform data up to the time that has passed by half the length of the analysis time length T _w than the time t _z, it performs DOA estimation (step S22).
The estimation of the sound source direction is the same as in step S15.

After the sound source direction estimation is completed, the image data G _z taken at time t _z is the center time of the analysis time is extracted from the moving image file (step S23), the sound source direction data for each of the calculated frequency ( θ _f , φ _f ) and the image data G _z are synthesized (step S24), and the sound source position estimation screen 35, which is the synthesized image, is displayed on the image display unit 25b of the display screen 25M, and the sound pressure level is also displayed. Show acquisition time of the image data G _z in the display section 25a (step S25). The sound source position estimation screen 35 is the same as that shown in FIG.
Finally, a sound source is estimated from the sound source position estimation screen 35 (step S26).
In addition, by repeating the operations from step S21 to step S26, the sound source direction at a plurality of times can be estimated efficiently, so that the sound source can be estimated efficiently and reliably. When the variation of the sound source direction drawn on the sound source position estimation screen 35 obtained in step S26 is large, it is preferable to change the installation position of the sound / video sampling unit 10 and perform the measurement again.

すなわち、互いに直交する２直線上にそれぞれ所定の間隔で配置された２組のマイクロフォン対（Ｍ１，Ｍ３）及びマイクロフォン対（Ｍ２，Ｍ４）を構成するマイクロフォンＭ１，Ｍ３に入力する音圧信号の到達時間差Ｄ₁₃と、前記マイクロフォン対（Ｍ２，Ｍ４）を構成するマイクロフォンＭ２，Ｍ４に入力する音圧信号の到達時間差Ｄ₂₄との比から、計測点と音源位置との水平角θを推定し、前記到達時間差Ｄ₁₃，Ｄ₂₄と、前記第５のマイクロフォンＭ５と他のマイクロフォンＭ１〜Ｍ４との到達時間差Ｄ_5j（ｊ＝１〜４）とから計測点と音源位置との成す仰角φを推定する。 The calculation method of the horizontal angle θ and the elevation angle φ in step S15 is as follows.
When the arrival time difference between the microphone Mi and the microphone Mj of each microphone pair (Mi, Mj) is D _ij , the horizontal angle θ and the elevation angle φ, which are the sound incident directions, are expressed by the following equations (1) and (2). Therefore, by analyzing the frequency of the output signals of the microphones M1 to M5 using FFT and calculating the arrival time difference D _ij between the microphones M _i and M _j at the target frequency f, the horizontal angle is calculated. θ and elevation angle φ can be obtained.

That is, arrival of sound pressure signals input to the microphones M1 and M3 constituting the two microphone pairs (M1, M3) and the microphone pairs (M2, M4) arranged at predetermined intervals on two orthogonal lines. The horizontal angle θ between the measurement point and the sound source position is estimated from the ratio between the time difference D ₁₃ and the arrival time difference D ₂₄ of the sound pressure signals input to the microphones M2 and M4 constituting the microphone pair (M2, M4). The elevation angle φ between the measurement point and the sound source position is estimated from the arrival time differences D ₁₃ and D ₂₄ and the arrival time differences D _5j (j = 1 to 4) between the fifth microphone M5 and the other microphones M1 to M4. To do.

音源方向の推定結果は、所定時間毎に記憶された撮影方向の画像データ毎に行う。 The arrival time difference D _ij is obtained as a cross spectrum P _ij (f) of signals input to the two microphone pairs (M _i , M _j ), and further, the phase angle information ψ ( rad) is calculated by the following equation (3).

The estimation result of the sound source direction is performed for each image data in the shooting direction stored every predetermined time.

As described above, in this embodiment, sound and video are simultaneously collected using the sound / video sampling unit 10 in which a plurality of microphones M1 to M5 and the camera 12 are integrated, and then A / D converted to obtain a sound. The pressure waveform data is saved in the sound file of the storage means 24, the image data is saved in the moving image file, and only the sound pressure waveform data is called from the storage means 24 and the display screen 25M of the display means 25 has a sound pressure level. display graph series waveform, this after specifying the time t _z for analyzing sound source direction on the graph, with a sound pressure waveform data analysis time length T _w centered on the specified time t _z to estimate the sound source direction by calculating the phase difference between the sound pressure signals of sound collected by the plurality of microphones M1 to M5, and the image data G _z taken at this estimated sound source direction the time t _z , And a sound source position estimation image in which a figure indicating the estimated sound source direction is drawn is generated and the sound source is estimated. Therefore, the estimation time of the sound source position can be greatly shortened.
Also, if there is a problem with the validity of the measurement, for example, the position of the measurement point is not appropriate, the determination can be made early, so that the sound source position can be estimated efficiently.

  In the above embodiment, the sound source direction is estimated by specifying the measurement timing while collecting the sound and the video. However, after the sound and video are collected, the measurement timing is specified and the sound source direction is estimated. You may go. By using the method for estimating the sound source direction by specifying the measurement timing of the present invention, it is possible to create a sound source position estimation image by extracting data at an arbitrary time. The number of sheets can be greatly reduced. Therefore, the estimation time of the sound source position can be greatly shortened. In addition, it is possible to grasp the change of the sound source position, and it is possible to verify the history of the propagated sound on site.

In the above example, the graph P (t) of the time-series waveform of the sound pressure level is the magnitude of the collected sound, but it is also possible to switch to a graph of the sound pressure level for each preset frequency band.
Specifically, the frequency band of the sound pressure waveform created by the sound pressure waveform creation means 28 is designated by the analysis frequency designation means 27 and stored using the frequency analysis means 28F provided in the sound pressure waveform creation means 28. The sound pressure waveform data called from the sound file of the means 24 may be frequency-analyzed to generate data for displaying a time-series waveform graph of the sound pressure level in the designated frequency band.
As a graph of a sound pressure waveform in a frequency band having a predetermined bandwidth centered on the frequency f, for example, from a graph of a sound pressure waveform at all frequencies, such as a graph p _f (t) indicated by a broken line in FIG. Many of them also have sharp peaks.
In this way, by using the sound pressure waveform graph with a limited frequency band, it is possible to estimate the position of the sound source for the sound transmitted from a specific device, such as the sound from the transformer of the power supply box and the vibration sound of the air conditioner. It can be performed with high accuracy.
In addition, when it is not necessary to analyze low-frequency sound as noise outside the vehicle, as in the case of sound source estimation in the passenger compartment, a mid-range sound can be obtained using a sound pressure waveform graph with a limited frequency band. It is preferable to perform only sound source estimation.

In the above example, sound and video are sampled while the sound / video sampling unit 10 is rotated. If sound is generated intermittently or a sound source that changes with time is expected, It is preferable to measure with the sound / video sampling unit 10 fixed.
In the above example, the horizontal angle θ and the elevation angle φ formed by the measurement point and the sound source position are estimated using the five microphones M1 to M5. However, when the sound source position is sufficient, the horizontal angle θ is sufficient. The microphone M5 may be omitted, and only two pairs of microphones (M1, M3) and (M2, M4) arranged at predetermined intervals on two straight lines that intersect with each other may be used.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the embodiment. It is apparent from the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

As described above, according to the present invention, since it is possible to estimate the sound source position by extracting any part of the measurement data in the field, the estimation time of the sound source position can be greatly reduced, Sound source estimation can be performed efficiently.
In addition, since the validity of the measurement can be determined at an early stage such that the position of the measurement point is not appropriate, the sound source position can be estimated efficiently.

10 sound / video sampling unit, 11 sound sampling means, M1-M5 microphone,
12 CCD camera, 13 microphone fixing part, 14 camera support,
15 struts, 16 turntables, 16r rotation members, 17 bases,
20 sound source position estimation device, 21 amplifier, 22 A / D converter, 23 video input / output means, 24 storage means, 25 display means, 25M display screen, 25a sound pressure level display section,
25b image display unit, 26 analysis time length setting means, 27 analysis frequency designation means,
28 sound pressure waveform creation means, 29 analysis position designation means, 30 sound pressure waveform data extraction means,
31 sound source direction estimation means, 32 image data extraction means, 33 data synthesis means,
34 Graphic representing sound source direction, 35 Sound source position estimation screen.

Claims (6)

  Sound pressure waveform data, which is the sound pressure signal data of the collected sound, is collected at the same time using a sound / video sampling unit in which a plurality of microphones and photographing means are integrated. And the image data of the video are respectively stored in the storage means, and only the sound pressure waveform data is extracted from the storage means to display a time-series waveform graph of the sound pressure level, and the sound source direction is analyzed on the graph. After designating the time position to be performed, using the sound pressure waveform data of a predetermined analysis time length including the designated time position, the phase difference between the sound pressure signals of the sounds collected by the plurality of microphones is calculated. Estimating the sound source direction, combining the estimated sound source direction and the image data captured within the analysis time, creating a sound source position estimation image in which a figure showing the estimated sound source direction is drawn, Estimation method of a sound source, which comprises of estimating the sound source using the sound source position estimation image. A first step of simultaneously collecting sound information and video information using a sound / video sampling unit in which a plurality of microphones and photographing means are integrated;
A second step of A / D converting the sound pressure signal of the collected sound and storing it as sound pressure waveform data in a storage means, and A / D converting the video signal and storing it as image data in the storage means; ,
A third step of extracting the sound pressure waveform data from the storage means, creating a graph of a time series waveform of the sound pressure level, and displaying the graph on the display screen of the display means;
A fourth step of designating a time position for analyzing the sound source direction by designating a specific point on the graph on the display screen;
A fifth step of extracting, from the storage means, sound pressure waveform data having a preset analysis time length including the designated time position;
A sixth step of calculating a phase difference between sound pressure signals of sounds collected by the plurality of microphones using the extracted sound pressure waveform data, and estimating a sound source direction;
A seventh step of extracting image data at a time between a start time and an end time of the analysis time from the storage means;
A sound source position estimation image in which a graphic representing the estimated sound source direction is drawn by combining the sound source direction data estimated in the sixth step and the image data extracted in the seventh step. An eighth step of creating and displaying;
The sound source estimation method according to claim 1, further comprising: a ninth step of estimating the sound source using the sound source position estimation image.   The sound source estimation method according to claim 2, wherein in the third step, the time-series waveform of the sound pressure level extracted from the storage means is a time-series waveform of the magnitude of the collected sound.   In the third step, the step of designating the frequency band of the sound pressure signal to be estimated, and the time series waveform of the sound pressure level extracted from the storage means are frequency-analyzed, and the sound pressure level in the designated frequency band A time series waveform is obtained, and a graph of a time series waveform of the obtained sound pressure level in the designated frequency band is displayed on a display screen of the display means. Estimation method. Providing a microphone group having two pairs of microphones arranged at predetermined intervals on two intersecting straight lines, a photographing means for photographing a sound source direction image and a display means, and propagating from the sound source collected by the microphone group A sound source estimation device that creates an image on which sound source position data is displayed from a sound pressure signal of a sound and a video signal obtained by photographing a sound source direction, displays the image on a display unit, and estimates a sound source,
An A / D converter that converts the sound pressure signal collected by each microphone and the video signal photographed by the photographing means into digital signals,
The sound pressure waveform data in which the A / D converted sound pressure signals are arranged in time series is stored as a sound file, and the moving picture data in which the A / D converted video signals are arranged in time series is stored in a moving picture file. Storage means for storing as,
Sound pressure waveform creating means for extracting sound pressure waveform data of a preset analysis time length from the sound file and creating a graph of a time-series waveform of sound pressure levels;
By specifying a specific point on the graph of the time series waveform, an analysis position specifying means for specifying a time position for analyzing the sound source direction;
Sound pressure waveform data having a predetermined analysis time length including a specified time position is extracted from the sound file, and the extracted sound pressure waveform data is subjected to frequency analysis to form the two sets of microphone pairs. Sound source direction estimating means for obtaining respective phase differences between the microphones and estimating the sound source direction from the ratio of the phase differences of the two obtained microphone pairs;
Image data extraction means for extracting image data at a time between the start time and the end time of the analysis time from the moving image file;
Sound source position estimated image creating means for creating a sound source direction estimated image in which a figure showing the estimated sound source direction is drawn by combining the estimated sound source direction data and the extracted image data; ,
The plurality of microphones and the photographing unit are integrally incorporated, and simultaneously collect sound information and video information,
The display means displays a graph of the sound pressure waveform created by the sound pressure waveform creation means and a sound source direction estimation image created by the sound source position estimation image creation means, .   A fifth microphone that is not on the plane formed by the two sets of microphone pairs is added to the microphone group, and the sound source direction estimation means includes a phase difference between the microphones constituting the two sets of microphone pairs, and A sound source direction is estimated using a fifth microphone and a phase difference between the microphones constituting the four pairs of microphones constituted by each of the four microphones constituting the two pairs of microphones. The sound source estimation apparatus according to claim 5.

Images