JP2010236939A - Method and apparatus for estimating sound source - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and an apparatus for the same which enable easy estimation of a source of generation of an observed sound, without using an image processing device. <P>SOLUTION: Using a sound pick-up/display device 10 constituted by uniting a plurality of microphones M1-M5 and a transparent display 12, the direction of the sound source and the loudness of the sound which are estimated from the information on the sound picked up by the microphones M1-M5 are drawn with round marks of a network pattern on the transparent display 12, and a measuring person looks in the transparent display 12 from the back 12n side. Thereby an object seen in overlapping with the round marks of the network pattern is estimated to be the sound source, while the three-dimensional coordinates of the object are calculated by providing a distance sensor 13 in the sound pick-up/display device 10. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a method and apparatus for estimating a sound source from sound information collected by a plurality of microphones.

Conventionally, as a method of estimating the direction of sound arrival, a microphone array in which a large number of microphones are arranged at equal intervals is constructed, and the sound source direction that is the direction of sound wave arrival is estimated from the phase difference of each microphone relative to the reference microphone. So-called acoustic techniques have been devised (for example, see Non-Patent Document 1).
On the other hand, not a phase difference between output signals of a plurality of microphones arranged at a measurement point, but a plurality of microphone pairs arranged in a straight line intersecting each other from the plurality of microphones, and the position between the two microphones constituting the pair There has been proposed a method of estimating a sound source direction from a ratio between an arrival time difference corresponding to a phase difference and an arrival time difference between two paired two microphones (see, for example, Patent Documents 1 to 3).

  Specifically, as shown in FIG. 6, two microphone pairs (M1, M3) and microphone pairs (four microphones M1 to M4) are arranged at predetermined intervals on two straight lines orthogonal to each other. M2, M4) are arranged so as to constitute the microphone pair (M1, M3) and the microphones constituting the microphone pair (M2, M4) and the arrival time difference of the sound pressure signals input to the microphones M1, M3 The horizontal angle θ between the measurement point and the position of the sound source is estimated from the ratio with the arrival time difference between the sound pressure signals input to M2 and M4, and the fifth microphone M5 is not on the plane formed by the microphones M1 to M4. Further, four microphone pairs (M5, M1), (M5, M2), (M5, M3), (M5, M4) are configured and arranged in the positions. The elevation angle φ formed by the measurement point and the position of the sound source is estimated from the arrival time difference between the microphones constituting the phone pair.

Thereby, compared with the case where the sound source direction is estimated using the microphone array, the sound source direction can be accurately estimated with a small number of microphones.
Also, at this time, after taking a video of the estimated sound source direction by providing a video sampling means such as a CCD camera, the data of this video and the data of the sound source direction are combined to obtain the estimated sound source direction in the video. If the sound pressure level is displayed as a graphic, the sound source can be grasped visually.

Japanese Patent Laid-Open No. 2002-181913 JP 2006-324895 A JP 2008-224259 A

Toshiro Oga, Yoshio Yamazaki, Yutaka Kaneda; Acoustic system and digital processing, Corona, 1995

However, in the conventional method, image data obtained by combining captured video data and sound source direction data is created, and this image data is displayed on the display to estimate the sound source. In addition to taking a long time, there is a problem in that the apparatus is complicated and upsized because an imaging means such as a camera and an image processing apparatus are necessary.
In addition, when the measurement point is not appropriate, it is necessary to change the measurement point again and perform measurement again, so that the work efficiency is poor.

  The present invention has been made in view of the conventional problems, and an object of the present invention is to provide a method and apparatus that can easily estimate the source of observed sound without using an image processing apparatus.

As a result of intensive studies, the inventors of the present invention have displayed the sound source direction estimated from the sound information collected by each microphone as a graphic on the transparent display. Focusing on the high possibility of being a sound source, it was found that a sound source can be quickly estimated with a simple configuration by using a transparent display instead of a camera and a display. It has been reached.
That is, the present invention is a method for estimating a sound source from sound information from a sound source collected by a plurality of microphones, and a horizontal angle and an elevation angle formed by a measurement point and the sound source from a phase difference of sound pressure signals between the microphones. And the estimated horizontal angle and elevation angle are displayed as a graphic indicating the direction of the sound source on a transparent display to estimate the sound source. If the sound source is estimated in this way, the sound source can be easily estimated without using an image processing apparatus.

In the present invention, the size of the graphic indicating the direction of the sound source is set to a size corresponding to the size of the sound pressure signal collected by each microphone, and the sound source can be estimated from the sound size. Accuracy is improved.
In the present invention, the distance sensor measures the distance between the object that appears to overlap the figure on the transparent display and the transparent display, and estimates the sound source position using the measured distance, the horizontal angle, and the elevation angle. Thus, since the coordinates of the sound source position can be calculated with high accuracy, the sound source can be reliably identified.

The sound source estimation apparatus of the present invention includes a sound collection unit having a plurality of microphones for measuring the sound pressure level of sound propagating from the sound source, and a display unit comprising a transparent display provided integrally with the sound collection unit. Sound collection / display means, sound source direction estimation means for estimating the horizontal angle θ and elevation angle φ between the measurement point and the sound source from the phase difference of the sound pressure signal between each microphone, and the estimated horizontal angle and elevation angle Is converted to coordinates on the transparent display, and the converted coordinates are displayed on the transparent display as a graphic showing the direction of the sound source. A sound source estimation apparatus that can be estimated can be obtained.
In addition, the sound source estimation device includes a distance sensor that measures the distance between the object that appears to overlap the figure on the transparent display and the sound collection / display unit, the distance r measured by the distance sensor, the horizontal angle θ, and the elevation angle φ. The sound source position coordinates can be calculated with high accuracy by providing sound source position estimating means for calculating the three-dimensional coordinates of the object and estimating the sound source position.

  In addition, the sound collection unit of the sound collection / display means includes two microphone pairs in which two microphones are arranged on two straight lines intersecting each other and separated by a predetermined distance, and the two sets The fifth microphone that is not on the plane formed by the pair of microphones, and the ratio of the phase difference between the microphones constituting the two pairs of microphones for each of a plurality of preset frequencies by the sound source direction estimating means or Respective arrival time difference ratios are obtained, the horizontal angle θ is estimated from the phase difference ratio or arrival time difference ratio, and the phase difference or arrival time difference between the microphones constituting the two microphone pairs, and the fifth microphone And four sets of microphones composed of each of the four microphones constituting the two pairs of microphones Since the elevation angle φ is estimated using the phase difference or arrival time difference between the microphones constituting the phone pair, the sound source direction (θ, φ) can be estimated efficiently and accurately with a small number of microphones. Can do.

  The summary of the invention does not enumerate all necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

It is a functional block diagram which shows the structure of the sound source position estimation apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the method to convert a horizontal angle and an elevation angle into the point on a transparent display. It is a figure which shows a display screen when a transparent display is looked at from the measurer side. It is a figure which shows the calculation method of the distance of a sound source and a sound collection and display apparatus. It is a figure which shows the other structure of a sound collection and display apparatus. It is a figure which shows the arrangement | sequence of the microphone in the conventional sound source estimation method.

  Hereinafter, the present invention will be described in detail through embodiments. However, the following embodiments do not limit the invention according to the claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

FIG. 1 is a functional block diagram showing the configuration of the sound source position estimating apparatus 1.
The sound source position estimation device 1 includes a sound collection / display device 10 and a calculation device 20.
The sound collection / display device 10 is obtained by integrating a sound collection unit 11 having a plurality of microphones M1 to M5, a transparent display 12, and a distance sensor 13.
The five microphones M <b> 1 to M <b> 5 constituting the sound collection unit 11 are respectively fixed to the microphone fixing unit 14. The transparent display 12 is fixed to the frame 15, and the distance sensor 13 is attached to the sensor mounting base 16.
The computing device 20 is connected to the sound collection / display device 10 by a cable (not shown).
The transparent display 12 is a transparent display in a non-display state. For example, an EL display in which an electroluminescent element (EL element) made of a thin film having high transmittance and a transparent electrode are sandwiched between glass substrates, a liquid crystal display element, and a polarizing plate A known transparent display such as a monochromatic display combining the above can be used.
As the distance sensor 13, a known sensor such as an optical sensor such as a laser displacement meter or an ultrasonic sensor can be used.

  As shown in FIG. 1, the center of the transparent display 12 is the origin, the x-axis direction is the left-right direction, the y-axis direction is the up-down direction, and the z-axis direction, which is the direction perpendicular to the display surface of the transparent display 12, is The direction. In this example, the microphone fixing portion 14 is attached to the center of the upper surface 15 a of the frame body 15, and the sensor mounting base 16 is attached to the upper side of the right side surface 15 b of the frame body 15. Sound information propagating from a sound source can be collected while holding the surfaces 15b and 15c.

  The arrangement of the microphones M1 to M5 is the same as that shown in FIG. 5, and two microphone pairs (M1, M1) are arranged with four microphones M1 to M4 arranged at predetermined intervals on two straight lines orthogonal to each other. M5) and the microphone pair (M2, M4) are arranged so that the fifth microphone M5 is not on the plane formed by the microphones M1 to M4. Specifically, a square formed by the microphones M1 to M4 is formed. It arranges at the position of the apex of the quadrangular pyramid as the bottom. Thereby, four pairs of microphones (M5, M1) to (M5, M4) are further configured.

In this example, the direction of the sound collection / display device 10 is set to a direction that passes through the intersection of the two orthogonal lines and forms approximately 45 ° with the two lines. That is, the z-axis direction in FIG. Hereinafter, the + z direction (front side) indicated by the white arrow in FIG.
Further, the surface 12m on the + z side of the transparent display 12 is the front surface, and the surface 12n on the -z side is the back surface. The measurer estimates the sound source from the back surface 12n side of the transparent display 12 while looking through the sound collection direction.

The arithmetic unit 20 includes an amplifier 21, an A / D converter 22, a storage unit 23, a sound source direction estimation unit 24, a coordinate conversion unit 25, a display graphic creation unit 26, and a distance calculation unit 27.
The amplifier 21 includes a low-pass filter, removes high-frequency noise components from the sound pressure signals of the sounds collected by the microphones M1 to M5, amplifies the sound pressure signals, and outputs them to the A / D converter 22. The A / D converter 22 creates sound pressure waveform data obtained by A / D converting each sound pressure signal, and outputs the sound pressure waveform data to the storage means 23.
Storage means 23, when the origin of the center position of the transparent display 12, the coordinate data of the center position O _m of the microphone _{M1~M5 (x m, y m,} z m), coordinates of the position O _r the distance sensor 13 Data (x _r , y _r , z _r ), virtual focal position data described later, and the sound pressure waveform data are stored. As shown in FIGS. 2A and 2B, the virtual focus position is the eye of the measurer when looking at the transparent display 12 from the back surface 12n side when the center position of the transparent display 12 is O. The coordinates of the position O ′. If the distance between the center position of the transparent display 12 and the position of the eye of the measurer is D, O ′ is (x _m , y _m , z _m −D).

The sound source direction estimating means 24 takes out the sound pressure waveform data from the storage means 23, obtains the phase difference between the microphones M1 to M5 from the extracted sound pressure waveform data, and determines the sound source direction from the obtained phase difference. The horizontal angle θ and the elevation angle φ are estimated and output to the coordinate conversion means 25.
The horizontal angle θ and the elevation angle φ are estimated for each predetermined frequency band. Details of the estimation of the sound source direction will be described later.
In the coordinate conversion means 25, as shown in FIGS. 2A and 2B, the coordinate data of the center position O _{m and} the coordinate data of the virtual focal position O ′ of the microphones M1 to M5 stored in the storage means 23 are used. The horizontal angle θ and the elevation angle φ between the measurement point estimated by the sound source direction estimating means 24 and the sound source are converted into coordinate data (θ ′, φ ′) on the transparent display 12. The x-axis of the transparent display 12 represents θ ′, and the y-axis represents φ ′. In a general display, the upper left corner of FIG. 1 is the origin of coordinates, but in this example, the center position of the transparent display 12 is the origin of coordinates O for the sake of simplicity.

The display graphic creating means 26 creates a graphic indicating the sound source direction on the coordinate data (θ ′, φ ′) obtained by converting the sound source direction data. FIG. 3 is a diagram showing an example thereof. In this example, the graphic indicating the direction of the sound source is a round mark R of a mesh pattern. The center of the circle is a point representing the coordinate data (θ ′, φ ′), and the radius is the magnitude of the sound pressure signal collected by each of the microphones M1 to M5. When displaying the sound source direction of a preset frequency band, or when displaying the sound source direction for each frequency band, the color of the circle R may be set according to the frequency band.
Thereby, as shown in FIG. 3, the measurer can estimate the sound source by looking into the transparent display 12 from the back surface 12n side. That is, the measurer estimates that the object 50 that appears to overlap the mesh-shaped circle R when the transparent display 12 is viewed from the rear surface 12n side is the sound source.
The distance sensor 13 is a distance between the object 50 and the sound sampling / display device 10 (actually, the object that appears to overlap the mesh-shaped circle R when the measurer looks into the transparent display 12 from the back surface 12n side. 50) and the distance (r) between the position where the distance sensor 13 is disposed, and r).
The distance calculating unit 27 calculates the three-dimensional coordinates of the object 50 using the horizontal angle θ and the elevation angle φ estimated by the sound source direction estimating unit 24 and the distance r measured by the distance sensor 13.

すなわち、互いに直交する２直線上にそれぞれ所定の間隔で配置された２組のマイクロフォン対（Ｍ１，Ｍ３）及びマイクロフォン対（Ｍ２，Ｍ４）を構成するマイクロフォンＭ１，Ｍ３に入力する音圧信号の到達時間差Ｄ₁₃と、前記マイクロフォン対（Ｍ２，Ｍ４）を構成するマイクロフォンＭ２，Ｍ４に入力する音圧信号の到達時間差Ｄ₂₄との比から、計測点と音源位置との水平角θを推定し、前記到達時間差Ｄ₁₃，Ｄ₂₄と、前記第５のマイクロフォンＭ５と他のマイクロフォンＭ１〜Ｍ４との到達時間差Ｄ_5j（ｊ＝１〜４）とから計測点と音源位置との成す仰角φを推定する。 The calculation method of the horizontal angle θ and the elevation angle φ is as follows.
When the arrival time difference between the microphone Mi and the microphone Mj of each microphone pair (Mi, Mj) is D _ij , the horizontal angle θ and the elevation angle φ, which are the sound incident directions, are expressed by the following equations (1) and (2). Therefore, by analyzing the frequency of the output signals of the microphones M1 to M5 using an FFT and calculating the arrival time difference D _ij between the microphones M _i and M _j at the target frequency f, the horizontal angle is calculated. θ and elevation angle φ can be obtained.

That is, arrival of sound pressure signals input to the microphones M1 and M3 constituting the two microphone pairs (M1, M3) and the microphone pairs (M2, M4) arranged at predetermined intervals on two orthogonal lines. The horizontal angle θ between the measurement point and the sound source position is estimated from the ratio between the time difference D ₁₃ and the arrival time difference D ₂₄ of the sound pressure signals input to the microphones M2 and M4 constituting the microphone pair (M2, M4). The elevation angle φ between the measurement point and the sound source position is estimated from the arrival time differences D ₁₃ and D ₂₄ and the arrival time differences D _5j (j = 1 to 4) between the fifth microphone M5 and the other microphones M1 to M4. To do.

水平角θ及び仰角φとは、ｆを中心周波数とする所定の帯域幅を有する周波数帯域毎に求めることができる。 The arrival time difference D _ij is obtained as a cross spectrum P _ij (f) of signals input to the two microphone pairs (M _i , M _j ), and further, the phase angle information ψ ( rad) and is calculated using the following equation (3).

The horizontal angle θ and the elevation angle φ can be obtained for each frequency band having a predetermined bandwidth with f as the center frequency.

Next, a sound source estimation method according to the present invention will be described.
First, after connecting the sound collection / display device 10 and the computing device 20 with a cable (not shown), the sound collection / display device 10 is directed in a direction that seems to be a sound generation source, and sound information is output from the microphones M1 to M5. Collect.
The sound information is input to the arithmetic unit 20, amplified by the amplifier 21, A / D converted by the A / D converter 22, and stored in the storage unit 23 as sound pressure waveform data.
The sound source direction estimating means 24 extracts the sound pressure waveform data from the storage means 23, obtains the phase difference between the microphones M1 to M5 from the extracted sound pressure waveform data, and determines the sound source direction from the obtained phase difference. The horizontal angle θ and the elevation angle φ are estimated.
The estimated horizontal angle θ and elevation angle φ are converted into coordinate data (θ ′, φ ′) on the transparent display 12 by the coordinate conversion means 25, and the center of the transparent display 12 is coordinate data (θ ′ , Φ ′), and a round dot R having a mesh pattern in which the radius is the magnitude of the sound pressure signal collected by each microphone is drawn.
The measurer looks into the transparent display 12 from the back surface 12n side, and estimates that the object (here, the speaker device) 50 that appears to overlap the mesh-shaped circle R as shown in FIG. 3 is the sound source.

When obtaining the position coordinates of the object 50 estimated as the sound source, the sound collection / display device 10 is rotated in the direction of the object 50 as shown in FIGS. The distance r ′ between the distance sensor 13 and the object 50 is measured.
Next, the distance r ′ is stored in the storage unit 23 in advance, and the coordinate data O _m of the center positions of the microphones M1 to M5 and the distance sensor 13 when the center position of the transparent display 12 is used as the origin. using the coordinate data O _r, it is converted to a distance r between the center position and the object 50 of the microphone M1 to M5.
Thereafter, the three-dimensional coordinates of the object 50 are calculated using the distance r and the horizontal angle θ and the elevation angle φ estimated by the sound source direction estimating means 24. In this case, the origin of the three-dimensional coordinates is the coordinates of the central point of the sound collection / display device 10 which is a measurement point.
As a result, not only the object 50 but also the three-dimensional coordinates of the object 50 can be obtained.

Thus, in this embodiment, the sound source direction estimated from the sound information collected by the microphones M1 to M5 using the sound collection / display device 10 in which the plurality of microphones M1 to M5 and the transparent display 12 are integrated. And the loudness of the sound are drawn on the transparent display 12 by a circle mark R of the mesh pattern, and the measurer looks at the transparent display 12 from the back surface 12n side, so that it appears to overlap the circle mark R of the mesh pattern. Since the object 50 is estimated to be a sound source, it is possible not only to easily estimate the observed sound generation source but also to perform image processing without using a photographing means and an image processing device. Therefore, the sound source can be estimated quickly.
In addition, after the distance sensor 13 is provided in the sound collection / display device 10 and the distance r ′ between the distance sensor 13 and the object 50 is measured, the distance r ′ is determined from the center position of the microphones M1 to M5 and the object 50. The three-dimensional coordinates of the target object 50 are calculated using the distance r and the horizontal angle θ and the elevation angle φ estimated by the sound source direction estimating unit 24. The three-dimensional coordinates can also be obtained.

In the embodiment, the microphones M1 to M5 are installed on the upper surface 15a of the frame 15 of the transparent display 12. However, the present invention is not limited to this, and may be installed on the right side surface 15b or the left side surface 15c. In addition, the distance sensor 13 is not necessarily integrated with the sound collection unit 11 and the transparent display 12. After the target object 50 as a sound source is estimated, the distance sensor 13 is used to measure the measurement point and the target object 50. The distance may be measured.
In the above example, the coordinate (θ ′, φ ′) of the sound source on the transparent display 12 is obtained by coordinate conversion of the estimated horizontal angle θ and elevation angle φ, but when the object 50 as the sound source is far away. The estimated horizontal angle θ and elevation angle φ may be used as the coordinates of the sound source as they are.
Further, the arithmetic unit 20 may be constituted by a microcomputer and integrated with the sound collection unit 11 and the transparent display 12.

Further, the distance sensor 13 is not an essential requirement of the present invention. That is, what is necessary is just to estimate what the sound source is, and when the coordinates of the sound source position are not necessary, the distance sensor 13 and the distance calculating means 27 may be omitted from the sound source position estimating device 1. As a result, both the sound collection / display device 10 and the arithmetic device 20 can be simplified, so that a sound source estimation device capable of estimating a sound source can be obtained with a simpler configuration.
As a sound collection / display device used in the sound source estimation device as described above, a sound collection / display device 10R including a disk-shaped transparent display 12R as shown in FIG. 5 is suitable.
At this time, if the microphones M1 to M5 are arranged on the upper portion of the annular frame 15R that holds the disc-shaped transparent display 12R, and the grip rod 17 is provided on the lower portion, it is convenient to carry.
Instead of the sound collection / display device 10 of the sound source position estimation apparatus 1, a sound collection / display device 10R with a distance sensor 13 attached may be used.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the embodiment. It is apparent from the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

As described above, according to the present invention, the source of observed sound can be easily estimated without using an image processing apparatus, so that sound source estimation work can be performed efficiently.
In addition, since a photographing device and an image processing device are unnecessary, a small and inexpensive sound source estimation device can be provided.

1 sound source position estimation device, 10 sound sampling / display device, 11 sound sampling unit,
12 transparent display, 12m surface of transparent display,
12n back of transparent display, 13 distance sensor, 14 microphone fixing part,
15 Frame, 15a Upper surface of the frame, 15b, 15c Side surface of the frame,
16 Sensor mounting base, M1-M5 microphone,
20 arithmetic units, 21 amplifiers, 22 A / D converters, 23 storage means,
24 sound source direction estimation means, 25 coordinate conversion means, 26 display figure creation means,
27 Distance calculation means, 50 object.

Claims (6)

  Sound information propagated from the sound source is collected by a plurality of microphones, and the horizontal angle and elevation angle between the measurement point and the sound source are estimated from the phase difference of the sound pressure signal between the microphones. A sound source estimation method, wherein a sound source is estimated by displaying an angle and an elevation angle as a graphic indicating a sound source direction on a transparent display.   The sound source estimation method according to claim 1, wherein a size of the figure is set in accordance with a size of a sound pressure signal collected by each microphone.   A distance sensor measures a distance between the transparent display and an object that appears to overlap a figure on the transparent display, and estimates a sound source position using the measured distance, the horizontal angle, and an elevation angle. The sound source estimation method according to claim 1 or 2. A sound collection / display means comprising a sound collection unit having a plurality of microphones for measuring the sound pressure level of sound propagating from a sound source, and a display unit comprising a transparent display provided integrally with the sound collection unit;
Sound source direction estimating means for estimating a horizontal angle and an elevation angle between a measurement point and a sound source from a phase difference of a sound pressure signal between the microphones;
Coordinate conversion means for converting the estimated horizontal angle and elevation angle into coordinates on the transparent display;
The sound source estimation apparatus, wherein the converted coordinates are displayed as a graphic indicating a sound source direction on the transparent display. A distance sensor that measures the distance between the object that appears to overlap the figure on the transparent display and the sound collection and display means;
5. A sound source position estimating unit that calculates a three-dimensional coordinate of the object using a distance measured by the distance sensor, the horizontal angle, and an elevation angle to estimate a sound source position. The sound source estimation apparatus according to 1.   The sound sampling unit is arranged on two straight lines intersecting each other, two microphone pairs in which two microphones are spaced apart by a predetermined distance, and a plane formed by the two microphone pairs. And the sound source direction estimating means obtains a sound phase difference or arrival time difference in each of the microphone pairs for each of a plurality of preset frequencies, and calculates the phase difference ratio or The horizontal angle is estimated from the ratio of the arrival time differences, and the phase difference or arrival time difference between the microphones constituting the two sets of microphone pairs, and the four microphones constituting the fifth microphone and the two sets of microphone pairs Between microphones that constitute four pairs of microphones, each consisting of a microphone Sound source estimating apparatus according to claim 4 or claim 5, characterized in that estimating the elevation by using the phase difference or time difference of arrival.

Images