JPWO2008143142A1 - Sound source localization apparatus, sound source localization method, and program - Google Patents

Abstract

The single channel direction detection unit (13) creates direction data for each channel from a set of sound source data (E12) corresponding to the channel while switching the channel, and sends it to the FFT unit (14) as direction data (E13) for each channel. Output. The FFT unit (14) obtains a direction data frequency component (E14) by Fourier transforming the channel-specific direction data (E13) and considering it as time-series direction data. A filter part (15) outputs what suppressed the frequency component of the even-order harmonic among the direction data frequency component (E14) as a filtered direction data frequency component (E15). The IFFT unit (16) obtains filtered direction data (E16) by performing an inverse Fourier transform on the filtered direction data frequency component (E15). The sound source localization unit (17) obtains the direction of the sound source from the filtered direction data (E16).

Description

  The present invention relates to a sound source localization device, a sound source localization method, and a program.

  Beam forming technology is known as a method for realizing highly accurate sound source localization. The sound source localization refers to specifying a sound source direction from a phase difference or intensity difference of sound source data input to each using a plurality of microphones. The beam forming technique refers to a technique that integrates multi-channel sound source data information to realize highly accurate sound source localization. As the beam forming technique, an adaptive beam forming technique and a delay sum type beam forming technique are known.

  The adaptive beamforming technique is a technique for measuring sound source data with a plurality of microphones and attenuating sound source data other than the target direction to estimate the direction of the sound source. The adaptive beam forming technique is described in Non-Patent Document 1, for example.

The delay-sum beamforming technique is a technique for measuring sound source data with a plurality of microphones and enhancing the sound source data in a target direction to estimate the direction of the sound source. The delay sum type beam forming technique is described in Patent Document 1, for example.
The Institute of Electronics, Information and Communication Engineers: Acoustic systems and digital processing JP 2001-309383 A

  When the above-described adaptive beamforming technique is used, sound source data other than the target direction can be attenuated by the number of (microphones−1). For this reason, when there is a lot of sound source data other than the target direction due to the reflection of sound waves and noise, the effects of reflection and noise of these sound waves cannot be suppressed.

  Even when the delay-and-sum beamforming technique is used, the influence of sound wave reflection and noise cannot be completely suppressed.

  It is an object of the present invention to provide a sound source localization device, a sound source localization method, and a program that can perform the sound source localization with high accuracy while suppressing the influence of reflection and noise.

In order to achieve the above object, a sound source localization apparatus according to the first aspect of the present invention includes:
A microphone array composed of at least three or more microphone elements;
Voice data for each microphone element is input from the microphone array, and for each channel corresponding to a combination of two microphones having a predetermined positional relationship, the direction of the sound source is shown with reference to the positions of the two microphones corresponding to the channel. Direction detection means for obtaining direction data;
The direction data for each channel is input from the direction detection means, the input direction data for each channel is concatenated, and the frequency component of the waveform represented by the concatenated direction data is obtained by Fourier transforming the data regarded as time series data. Fourier transform means;
Filter means for inputting the frequency component obtained by the Fourier transform means, and outputting as a filtered frequency component a frequency component of the even-order harmonics suppressed among the inputted frequency components;
An inverse Fourier transform unit that inputs a filtered frequency component from the filter unit, and obtains filtered data by performing an inverse Fourier transform on the filtered frequency component;
Sound source localization means for obtaining the direction of the sound source from the filtered data obtained by the inverse Fourier transform means.

  The channel may correspond to a combination of two adjacent microphones.

The filter means includes
The frequency component obtained by the Fourier transform means is input, the frequency component of the even-order harmonic is suppressed among the input frequency components, and the frequency component of the odd-order harmonic is further attenuated and output as the filtered frequency component. May be.

The direction detecting means includes
A single channel direction detection means for outputting a channel selection signal for selecting a channel, inputting voice data of a channel corresponding to the channel selection signal, and obtaining direction data for each channel;
Channel switching means for inputting voice data for each microphone element from the microphone array and outputting voice data corresponding to the channel indicated by the selection signal input from the single channel direction detecting means may be further provided.

  The microphone array may be composed of omnidirectional microphone elements.

In order to achieve the above object, a sound source localization method according to the second aspect of the present invention includes:
Audio data for each microphone element is input from a microphone array composed of at least three or more microphone elements, and for each channel corresponding to a combination of two microphones in a predetermined positional relationship, two microphones corresponding to the channel A direction detecting step for obtaining direction data indicating the direction of the sound source relative to the position;
A Fourier transform step for obtaining frequency components of a waveform represented by the direction data represented by concatenating the direction data for each channel obtained by the direction detection step and performing Fourier transform on what is regarded as time-series direction data;
A filter step of outputting a frequency component obtained by suppressing the frequency component of the even-order harmonics among the frequency components obtained by the Fourier transform step as a filtered frequency component;
An inverse Fourier transform step for obtaining a filtered data by performing an inverse Fourier transform on the filtered frequency component output by the filter step;
A sound source localization step for obtaining the direction of the sound source from the filtered data obtained by the inverse Fourier transform step.

In order to achieve the above object, a program according to the third aspect of the present invention provides:
Computer
Two channels corresponding to each channel corresponding to a combination of two microphones in a predetermined positional relationship based on audio data for each microphone element input from a microphone array composed of at least three or more microphone elements. Direction detection means for obtaining direction data indicating the direction of the sound source with respect to the position of the microphone;
Fourier transform means for concatenating the direction data for each channel obtained by the direction detection means and obtaining a frequency component of the waveform represented by the direction data represented by Fourier transform of what is regarded as time series data,
Filter means for obtaining a filtered frequency component obtained by suppressing the frequency component of the even-order harmonics among the frequency components obtained by the Fourier transform means;
Inverse Fourier transform means for obtaining filtered data by performing inverse Fourier transform on the filtered frequency component obtained by the filter means;
It is made to function as a sound source localization means which calculates | requires the direction of a sound source from the filtered data which the said inverse Fourier transform means calculated | required.

  According to the sound source localization apparatus, the sound source localization method, and the program according to the present invention, by performing Fourier transform on multi-channel sound source data, the influence of sound wave reflection and noise is suppressed, and highly accurate sound source localization is performed. Can do.

FIG. 1 is a functional block diagram showing an overall configuration of a sound source localization apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram showing a physical configuration of a main part in the sound source localization apparatus shown in FIG. FIG. 3 is a diagram schematically showing the configuration of the microphone array. FIG. 4 is a diagram for explaining a reference for an angle when expressing single channel direction data. FIG. 5 is a flowchart showing an example of a sound source localization process of the sound source localization apparatus. FIG. 6 is a flowchart showing the channel-specific direction detection processing in FIG. FIG. 7 is a flowchart showing the localization process in FIG. FIG. 8 is a diagram defining the certainty of IID (Interaural Intensity Difference). FIG. 9 is a diagram showing the measurement angle and ideal angle of direction data in each channel. FIG. 10 shows a waveform formed by time-series direction data. FIG. 11A is a diagram illustrating an example of frequency components of the direction data before performing the filtering process. FIG. 11B is a diagram illustrating an example of frequency components of the direction data after performing the filtering process. FIG. 11C is a diagram illustrating an example of frequency components of direction data after performing filter processing for attenuating the intensity of odd-order harmonics. FIG. 12 is a diagram illustrating a waveform formed by the filtered direction data. FIG. 13 is a diagram showing a modification of the arrangement of the microphones. FIG. 14 is a diagram showing a relationship between an arbitrary reference direction and a channel in the modification shown in FIG. FIG. 15 is a diagram showing a waveform of data for performing FFT (Fast Fourier Transform) processing in the modification shown in FIG. FIG. 16 is a diagram illustrating a relationship between an arbitrary reference direction and a channel in another modification of the microphone arrangement.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Sound source localization apparatus 11 Microphone array 12 Channel switching part 13 Single channel direction detection part 14 FFT part 15 Filter part 16 IFFT (Inverse Fast Fourier Transform) part 17 Sound source localization part 18 Control part 19 Temperature measuring part 20 Sound source 50 Direction detection part

  Hereinafter, a sound source localization apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the sound source localization apparatus 10 of the present embodiment includes a microphone array 11, a direction detection unit 50, an FFT (Fast Fourier Transform) unit 14, a filter unit 15, and an IFFT (Inverse Fast Fourier Transform). A unit 16, a sound source localization unit 17, a control unit 18, and a temperature measuring unit 19 are provided.

  The sound source localization device 10 is a device that detects the sound wave SW output from the sound source 20 with a plurality of microphones constituting the microphone array 11 and obtains an accurate direction of the sound source from sound source data output from the plurality of microphones.

  The microphone array 11 is a device that inputs the sound wave SW output from the sound source 20. The microphone array 11 is composed of a plurality of microphones.

  As shown in FIG. 3, the microphone array 11 is composed of 16 microphones (microphones 11A to 11P) arranged at equal intervals on the circumference. Each microphone desirably has no directivity. The sound waves SW output from the sound source 20 are input to the microphones 11A to 11P, respectively. Each of the microphones 11 </ b> A to 11 </ b> P converts the input sound wave SW into analog audio data (analog sound source data E <b> 11) and outputs the analog audio data to the channel switching unit 12. That is, the microphone array 11 outputs all the analog sound source data E11 output from each of the 16 microphones 11A to 11P to the direction detection unit 50.

  In this specification, the concept of channel is used to identify a set of adjacent microphones. When there are 16 microphones, the number of channels is 16.

  The direction detection unit 50 obtains direction data indicating the direction of the sound source from the input analog sound source data E11 for each channel corresponding to a combination of two microphones having a predetermined positional relationship. The direction detection unit 50 includes a channel switching unit 12 and a single channel direction detection unit 13.

  The channel switching unit 12 is configured by a multiplexer, for example. When the number of channels is 16, the channel selection signal Ech is composed of, for example, a digital signal of 4 bits (2 4 = 16). The channel switching unit 12 selects, from the analog sound source data E11, sound source data E12 of a pair of adjacent microphones corresponding to the channel indicated by the channel selection signal Ech input from the single channel direction detection unit 13. Then, the channel switching unit 12 converts the selected sound source data E12 into digital signals by A / D (Analog / Digital) converters 12a and 12b, and outputs the digital signals to the single channel direction detection unit 13.

  The single channel direction detection unit 13 obtains direction data indicating the direction of the sound source for each channel from the input signal of the set of sound source data E12. Then, the single channel direction detection unit 13 outputs the obtained direction data to the FFT unit 14 as channel-specific direction data E13.

  The single channel direction detection unit 13 switches a set of sound source data E12 input to the single channel direction detection unit 13 by outputting a channel selection signal Ech to the channel switching unit 12. And the single channel direction detection part 13 calculates | requires direction data for every channel, switching a channel sequentially. Hereinafter, switching between channels will be described in detail.

  When the channel 1 is selected, the single channel direction detection unit 13 outputs a channel selection signal Ech such that the sound source data E12 of the microphones 11A and 11B is output from the channel switching unit 12 to the channel switching unit 12. Then, the single channel direction detection unit 13 obtains the direction data of the channel 1 from the input sound source data E12.

  Moreover, the single channel direction detection part 13 will switch to the next channel, if the direction detection of one channel is completed. Specifically, the channels are switched in the order of channel 1 → channel 2 →. For example, channel 2 corresponds to a set of microphones 11B and 11C. The same applies to the channel 3 and subsequent channels, and the channel 16 corresponds to a set of microphones 11P and 11A. And the single channel direction detection part 13 calculates | requires the direction data for every channel, whenever a channel is switched.

  Here, the reference | standard of the angle at the time of expressing direction data is demonstrated using FIG.

  FIG. 4 illustrates a case where the direction of the sound source 20 is detected using the sound source data E12 output from the channel 1, that is, the microphones 11A and 11B. As shown in FIG. 4, the center point of the line segment L1 connecting the microphone 11A and the microphone 11B is a center point O. Further, an angle formed by a half line L2 extending perpendicularly to the line segment L1 from the center point O toward the sound source 20 and a line segment L3 connecting the center point O and the sound source 20 is defined as a sound source direction θ. Here, with respect to the center point O, the direction of the microphone 11A is -90 °, the direction of the microphone 11B is 90 °, and the direction in which the half line L2 extends is 0 °. In other channels, the microphone with the younger alphabet (in the case of P and A, where P is younger than A) is replaced with microphone 11A, and the other microphone is replaced with microphone 11B. Think about the angle standard.

  The FFT unit 14 regards the channel-specific direction data E13 input from the single channel direction detection unit 13 as time-series data, and performs fast Fourier transform on the waveform represented by the channel-specific direction data E13. Then, the FFT unit 14 outputs the converted value to the filter unit 15 as the direction data frequency component E14.

  The filter unit 15 performs a filtering process on the direction data frequency component E14 input from the FFT unit 14. Then, the filter unit 15 outputs the filtered value to the IFFT unit 16 as a filtered direction data frequency component E15.

  The IFFT unit 16 performs an inverse Fourier transform on the filtered direction data frequency component E16 input from the filter unit 15. Then, the IFFT unit 16 outputs the converted value to the sound source localization unit 17 as filtered direction data E16.

  The sound source localization unit 17 estimates the direction of the sound source 20 from the filtered direction data E16 input from the IFFT unit 16, and outputs it as localization data E17.

  The control unit 18 outputs a control signal E18 for controlling the timing of the channel switching unit 12, the single channel direction detection unit 13, the FFT unit 14, the filter unit 15, the IFFT unit 16, or the sound source localization unit 17.

  The temperature measuring unit 19 includes a temperature sensor that measures the temperature. The temperature measuring unit 19 converts the measured temperature data into an air temperature signal E19 that is an electrical signal, and outputs it to the single channel direction detecting unit 13. Note that the temperature measuring unit 19 does not have to use an approximate value assuming that the temperature is normal temperature.

  The sound source 20 is a generation source of the sound wave SW and is a device to be localized by the sound source localization device. The sound source 20 does not need to be a sound source that conforms to a specific standard such as a MIDI (Musical Instrument Digital Interface) sound source, and may be a person or an animal as long as it generates a sound wave SW.

  The sound source localization apparatus 10 shown in FIG. 1 physically includes a computer 110 as shown in FIG. The computer 110 includes a multiplexer 111, an A / D converter 112, a ROM (Read Only Memory) 113, a RAM (Random Access Memory) 114, a CPU (Central Processing Unit) 115, and an input / output unit 116. In the functional configuration of the sound source localization apparatus 10 illustrated in FIG. 1, the computer 110 includes a direction detection unit 50, an FFT unit 14, a filter unit 15, an IFFT unit 16, a sound source localization unit 17, and a control unit 18. And.

  The multiplexer 111 is a device corresponding to the channel switching unit 12. When the analog sound source data E11 from the microphone array 11 is input, the multiplexer 111 sequentially switches and outputs the analog sound source data E11 according to the control of the CPU 115.

  The A / D converter 112 converts the analog sound source data E11 output from the multiplexer 111 into digital data and outputs the digital data.

  The CPU 115 performs operations such as calculation and control using the RAM 114 as a main memory and a work area according to an operation program stored in the ROM 113. Thereby, the CPU 115 realizes the single channel direction detection unit 13, the FFT unit 14, the filter unit 15, the IFFT unit 16, the sound source localization unit 17, the control unit 18, and the like.

  The input / output unit 116 provides data indicating the determined direction of the sound source to other devices. Further, the input / output unit 116 outputs a channel selection signal Ech to the multiplexer 111 under the control of the CPU 115.

  Next, the operation of the sound source localization apparatus having the above configuration will be described with reference to the flowchart of FIG.

  When the sound source localization processing is started, first, the single channel direction detection unit 13 (CPU 115) initializes the channel number ch to “1” (step S10).

  Next, the single channel direction detection unit 13 performs channel switching (step S20). In this step, the single channel direction detection unit 13 outputs to the channel switching unit 12 a channel selection signal Ech such that a set of sound source data E12 corresponding to the channel number ch is input from the channel switching unit 12. Since the channel number ch is “1” immediately after the start of the sound source localization process, the single channel direction detection unit 13 outputs a channel selection signal Ech to which a set of sound source data E12 of the microphones 11A and 11B is input.

  The channel switching unit 12 outputs the digital sound source data E12 of the microphones 11A and 11B to the single channel direction detection unit 13 in accordance with the channel selection signal Ech from the single channel direction detection unit 13.

  The single channel direction detection unit 13 performs channel-specific direction detection processing on the input digital sound source data E12 of the microphones 11A and 11B (step S30). In this step, the single channel direction detection unit 13 performs direction detection for one channel by a conventional method.

  FIG. 6 is a flowchart for explaining the channel-specific direction detection processing in the flowchart shown in FIG. First, the single channel direction detection unit 13 performs frequency analysis by performing FFT processing on each of the two input sound source data E12. Thereby, the single channel direction detection part 13 calculates | requires the spectrum information of the two sound source data E12, respectively (step S31). Note that the single channel direction detection unit 13 may remove noise components by performing filter processing after performing frequency analysis.

  Next, the single channel direction detection unit 13 extracts overtone components from the spectrum information of the two sound source data E12 (step S32).

Next, the single channel direction detection unit 13 calculates actual data IPD (Interaural Phase Difference) from the phase difference for each overtone (including fundamental tone) (step S33). Specifically, the actual data IPD (defined as Δφ _R ) for each overtone can be obtained from Equation (1).
(Equation 1)
Δφ _R = arctan (I [Sb] / R [Sb]) − arctan (I [Sa] / R [Sa]) (1)
However,
Sa: Spectral value of overtone of microphone 11A Sb: Spectral value of harmonic overtone of microphone 11B R [Sa], R [Sb]: Real part of overtone spectrum I [Sa], I [Sb]: Imaginary part of overtone spectrum

Next, the single channel direction detection unit 13 calculates actual data IID (Interaural Intensity Difference) from the intensity difference for each harmonic (step S34). Specifically, out of the spectrum information of the sound source data of the microphones 11A and 11B, if the dB (decibel) value of the spectrum of each harmonic is defined as Da and Db, the actual data IID (defined as ΔIs) for each harmonic is expressed by the equation. It can be obtained from (2).
(Equation 2)
ΔIs = Da−Db (2)

Next, a single-channel direction detecting section 13 obtains the direction information S _I using actual data IID for each harmonic obtained by Equation (2). The direction information S _I is obtained by using the actual data IPD when the harmonic frequency is less than the threshold fth, and by using the actual data IID when the harmonic frequency is greater than or equal to the threshold fth. Here, the direction information S _I is obtained by taking into account the actual data IID for each harmonic to fmax is the maximum frequency to be considered from the threshold fth. That is, the direction information S _I can be obtained from Expression (3) when a function in which harmonic frequencies are arranged in ascending order is defined as H (i). The function H (i) can be expressed as H (0) = f ₀ , H (1) = 2f ₀ , H (2) = 3f ₀ ,..., For example, where the fundamental frequency is f ₀ .

  nth is an integer value that satisfies H (nth) ≧ fth and is closest to fth. For example, when fth = 1500 Hz, H (0) = 600 Hz, H (1) = 1200 Hz, H (2) = 1800 Hz, and H (3) = 2400 Hz, the value of nth is 2. N-1 is an integer value at which H (n-1) is fmax.

The threshold fth can be obtained from Equation (4), where ν is the speed of sound (m / s) and λ is the distance (m) between the microphone 11A and the microphone 11B.
(Equation 4)
fth = ν / λ (4)

For example, when ν is 340 m / s and λ is 20 cm, the threshold fth is 1700 Hz. In this case, the IPD frequency harmonics is less than 1700 Hz, and calculates the direction information _{S I} using IID above 1700 Hz. Note that the speed of sound ν can be obtained from the air temperature signal E19 (air temperature) detected by the temperature measuring unit 19.

Here, it can be considered that the closer to the S _I 0 determined by the equation (3), the sound source is present in the front direction (the extending direction of the ray L2 in Fig. 4). Further, it can be considered that S _I is if it is negative the right direction (direction of the microphone 11B in FIG. 4), (the direction of the microphone 11A of FIG. 4) left if positive is S _I, the sound source each occurrence .

Next, the single channel direction detection part 13 calculates | requires IPD used as a model (step S35). Specifically, an angle of 5 ° intervals theta ', by defining the frequency of the harmonic and f, (defined as [Delta] [phi _M) model IPD can be obtained from equation (5).
(Equation 5)
Δφ _M = (2πf / ν) × λ (sin θ ′) (5)

Next, the single channel direction detection unit 13 obtains the certainty of IPD and the certainty of IID (step S36). Specifically, the actual data IPD obtained by the expression (1) is compared with the model IPD, and the certainty of the IPD with respect to the direction every 5 ° is obtained by a Gaussian probability distribution. Further, the direction information S _I obtained by the equation (3) the confidence of the actual data IID, defined as shown in the table of FIG. 8.

FIG. 8 is a table that defines the certainty factor of the IID. For example, the reliability of IID when θ ′ is 90 ° to 35 ° is 0.35 when the direction information S _I obtained by the equation (3) is “+”, and 0.65 when “−”. It becomes.

  Next, the single channel direction detection unit 13 extracts the direction information and outputs it to the FFT unit 14 as channel-specific direction data E13 (step S37). Specifically, the certainty of IPD and the certainty of IID are integrated by the Demster-Shafer theory that integrates the basic probabilities inferred from independent evidence, and the direction information with the highest certainty is made the true direction. . The single channel direction detection unit 13 completes the channel-specific direction detection process (step S30) when the direction information extraction is completed.

  When the channel-specific direction detection process is completed, the single channel direction detection unit 13 determines whether or not the channel number ch is 16 (step S40). When the single channel direction detector 13 determines that the channel number ch is not 16, that is, the direction detection for 16 channels is not completed, the channel number ch is incremented (step S50), and the channel is switched (step S20). Return processing to.

  On the other hand, when the single channel direction detection unit 13 determines that the channel number ch is 16, that is, the direction detection for 16 channels has been completed, the single channel direction detection unit 13 proceeds to the localization process (step S60).

  FIG. 7 is a flowchart for explaining the localization processing in the flowchart shown in FIG. First, the FFT unit 14 creates time-series direction data from the channel-specific direction data E13 (step S61). Specifically, the FFT unit 14 regards the direction data for 16 channels acquired in the channel-specific direction detection processing (step S30) as time-series data, and creates time-series direction data. Here, the direction data of each channel will be described with reference to FIG.

  FIG. 9 is a diagram showing an example of the measurement angle and ideal angle of direction data in each channel. The measurement angle is direction data actually acquired in the channel-specific direction detection process (step S30) when the microphone array 11 and the sound source 20 are in the positional relationship shown in FIG. The ideal angle is direction data to be acquired in the channel-specific direction detection process (step S30) when the microphone array 11 and the sound source 20 are in the positional relationship shown in FIG.

  As illustrated in FIG. 9, the measurement angle actually acquired in the channel-specific direction detection process (step S30) is different from the ideal angle. This is because there is sound reflection, noise, and the like in the actual direction detection. When the data shown in FIG. 9 is plotted with the horizontal axis as a channel and the vertical axis as an angle, a triangular wave as shown in FIG. 10 is obtained.

In FIG. 10, the vertical axis represents an angle (°), and the horizontal axis represents a channel (ch). Since the channel is selected at a constant period, it can be considered that channel (ch) = time (s). In other words, the graph of FIG. 10 is obtained by connecting and plotting the channel-specific direction data E13 according to the detection (selection) timing, and has a constant periodic waveform. FIG. 10 shows time-series direction data for three periods (16 × 3 channels).
In FIG. 10, the waveform formed by the measurement angle is indicated by a solid line, and the waveform formed by the ideal angle is indicated by a broken line.

  As described above, the time-series direction data is data in which the direction data acquired in the channel-specific direction detection process (step S30) is simply arranged in time series. For this reason, when the sound source 20 does not move, the time-series direction data becomes the same or close value even if acquired every time, and the waveform becomes a periodic waveform.

  The FFT unit 14 performs FFT processing on the created time-series direction data (step S62). That is, the FFT unit 14 regards the horizontal axis of the waveform shown in FIG. 10 as time, and converts the time-series direction data into the direction data frequency component E14 by a well-known FFT process. Then, the FFT unit 14 outputs the direction data frequency component E14 to the filter unit 15.

  The filter unit 15 filters the direction data frequency component E14 input from the FFT unit 14 (step S63). The time-series direction data composed of ideal angles forms a shape close to a triangular wave on the time axis, as indicated by a broken line in FIG. This is because the microphones are arranged at regular intervals, and therefore, when the sound source 20 is sufficiently separated from the size of the microphone array 11, the angle of the sound source 20 viewed from the set of microphones corresponding to the selected channel. This is because it can be estimated that each time the channel is switched, the angle changes by a certain angle (360 ° / 16 = 22.5 °). On the other hand, time-series direction data composed of measurement angles does not have a shape close to a triangular wave on the time axis as shown by a solid line in FIG. 10 due to the influence of reflection and noise. The filter unit 15 performs filtering so as to remove the component due to reflection and noise from the direction data frequency component E14.

  Specifically, the filter unit 15 extracts only the frequency components constituting the triangular wave and removes other frequency components. That is, since the triangular wave is composed only of the fundamental wave and the odd-order harmonics, the even-order harmonics are removed. With reference to FIGS. 11A and 11B, an example of the filter processing will be described. In FIGS. 11A and 11B, f indicates the frequency of the fundamental wave.

  FIG. 11A shows an example of the frequency component of the direction data before performing the filtering process. As shown in FIG. 11A, before performing the filter processing, the frequency components of the second harmonic to the sixth harmonic are included in addition to the frequency components of the fundamental wave.

  Here, a filtering process for cutting frequency components of even-order harmonics (other than fundamental waves and odd-order harmonics) is performed. FIG. 11B shows an example of the frequency component of the direction data after the filtering process. As shown in FIG. 11B, after the filtering process, only the frequency components of the fundamental wave and the odd harmonics are left. In other words, the frequency components of even harmonics that are considered to be generated by noise or reflection are cut off. Further, the phase (phase spectrum) of the remaining frequency component has not changed, and the intensity (intensity spectrum) has not changed. That is, in the filtering process, by cutting only the even-numbered harmonics of the direction data frequency component E14, the influence due to noise and reflection can be removed, and the time-series direction data after IFFT processing can be made closer to a triangular wave.

  The filter unit 15 outputs the filtered data to the IFFT unit 16 as a filtered direction data frequency component E15.

  The IFFT unit 16 performs inverse transformation of IFFT processing, that is, FFT processing executed by the FFT unit 14 on the filtered direction data frequency component E15 input from the filter unit 15 (step S64). The IFFT unit 16 converts the filtered direction data frequency component E15 into filtered direction data E16 that is time-series direction data by IFFT processing. The filtered direction data E16 will be described with reference to FIG.

  FIG. 12 shows a waveform formed by the filtered direction data E16. In FIG. 12, the vertical axis represents the angle (°), and the horizontal axis represents time (s). The filtered direction data E16 is composed of direction data for 16 channels, similarly to the direction data before filtering. Therefore, in FIG. 12, the horizontal axis is time (s) in order to show a waveform as time-series direction data, but the horizontal axis may be considered as a channel (ch). FIG. 12 shows time-series direction data for three periods (16 × 3 channels). In FIG. 12, the time for one cycle is divided by a dotted line.

  As shown in FIG. 12, the waveform formed by the filtered direction data E <b> 16 is a waveform close to a triangular wave as compared to the waveform formed by the direction data before filtering (the waveform indicated by the solid line in FIG. 11). This is because the frequency components due to noise and reflection are removed because the frequency components of even-order harmonics are cut from the direction data frequency component E14 in the filter processing. When the IFFT process is completed, the IFFT unit 16 outputs the filtered direction data E16 to the sound source localization unit 17.

  The sound source localization unit 17 obtains the direction of the sound source 20 from the filtered direction data E16 input from the IFFT unit 16 and outputs it as localization data E17 (step S65). As described above, frequency components due to noise and reflection are removed from the filtered direction data E16. The sound source localization unit 17 obtains localization data E17 based on the filtered direction data E16. An example of a specific method for obtaining the localization data is shown below.

  As described above, the microphones are arranged at equal intervals on the circumference, and the single channel direction detection unit 13 detects the direction while shifting the set of microphones. For this reason, it can be considered that the horizontal axis shown in FIG. 12 indicates not only the time but also the position on the channel or the circumference. Therefore, for example, the direction of the sound source 20 can be specified from a channel corresponding to a point (a point indicated by a black dot in FIG. 12) at which the angle obtained by direction detection is 0 ° or a position on the circumference.

  When the sound source localization unit 17 outputs the obtained localization data E17 to the outside, the localization process (step S60) is completed. Thereby, the sound source localization process is completed.

  In addition, this invention is not limited to the said Example, A various deformation | transformation and application are possible.

  In the above embodiment, the number of microphones is 16, but the number of microphones is arbitrary. In order to increase accuracy, the number of microphones may be set to 24 or 32, for example. Further, the number of microphones may be reduced to eight, for example, for speeding up and power saving.

  In the said embodiment, although the microphone was arrange | positioned at equal intervals, arrangement | positioning of a microphone is arbitrary. For example, as shown in FIG. 13, five microphones may be arranged at positions 11A, 11C, 11D, 11I, and 11L.

  In this case, for example, an arbitrary reference direction Vr is determined as shown in FIG. The direction of the channel with respect to the reference direction Vr is represented by an intersection angle φ. In FIG. 14, the channel direction is indicated by a straight arrow. The reference direction Vr can be the same as the channel direction of the pair of microphones 11A and 11C, for example. The directions of the channels of the microphones 11C and 11D, the microphones 11D and 11I, the microphones 11I and 11L, and the microphones 11C and 11D are crossed angles φ1, φ2, φ3, and the reference direction Vr, respectively. There is a relationship of φ4.

  In the case of the configuration shown in FIG. 14, as shown in FIG. 15, data is plotted with the vertical axis representing the angle (the direction of the sound source 20 with respect to the reference direction for each channel) and the horizontal axis representing the crossing angle φ. Then, FFT processing may be performed on a waveform composed of the plotted data by regarding the horizontal axis (crossing angle φ) in the figure as time.

  Further, the microphone need not be arranged on the circumference. FIG. 16 shows an example in which four microphones 11Q, 11R, 11S, and 11T are arranged to form a parallelogram. In the example of FIG. 16, the reference direction Vr is defined as a direction from the microphone 11T to the microphone 11Q, for example. Also in this case, the direction of each channel with respect to the reference direction Vr is obtained as the intersection angles φ1, φ2, φ3, and φ4 as in the example shown in FIG. Thereafter, similarly to the example shown in FIG. 15, data may be plotted and FFT processing may be performed.

  In the above embodiment, the sound source localization unit 17 specifies the direction of the sound source 20 from the point where the angle obtained by the direction detection is 0 °. However, the direction of the sound source 20 may be specified from the point at which the waveform of the filtered direction data E16 shown in FIG. Alternatively, the value of each point of the filtered direction data E16 may be handled as the direction data of each channel, and the direction data of all the channels may be comprehensively determined to specify the direction of the sound source 20.

  Moreover, in the said embodiment, the filter part 15 performed the filter process which only cuts the frequency component of an even-order harmonic from the direction data frequency component E14. However, in order to make the waveform formed by the filtered direction data E15 closer to a triangular wave, not only the frequency components of the even-order harmonics are cut, but also a filter process for attenuating the intensity of the odd-order harmonics may be performed. .

  Specifically, after the frequency component of the even-order harmonic is cut, for example, the intensity obtained by dividing the intensity of the odd-order harmonic by the order of the harmonic is defined as the intensity. That is, only the amplitude component is changed so that the phase does not change. That is, attenuation is performed while maintaining the ratio between the real part and the imaginary part of the odd-order harmonics.

  FIG. 11C shows an example of the frequency component of the direction data after performing the filtering process for attenuating the intensity of the odd-order harmonics. As shown in FIG. 11 (c), after the filtering process, the frequency components of the fundamental wave and odd harmonics remain, but the intensity of the odd harmonics is attenuated according to the order. . However, even in this case, the phase of the remaining frequency component does not change. By performing such a filtering process on the direction data frequency component E14, it is possible to remove the influence due to noise and reflection and make the filtered direction data E16 closer to a triangular wave.

  In the embodiment described above, the sound source localization unit 17 obtains only the direction of the sound source from the direction data for 16 channels. However, the distance to the sound source may be obtained by a conventional method. For example, the distance to the sound source can be obtained by superimposing direction data for 16 channels.

  The above hardware configuration and flowchart are examples, and can be arbitrarily changed and modified. In the above embodiment, the sound source localization device 10 is described as being configured with discrete components. For example, most of the processing executed by the sound source localization device 10 is performed in a processor circuit such as a CPU or a DSP (Digital Signal Processor). It is also possible to execute. With such a configuration, the circuit configuration can be simplified.

  In the above-described embodiment, an example in which the present invention is applied to a sound source localization apparatus and a localization method has been described. However, in the present invention, noise is extracted from a data sequence that becomes a constant periodic function by plotting with reference to an arbitrary parameter. Widely applicable when removing. For example, it is assumed that a periodic waveform is obtained on the basis of this parameter by plotting the data group based on a certain parameter. In this case, FFT is performed by regarding the parameter as time. Thereby, the harmonic component of the waveform is suppressed and noise is removed. Thereafter, IFFT is performed to remove noise, and subsequent processing can be performed using the obtained waveform.

  This application is based on Japanese Patent Application No. 2007-133329 filed on May 18, 2007. The specification, claims, and entire drawings are incorporated herein by reference.

  The present invention can be suitably used in all technical fields that use sound source localization, such as robots and monitoring systems.

Claims (7)

A microphone array composed of at least three or more microphone elements;
Voice data for each microphone element is input from the microphone array, and for each channel corresponding to a combination of two microphones having a predetermined positional relationship, the direction of the sound source is shown with reference to the positions of the two microphones corresponding to the channel. Direction detection means for obtaining direction data;
The direction data for each channel is input from the direction detection means, the input direction data for each channel is concatenated, and the frequency component of the waveform represented by the concatenated direction data is obtained by Fourier transforming the data regarded as time series data. Fourier transform means;
Filter means for inputting the frequency component obtained by the Fourier transform means, and outputting as a filtered frequency component a frequency component of the even-order harmonics suppressed among the inputted frequency components;
An inverse Fourier transform unit that inputs a filtered frequency component from the filter unit, and obtains filtered data by performing an inverse Fourier transform on the filtered frequency component;
Sound source localization means for obtaining the direction of the sound source from the filtered data obtained by the inverse Fourier transform means;
Comprising
A sound source localization device characterized by that. The channel corresponds to a combination of two adjacent microphones;
The sound source localization apparatus according to claim 1. The filter means includes
The frequency component obtained by the Fourier transform means is input, the frequency component of the even-order harmonic is suppressed among the input frequency components, and the frequency component of the odd-order harmonic is further attenuated and output as the filtered frequency component. ,
The sound source localization apparatus according to claim 1. The direction detecting means includes
A single channel direction detection means for outputting a channel selection signal for selecting a channel, inputting voice data of a channel corresponding to the channel selection signal, and obtaining direction data for each channel;
Channel switching means for inputting voice data for each microphone element from the microphone array and outputting voice data corresponding to a channel indicated by the selection signal input from the single channel direction detecting means;
Further comprising
The sound source localization apparatus according to claim 1. The microphone array is
Consists of omnidirectional microphone elements
The sound source localization apparatus according to claim 1. Audio data for each microphone element is input from a microphone array composed of at least three or more microphone elements, and for each channel corresponding to a combination of two microphones in a predetermined positional relationship, two microphones corresponding to the channel A direction detecting step for obtaining direction data indicating the direction of the sound source relative to the position;
A Fourier transform step for obtaining frequency components of a waveform represented by the direction data represented by concatenating the direction data for each channel obtained by the direction detection step and performing Fourier transform on what is regarded as time-series direction data;
A filter step of outputting a frequency component obtained by suppressing the frequency component of the even-order harmonics among the frequency components obtained by the Fourier transform step as a filtered frequency component;
An inverse Fourier transform step for obtaining a filtered data by performing an inverse Fourier transform on the filtered frequency component output by the filter step;
A sound source localization step for obtaining a direction of a sound source from the filtered data obtained by the inverse Fourier transform step;
Comprising
A sound source localization method characterized by that. Computer
Two channels corresponding to each channel corresponding to a combination of two microphones in a predetermined positional relationship based on audio data for each microphone element input from a microphone array composed of at least three or more microphone elements. Direction detection means for obtaining direction data indicating the direction of the sound source with respect to the position of the microphone;
Fourier transform means for concatenating the direction data for each channel obtained by the direction detection means and obtaining a frequency component of the waveform represented by the direction data represented by Fourier transform of what is regarded as time series data,
Filter means for obtaining a filtered frequency component obtained by suppressing the frequency component of the even-order harmonics among the frequency components obtained by the Fourier transform means;
Inverse Fourier transform means for obtaining filtered data by performing inverse Fourier transform on the filtered frequency component obtained by the filter means;
Sound source localization means for obtaining the direction of the sound source from the filtered data obtained by the inverse Fourier transform means,
A program characterized by functioning as

Images