JP2010283676A - Sound detection apparatus, sound detection method and imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect only a sound source in the room by reliably excluding sounds outside the room. <P>SOLUTION: A sound detection apparatus includes: a plurality of microphones 112A, 112B disposed while being spaced apart from each other; a sound direction information-calculating section 132 for calculating an incident angle θ of sounds to the microphones based on a phase difference between sound information collected by the microphones 112A, 112B; a sound directivity-discriminating section 134 for discriminating directivity of sounds generated from a sound source based on the incident angle θ; and a sound detecting section 120 for detecting only sound information for which it is determined that the directivity has been fixed, based on the sound information collected by the microphones 112A, 112B. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a voice detection device, a voice detection method, and an imaging system.

  Recently, in surveillance devices for security applications, it is assumed that suspicious persons and intruders will be detected by detecting moving objects using images from surveillance cameras and detecting abnormal sounds using sound from microphones. ing.

JP 2007-47088 A JP 2003-189139 A Japanese Patent Laid-Open No. 11-64090

  In surveillance devices such as security applications, methods for detecting suspicious individuals, etc. using sound include sound pressure detection using only the sound pressure of the collected sound, and sound detection using sound features. is there. In these methods, sound collected using a microphone or the like is used. However, when it is desired to detect a suspicious person or the like based on indoor sound, both outdoor sounds are detected and erroneous detection can be performed. There is sex. For this reason, a method of attenuating outdoor sound by providing directivity with an array microphone or the like, a method of removing stationary noise, a method of learning and removing a feature amount of sound from outside the room, and the like are considered.

  However, these methods are effective to some extent when outdoor sound information is known in advance, but they are not effective at all for non-stationary or unknown sound from outside. There is a high possibility of false detection of outdoor noise that should not be detected.

  For example, the above-described method of attenuating outdoor sound by providing directivity with an array microphone or the like provides directivity in the direction of indoor sound intended for detection, so that sound from other directions can be obtained. It is to suppress. However, there is a high possibility that noise from outside the room is picked up from the direction of directivity, and this method increases the possibility of erroneous detection.

  Further, the method for removing stationary noise is to learn and remove stationary noise levels such as spectral subtraction, but it is hardly effective for non-stationary noise.

  Furthermore, the method of learning and excluding the feature amount of the sound from the outside is learned in advance, and the feature amount of the sound from the outside is learned in advance, and if the collected sound matches the feature amount, it is regarded as the sound from the outside. However, since learning is required in advance, unlearned speech cannot be removed. For this reason, when the feature amount of the outdoor sound is similar to the feature amount of the indoor sound, there is a problem that the outdoor sound cannot be removed.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a novel and improved technique capable of detecting only necessary sound from sounds from a plurality of sound sources. An object of the present invention is to provide a voice detection device, a voice detection method, and an imaging system.

  In order to solve the above-described problem, according to an aspect of the present invention, a plurality of microphones that are spaced apart from each other, and a voice to the microphone based on a phase difference between voice information collected by the microphone From the sound direction information calculating unit for calculating the incident angle of the sound, the sound directionality determining unit for determining the directionality of the sound emitted from the sound source based on the incident angle, and the sound information collected by the microphone, There is provided a voice detection device including a voice detection unit that detects only voice information for which it is determined that the directionality is fixed.

  The sound information collected by each of the plurality of microphones includes a Fourier transform unit that performs time-frequency transform such as Fourier transform, and the sound direction information calculation unit includes a phase difference of each frequency obtained by the Fourier transform. The incident angle is calculated for each frequency based on the sound direction, the sound direction determination unit determines the directionality for each frequency, the sound detection unit for the sound information collected by the microphone, Only the audio information of the frequency for which it is determined that the directionality is fixed may be detected.

  The sound directionality determination unit may determine whether or not the directionality is determined based on a temporal change in the incident angle for a sound having a specific frequency.

  The sound directionality determination unit may determine whether or not the directionality is determined based on the degree of coincidence of incident angles for a plurality of frequencies of sound.

  In order to solve the above-described problem, according to another aspect of the present invention, based on the phase difference of audio information collected by a plurality of microphones arranged apart from each other, the audio to the microphone is From the sound direction information calculating step for calculating the incident angle, the sound direction determining step for determining the directionality of the sound emitted from the sound source based on the incident angle, and the sound information collected by the microphone, There is provided a voice detection method including a voice detection step of detecting only voice information for which directionality is determined to be determined.

  The sound information collected by each of the plurality of microphones includes a step of Fourier transform, and in the sound direction information calculation step, for each frequency based on a phase difference of each frequency obtained by the Fourier transform The incident angle is calculated, the directionality is determined for each frequency in the sound direction determination step, and the directionality is determined for the sound information collected by the microphone in the sound detection step. Only audio information having a frequency determined to be detected may be detected.

  In the sound direction determination step, it may be determined whether or not the directionality is determined based on a temporal change in the incident angle for a sound having a specific frequency.

  Further, in the sound directionality determining step, it may be determined whether or not the directionality is determined based on the degree of coincidence of incident angles for a plurality of frequencies of sound.

  In order to solve the above-described problem, according to another aspect of the present invention, a plurality of microphones arranged apart from each other and a microphone based on a phase difference between sound information collected by the microphone. A sound direction information calculating unit for calculating the incident angle of the sound to the sound, a sound direction determining unit for determining the direction of the sound emitted from the sound source based on the incident angle, and the sound collected by the microphone An imaging system comprising: a voice detection unit that detects only voice information for which the directionality is determined from information; and a tracking camera that changes a shooting direction based on a detection result by the voice detection unit. Is provided.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the audio | voice detection apparatus, the audio | voice detection method, and imaging system which can detect only the required audio | voice among the audio | voices from a several sound source.

It is a schematic diagram which shows the structure of the audio | voice monitoring apparatus which concerns on the 1st Embodiment of this invention. It is a schematic diagram which shows the method of detecting the direction of a sound source. It is the schematic diagram which showed the structure of FIG. 2 in detail. It is a flowchart which shows the process sequence performed with the audio | voice monitoring apparatus of 1st Embodiment. It is a schematic diagram which shows the tracking camera system which concerns on 2nd Embodiment.

Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol. The description will be made in the following order.
1. 1. First Embodiment (1) Configuration Example of Voice Monitoring Device (2) Processing Performed by Voice Monitoring Device Second Embodiment (1) Configuration Example of Imaging System

<1. First Embodiment>
(1) Configuration Example of Voice Monitoring Device First, a schematic configuration of the voice monitoring device 100 according to the first embodiment of the present invention will be described with reference to FIG. As shown in FIG. 1, the voice monitoring apparatus 100 according to the present embodiment includes a microphone module 110, a voice detector 120, and a sound directionality discriminator 130.

  As shown in FIG. 1, the voice monitoring apparatus 100 is arranged indoors. Further, as shown in FIG. 1, there is an indoor sound source in the room, and there is an outdoor sound source outside the room separated by a wall. Here, the indoor sound source is, for example, a sound source by a suspicious person who has entered the room. In addition, the outdoor sound source is a sound source whose source is a person, a vehicle, a device, or the like that travels outside the room separated by walls, ceilings, floors, and the like. The microphone module 110 includes a plurality of microphones, and can collect sound from both the indoor sound source and the outdoor sound source.

  One of the differences between the sound generated by the indoor sound source and the sound generated by the outdoor sound source is the presence or absence of direct sound. The sound generated by the sound source in the room where the sound monitoring apparatus 100 is arranged is a direct sound and often enters the sound collecting microphone module 110 almost directly. On the other hand, since the sound from the outdoor sound source always passes through the wall, ceiling, floor, etc., most of the sound is indirect and there is no component that directly enters the microphone module 110. For this reason, the direction information of the sound source cannot be obtained only with monaural sound picked up by a normally used omnidirectional microphone or a directional microphone including an array microphone. For this reason, it is very difficult to discriminate sounds from outside.

  In addition, sound from a plurality of different frequencies is generated from the indoor sound source and the outdoor sound source, but the incident angle when sound for each frequency is incident on the microphone module 110 depends on whether the sound is from outside or from inside the room. There are differences in features as follows. The first difference is a change in incident angle over time when attention is paid to one frequency. The indoor sound source is stable at a constant angle, but the outdoor sound source is not stable in angle. The second difference is that when the incident angles are compared for each frequency, there is a high possibility that the incident angle is the same for each frequency in the indoor sound source, but the incident angle is likely to be different for each frequency in the outdoor sound source.

  In the example of FIG. 1, the sound from the indoor sound source is directly transmitted from the sound source to the microphone module 110 in one direction. On the other hand, since the sound from the outdoor sound source is first transmitted to the wall and then transmitted along the wall, it is transmitted to the microphone module 110 from a wide range of the wall. Therefore, the sound from the outdoor sound source has lost its directionality and is indirectly transmitted to the microphone module 102 from multiple directions.

  In this embodiment, based on the difference between the sound of the indoor sound source and the outdoor sound source as described above, the directionality of the collected sound is obtained based on the characteristics of the first and second differences, It is determined whether the sound is from the room or from the room. FIG. 2 is a schematic diagram showing a method for detecting the direction of the sound source. As shown in FIG. 2, the microphone module includes a microphone (microphone) A and a microphone (microphone) B. The microphone A, the microphone B, and the sound source are arranged in the space, and the incident angle of the sound from the sound source entering the microphone A and the microphone B with respect to the straight line connecting the microphone A and the microphone B is θ.

If the distance between the microphones A and B is D, the sound arrival distance difference d between the two microphones A and B can be obtained from the following equation (1).
d = D cos θ (1)
The phase difference φ (f) for each frequency can be expressed by the following equation (2), where S is the sound speed.
φ (f) = 2πfd / S (2)
Accordingly, for the sound of each frequency f acquired by the microphone A and the microphone B, the incident angle θ can be obtained for each frequency f from the phase difference of each frequency f.

  Calculation of the incident angle θ is performed on audio information of each frequency generated from the sound source. In the sound monitoring apparatus 100, the sound information collected by the microphones A and B is Fourier-transformed by FFT in the microphone module 110, and sound pressure (power) and phase are detected for each frequency. Then, the sound direction discriminator 130 calculates the phase difference φ (f) between the phase of the sound of the microphone A and the phase of the sound of the microphone B for each frequency, and the equations (1) and (2) The incident angle θ is obtained from the equation.

  The sound directionality discriminator 130 discriminates the directionality of the sound source from the obtained incident angle θ for each frequency. Specifically, the sound directionality discriminator 130 discriminates whether the collected sound is from an indoor sound source or from an outdoor sound source, and uses the result information (likelihood) as a sound detector. 120 is output.

  The sound detector 120 receives sound information as a result of Fourier-transforming the collected sound information from the microphone module 110. The sound detector 120 outputs, as a detection result, sound having a frequency at which the sound source is determined to be indoors based on the sound direction determination result information sent from the sound direction determiner 130, and the sound source is outdoors. The sound of the frequency determined to be deleted is deleted.

  Next, the process performed by the voice monitoring apparatus 100 will be described in more detail based on FIG. FIG. 3 is a schematic diagram showing the configuration of the voice monitoring apparatus 100 of FIG. 2 in more detail. As shown in FIG. 3, the microphone module 110 includes a microphone (A) 112A, a microphone (B) 112B, a Fourier transformer (FFT) 114A, a Fourier transformer (FFT) 114B, and a sound pressure / phase information separation unit 116. . Microphone (A) 112A and microphone (B) 112B collect sound from indoor sound sources and outdoor sound sources.

The output of the microphone (A) 112A is sent to a Fourier transformer (FFT) 114A. The Fourier transformer (FFT) 114A Fourier-transforms the input audio information on the frequency axis, and outputs an output X _A (f) for each frequency f. The output of the microphone (B) 112B is sent to the Fourier transformer (FFT) 114B. Fourier transformer (FFT) 114B is Fourier transformed into a frequency axis audio information, and outputs the output _X B (f) for each frequency f. Here, X _A (f) and X _B (f) calculated by Fourier transform are X _A (f) = P _A (f) + φ _A (f) · i, X _B (f) = P _B ( f) + φ _B (f) · expressed as i (i is a complex number). P _A (f) and P _B (f) are sound pressures (power), and φ _A (f) and φ _B (f) are phases.

The outputs of the Fourier transformers 114A and 114B are input to the sound pressure / phase information separation unit 116, and the sound pressure and phase are separated. The sound pressures P _A (f) and P _B (f) are output to the sound detector 120, and the phases φ _A (f) and φ _B (f) are input to the sound directionality discriminator 134.

The sound directionality discriminator 134 includes a sound direction information calculation unit 132 and a sound directionality calculation unit 134. The sound direction information calculation unit 132 calculates φ (f) = φ _A (f) −φ _B (f) for each frequency, and calculates the sound collected by the microphone (A) 112A and the microphone (B) 112B. The phase difference φ (f) is calculated. And the sound direction information calculation part 132 calculates incident angle (theta) (f) for every frequency from (1) Formula and (2) Formula.

  Information on the incident angle θ (f) for each frequency is input to the sound directionality determination unit 134. The sound directionality determination unit 134 determines the directionality of the sound generated from the sound source based on the incident angle θ (f) for each frequency, and whether or not the sound of each frequency is generated from the indoor sound source. Is determined.

  More specifically, the sound directionality determination unit 134 obtains a fluctuation during a certain period T with respect to the incident angle θ (f) of each frequency, and when the fluctuation is large, the sound of the frequency generates an outdoor sound source. It is determined that On the other hand, when the variation in the incident angle θ (f) is small, the sound directionality determination unit 134 determines that the sound having the frequency is generated from the indoor sound source.

  In addition, the sound directionality determination unit 134 determines whether or not the incident angles θ (f) of the respective frequencies coincide with each other, and the sound having a frequency within which the incident angles θ (f) are coincident or within a predetermined range. Is determined to have an indoor sound source as a generation source. On the other hand, when the incident angles θ (f) of the respective frequencies do not coincide with each other, or when the incident angles θ (f) of the respective frequencies are not within the predetermined range, an outdoor sound source is used as the generation source for the sound of those frequencies. It is determined that it is to be. For example, among the detected sounds having the frequencies f1, f2, f3, f4, and f5, the incident angles of f1, f2, and f3 are the same at 30 °, and the incident angle of f4 is 40 °. Assume that the incident angle of f5 is 50 °. In this case, it is determined that the sound of the frequencies f1, f2, and f3 is generated from the indoor sound source, and the sound of the frequencies f4 and f4 is determined to be generated from the outdoor sound source. Further, among the plurality of frequencies f1 to f8, the incident angles of f1, f2, and f3 are the same at 30 °, the incident angle of f4 is 40 °, the incident angle of f5 is 50 °, and f6 , F7, and f8 are assumed to have the same incident angle of 60 °. In this case, the frequencies f1, f2, and f3 are determined as indoor sound sources, the frequencies f6, f7, and f8 are also determined as indoor sound sources, and the frequencies f4 and f4 are determined as outdoor sound sources. In this case, it is determined that there are two sound sources in the room, a sound source with an incident angle of 30 ° and a sound source with an incident angle of 60 °.

  As described above, the sound directionality determining unit 134 is either a change in the incident angle θ over time when focusing on one frequency or a change in the incident angle θ when comparing the incident angles for each frequency. When it is larger than the reference value, it is determined that the sound source is outdoor.

  The discrimination result by the sound direction discriminator 130 is output to the sound detector 120 as sound direction information. The sound directionality information is expressed as the probability (likelihood 0 to 1) of using the indoor sound source as the generation source for the sound of each frequency. The closer the likelihood is to 1 for a certain frequency, the higher the probability that the sound of that frequency will be generated from the indoor sound source. In the above example, among the sounds of the five frequencies f1, f2, f3, f4, and f5, the incident angles for f1, f2, and f3 are the same at 30 °, and the incident angle of f4 is 40 °. When the incident angle of f5 is 50 °, the voices of f1, f2, and f3 have a likelihood close to 1. On the other hand, the likelihood is close to 0 for the sounds of the frequencies f4 and f5.

The sound detector 120 detects the sound of the sound pressure information input from the microphone module 110 based on the likelihood input from the sound direction discriminator 130. At this time, for frequencies higher likelihood, the sound inputted from the sound directional discriminator 130 pressure P _{A (f),} and outputs the P B _(f), for the low frequency of likelihood, the likelihoods Accordingly, the sound pressures P _A (f) and P _B (f) are decreased or excluded. Thereby, only the sound which uses the indoor sound source as the generation source is output from the sound detector 120. The detection result output from the sound detector 120 may include sound direction information (incident angle θ) and sound direction determination result sent from the sound direction determination device 130.

(2) Processing Performed by Voice Monitoring Device Next, processing procedures performed by the voice monitoring device 100 of the present embodiment will be described based on the flowchart of FIG. First, in step S10, the microphone module 110 collects outdoor and indoor audio. In the next step S 12, sound direction information (incident angle θ) for each frequency is extracted from the sound collected in step S 10, and the sound direction information is output to the sound directionality determination unit 134.

  In the next step S14, voice information (sound pressure) is extracted from the voice collected in step S10, and this voice information is output to the voice detector 120. In the next step S16, the sound directionality (likelihood) is determined from the sound direction information (incident angle θ) obtained in step S12 by the sound directionality determination unit 134, and the sound directionality information is output to the sound detector 120. To do.

  In the next step S18, voice detection is performed by the voice detector 120 using the voice information and sound directionality information obtained in steps S14 and S16. Here, the sound information determined as the outdoor sound source is excluded based on the sound directionality information, and only the sound of the indoor sound source is detected. In step S20, the detection result of the sound detector 120 is output.

  As described above, the sound direction information is extracted by obtaining the incident angle θ of the sound from the sound source for each frequency for the sound collected by the plurality of microphones A and B included in the microphone module 110. The sound directionality determination is determined by comprehensively judging the stability of the incident angle θ for each frequency and the coincidence of the incident angles θ of a plurality of frequencies based on the sound direction information.

  As described above, according to the first embodiment, the incident angle θ is obtained for each frequency, and it is determined whether the sound is generated from the room based on the coincidence of the incident angles θ. It becomes possible. Therefore, even when the outdoor sound is relatively loud, or when the outdoor sound and the indoor sound feature amount are similar, it is possible to easily and reliably distinguish the outdoor sound from the indoor sound. It becomes possible, and the erroneous detection of the sound from the outside can be surely suppressed. Therefore, for example, it is possible to reliably detect intrusion of a suspicious person or the like into the room based only on the room sound.

<2. Second Embodiment>
(1) Configuration Example of Imaging System Next, a second embodiment of the present invention will be described. FIG. 5 is a schematic diagram illustrating an imaging system 200 according to the second embodiment. The imaging system 200 includes a tracking camera 210 that changes the shooting direction based on the sound detection result. In addition, the imaging system 200 includes a sound detector 120 and a sound direction discriminator 130 as in the sound monitoring apparatus 100 of the first embodiment.

  The tracking camera 210 includes an imaging optical system, an imaging element that photoelectrically converts a subject image formed by the imaging optical system, and a drive unit that changes the imaging direction by changing the direction of the optical axis of the imaging optical system. . The tracking camera 210 includes a plurality of microphones as in the microphone module 110 of the first embodiment. The sound information collected by the plurality of microphones is separated into sound pressure information and phase information as in the first embodiment, and is input to the sound detector 120 and sound direction discriminator 130. Then, the detection result is output from the sound detector 120 by the same method as in the first embodiment.

  In the second embodiment, the detection result of the sound detector 120 is input to the tracking camera 210. Here, the detection result output from the sound detector 120 includes sound direction information (incident angle θ) and sound direction determination result sent from the sound direction determination unit 130. The tracking camera 210 drives the drive unit based on the input of the detection result, and photographs the speaker by pointing the photographing lens toward the speaker who is transmitting the sound indoors.

  Also in the second embodiment, the voice of the outdoor person is excluded by the voice detector 120. The tracking camera 210 determines the shooting direction based on sound information (sound pressure) using the room as a sound source, sound direction information (incident angle θ) of the room sound source, and sound directionality. Therefore, the shooting direction of the tracking camera 210 is prevented from being directed toward the outdoor sound source, and the shooting direction of the tracking camera 210 can be directed only to the indoor speaker.

  Therefore, for example, in the case of a video conference, even when the sound source is outside the conference room, the shooting direction of the tracking camera 210 can be directed only to the indoor sound source (speaker).

  As described above, according to the second embodiment, in the imaging system 200 including the tracking camera 210, the shooting direction of the tracking camera 210 can be determined based only on the indoor sound source. Therefore, it is possible to reliably prevent the shooting direction of the tracking camera 210 from being directed toward an outdoor sound source when performing a video conference.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention. For example, in each of the above-described embodiments, the present invention is applied to the discrimination between the outdoor sound source and the indoor sound source. However, the present invention can be widely applied to the case where only the necessary sound is extracted by determining the direction of the sound source.

DESCRIPTION OF SYMBOLS 100 Audio | voice monitoring apparatus 110 Microphone module 112A, 112B Microphone 120 Audio | voice detector 132 Sound direction information calculation part 134 Sound directionality discrimination | determination part 114A, 114B Fourier transformer (FFT)
200 Imaging System 210 Tracking Camera

Claims (9)

A plurality of microphones spaced apart from each other;
A sound direction information calculation unit that calculates an incident angle of sound to the microphone based on a phase difference of sound information collected by the microphone;
A sound directionality determining unit that determines the directionality of the sound emitted from the sound source based on the incident angle;
A voice detection unit that detects only voice information in which the directionality is determined from voice information collected by the microphone;
A voice detection device comprising: A Fourier transform unit for Fourier transforming the sound information collected by each of the plurality of microphones;
The sound direction information calculation unit calculates the incident angle for each frequency based on the phase difference of each frequency obtained by the Fourier transform,
The sound direction determining unit determines the direction for each frequency,
The voice detection device according to claim 1, wherein the voice detection unit detects only voice information of a frequency for which it is determined that directionality is determined for voice information collected by the microphone.   The sound detection device according to claim 2, wherein the sound direction determination unit determines whether or not the directionality is determined based on a temporal change in the incident angle with respect to a sound having a specific frequency. .   The sound detection device according to claim 2, wherein the sound directionality determination unit determines whether or not the directionality is determined based on a degree of coincidence of incident angles with respect to sound having a plurality of frequencies. Sound direction information calculation step for calculating the incident angle of the sound to the microphone based on the phase difference of the sound information collected by a plurality of microphones arranged apart from each other;
A sound directionality determining step for determining the directionality of the sound emitted from the sound source based on the incident angle;
A voice detection step for detecting only voice information for which the directionality is determined from voice information collected by the microphone;
A voice detection method comprising: A step of Fourier transforming sound information collected by each of the plurality of microphones;
In the sound direction information calculation step, the incident angle is calculated for each frequency based on the phase difference of each frequency obtained by the Fourier transform,
In the sound directionality determination step, the directionality is determined for each frequency,
The voice detection method according to claim 5, wherein, in the voice detection step, only voice information having a frequency at which it is determined that directionality is determined is detected from voice information collected by the microphone.   The voice detection device according to claim 6, wherein in the sound directionality determination step, it is determined whether or not the directionality is determined based on a temporal change in the incident angle for a sound having a specific frequency. .   The sound detection device according to claim 6, wherein in the sound directionality determination step, it is determined whether or not the directionality is determined based on a degree of coincidence of incident angles for a plurality of frequencies of sound. A plurality of microphones spaced apart from each other;
A sound direction information calculation unit that calculates an incident angle of sound to the microphone based on a phase difference of sound information collected by the microphone;
A sound directionality determining unit that determines the directionality of the sound emitted from the sound source based on the incident angle;
A voice detection unit that detects only voice information in which the directionality is determined from voice information collected by the microphone;
Based on the detection result by the voice detection unit, a tracking camera that changes the shooting direction;
An imaging system comprising:

Images