JP6375475B1 - Sound source direction tracking system - Google Patents

Abstract

Provided is a system for automatically following a front direction of a system including a microphone and a camera to a sound source direction by a rotation mechanism in a video conference system, a robot for assisting a person, and the like.
A system that automatically follows the direction of a sound source detects the advance / delay by detecting a phase difference with a wide band accompanying a time difference between two microphone signals in order to make the front direction of the device follow the direction of the sound source. Then, the rotation is controlled in a direction in which the advance delay is zero by feedback control. In that case, it is roughly detected whether the front direction of the rotated device is located near the sound source direction by rotation control, and if it is located near the sound source direction, a microphone system signal output having a directivity characteristic of a fixed distance arrangement To determine the advance / delay and perform rotation control within a limited angle.If it is not in the vicinity of the sound source direction, use an omnidirectional microphone signal placed at a fixed distance to determine the advance / delay and perform omnidirectional rotation control. Follow the sound source direction.
[Selection] Figure 1

Description

  The present invention is a technology applicable to a video conference system, a system that directs a camera viewpoint and a microphone directivity in a speaker direction with a robot that assists a person. The present invention relates to a method for eliminating the influence of reflected waves in order to direct a system including a microphone to a sound source direction quickly and accurately.

Since the voice has a wide specific band, it is necessary to detect the direction of voice arrival in a wide band. As a conventional technique, there is a method of performing wideband detection by performing frequency component decomposition synthesis (FFT / IFFT), but processing time is required for detection, and the speed of control is impaired. On the other hand, two omnidirectional microphones are arranged close to each other by about several centimeters, the two signals are subtracted and then integrated, and the first signal and the second signal obtained by adding the two signals are multiplied. Japanese Patent Application Laid-Open No. 2004-133867 discloses a system that follows a 360 ° sound wave arrival direction by performing rotation control using a signal.
However, in the method of Patent Document 1, if there is a reflected wave, the sound source arrival direction detection is not correctly detected due to the influence of the reflected wave, and a control error occurs. When the sound source arrival direction detection is controlled by taking a long time average of the detection signal, the error becomes small, but a quick response cannot be made.

Japanese Patent Application Laid-Open No. 2004-228561 shows a method for making the sound easy to hear even when it deviates from the assumed sound source position. However, since rotation control is not performed, the applicable sound source angle range is limited.
In addition, there is a problem of being affected by reflected waves that cannot be achieved quickly.

  Patent Document 3 discloses a device that directs a camera in the direction of a sound source, but uses a means for obtaining sound intensity, and there is a problem of being affected by a reflected wave that cannot be achieved quickly.

Sound source direction tracking system JP 2016-201595 Applicant Tokiko Inoue Acoustic input device JP2009-135594 Applicant Panasonic Electric Works Co., Ltd. Sound source exploration device Japanese Patent Application No. 63-168011 Applicant Toshiba Corporation

  In a system that can automatically follow the sound source direction with the rotation mechanism, the front direction of the system including the microphone avoids the influence of the reflected wave even in the environment where the reflected wave from the wall and window is generated, and quickly and accurately follows the sound source direction. It is a problem to make it.

First, the control error due to the influence of the reflected wave is quantitatively analyzed.
FIG. 2 shows a configuration for detecting the direction in which the front direction of the system is shifted to the left or right with respect to the sound source direction. When the microphone system angle (θe) is changed with respect to the sound wave arrival direction, the arrival time from the sound source changes, and the phase also changes as shown in Equations 1 and 3. When the signal is subtracted, integrated and multiplied, Vdet ′ is obtained (Expression 8), and a value corresponding to the phase difference between the microphones A and B is output. When φb = φa, the output is zero.When φb> φa, a positive value is output. When φb <φa, a negative value is output. By stopping, it follows the direction of the sound source.

Microphone A point received signal: Ea
Amplitude: V Arrival time from sound source: Ta
In the vector notation showing the phase in Equation 1, it becomes Equation 2.

Microphone A point received signal: Eb
Amplitude: V Arrival time from sound source: Tb
In the vector notation showing the phase in Equation 3, Equation 4 is obtained.

As shown in Expression 8, the configuration of FIG. 2 can obtain an output corresponding to the phase difference between φb and φa, and is rotationally controlled in the direction of φb = φa.

FIG. 3, which analyzes the control error when there is a reflected wave, shows a model when there is a reflected wave.
fa: Sound source frequency
τu: Difference in propagation time of reflected wave with respect to direct wave
Kdu: Reflected wave level / direct wave level θu: Reflected wave arrival angle (direct wave reference)
θe: Microphone system rotation angle
d: Distance between microphones c: Sonic velocity Tm = d / c

When analyzing the influence of the reflected wave, it is necessary to analyze from the viewpoint of the phase difference between the direct wave and the reflected wave with respect to the frequency.
In FIG. 3, when the sound source frequency is fa and the delay time of the direct wave and the reflected wave is τu, the phase shift Δθ due to the delay is

Since the rotation control is controlled in the direction of φa = φb as shown in Expression 8, the control error can be calculated by calculating the difference between θa and θb with respect to θe by numerical calculation.
4 and 5 show the phase difference (θb−θa) between microphones A and B when the microphone tilt angle (θe) is varied from −180 ° to 180 ° with the frequency as a parameter (d = 4 cm).
FIG. 4 shows the case where the reflected wave level is zero, but the phase difference becomes zero in the front direction of the system (θe = 0) regardless of the sound source frequency.
However, when there is a reflected wave, the reflected wave level is 0.5 (-6 dB), the delay time is 2 mSec, and θu = 90 °, a large phase difference depending on the sound source frequency occurs as shown in FIG.

Next, numerical calculation results of the control angle error (θe) in the case where the delay time is variable and the case where the reflected wave level is variable are shown.
FIG. 6 shows the angle error when the delay times are different. Corresponding to the sound source frequency change, the change of the angle error repeats, but its frequency cycle is “1 / delay time”, and if the delay time is long, the frequency cycle becomes short.
FIG. 7 shows a control error when the reflected wave level (Kdu) with respect to the direct wave is varied. When the reflected wave level is about 0.1 (-20dB), the error is about ± 6 °, and the frequency changes evenly in the plus and minus directions depending on the frequency. Therefore, when the input frequency changes with time, control with a time average is used. The error can be reduced. However, when KdU increases to 0.5 (−6 dB), the maximum error increases to about 25 °, and the probability of the maximum control error increases in the case of rapid control. Note that the maximum control error angle with respect to Kdu is almost directly proportional.
FIG. 8 shows the maximum control error with respect to the reflected wave arrival angle (θu) with respect to the desired wave at Kdu = 0.5 (−6 dB). When the front direction (θu = 0 °) and the rear direction (θu = 180 °), the error is small and the horizontal direction is large.

FIG. 1 shows the overall configuration of the solving means. Basically, the block 30 determines the advance / delay of the phase difference that has been widened due to the time difference between the two microphone signals 10A, 10B, 10C, and 10D arranged at a distance, and detects the arrival direction of the sound wave. .
Here, the determination of the advance / delay of the phase difference with a wider band is an example of the configuration shown in FIG. 2, and when there is no reflected wave, the phase difference even if the sound source frequency changes as shown in FIG. This means that the detection characteristics are not substantially changed. In particular, it is used that the plus / minus polarity of the phase difference between microphones when θe is variable does not depend on the frequency.
Further, in the case of quick control, it is determined in block 60 whether the front direction of the system is in the vicinity of the sound source direction, and the directivity of the two microphone systems is switched between non-directional and directional with the switches 22A and 22B. However, the control error due to the influence of the reflected wave is reduced.
When the front direction of the system is not near the sound source direction, rotation control in the range of maximum ± 180 ° is necessary, and the direction of arrival cannot be detected unless it is an omnidirectional microphone system. However, when the system front direction is within ± 60 ° of the sound source direction, it becomes possible to detect the direction of arrival by using two microphone systems having directivity near the system front direction. The rotation control error due to the reflected wave can be reduced.

FIG. 9 shows the numerical simulation results of the control error characteristics due to the influence of the reflected wave when the directivity formation is used together (with BF) and when it is not used (without BF).
Here, the reflected wave level (Kdu) based on the desired wave level is 0.5 (−6 dB), and the error (worst value) is plotted at the frequency where the sound source frequency is varied and the error becomes the largest.
This graph is an evaluation at the worst frequency, and depends on the reflected wave level, but does not depend on the delay time. By using the directivity formation together, the control error can be reduced. In particular, the error with respect to an angle other than the reflected wave arrival angle of ± 45 ° is greatly improved. In many cases, the reflected wave level within an arrival angle of ± 45 ° (reflected wave from the same direction) is small, and the error is greatly improved as a whole.
Due to this characteristic of the present invention, a system in which the front direction of the system automatically follows the direction of speech (sound source direction) by a rotation mechanism, and a speaker in another location starts speaking even when there is a reflected wave. Swift and accurate rotation control is possible.

Selection diagram Overall configuration of the present invention Sound source direction detection basic method Reflected wave effect model Phase difference between microphones when the reflected wave level is zero Phase difference between microphones when reflected wave level is 0.5 (-6dB) and delay time is 2mSec Control angle error for variable delay time of reflected wave Control error for <reflected wave level / direct wave level> Control angle error for reflected wave arrival angle Change in control error when there is no directivity formation (figure showing the effect of the present invention) Example 1: Specific overall configuration Example 1: Directionality formation method Figure 11 directivity characteristics Example 2: Rough sound wave arrival direction detection method Example 3: Microphone signal output method combining directional signals

Next, Examples 1 to 3 listed below are modes for carrying out the invention.

(Example 1)
FIG. 10 shows a first embodiment of the present invention.
Four microphones consisting of 10A, 10B, 10C, and 10D are arranged at the four corners of a plane quadrangle with a side of 4 cm. At the time of omnidirectional detection, control is performed by an omnidirectional microphone. In this case, a 10A omnidirectional microphone A (VomL) and a 10B microphone B (VomR) are used. It is detected by rotation control that the front direction of the system is close to the sound source direction, and in that case, the directivity forming signal is switched. In this case, the directivity is set in the block 20A with the microphone C (10C) and microphone A (10A) pair. The formed signal (VbfL) and the signal (VbfR) having the directivity formed in the block 20B by the microphone D (10D) and microphone B (10B) pair are used. VbfL and VbfR correspond to sound reception at a distance d away, and a wide-band phase difference is generated according to the sound source arrival angle, so that the arrival direction can be detected. The microphone output signals detected by the block 60 to determine whether or not the sound wave arrival direction is the front direction of the system and switched by the switches 22A and 22B are ViL and ViR. Input and detect direction of arrival. In that case, the amplitude of the reflected wave arriving from an angle away from the front direction of the system is attenuated by the directivity forming effect, and an angle control error hardly occurs even in the case of a quick response.

A directivity forming embodiment is shown in FIG.
This is a method based on the subtraction / integration method.
The microphone signal composed of 1A and 1B is subtracted, multiplied by an integral coefficient, and added to form directivity.
The directivity function D (θ) of the method of FIG. 11 is as follows.
k = ω / c c: sound velocity ω: sound source angular frequency
d: When the distance between microphones

here

Assuming
D (θ) can be approximated as

Therefore, the directivity can be reversed depending on whether the coefficient Ka is positive or negative.
When Ka is positive: Beam formation in 180 ° direction
When Ka is negative: An example of numerical calculation of beamforming directivity in the 0 ° direction is shown in FIG.

(Example 2)
An embodiment for detecting the arrival of a rough sound wave will be described. This is logic that performs 4-directional directivity formation with 3 microphones and detects that the amplitude value in the front direction of the system is larger than the amplitude in the other direction (FIG. 13).
The three-mic signal is generated by branching from the signals of the microphone A, microphone B, and microphone C in FIG. An example of means for realizing directivity formation is the same as that shown in FIG.
The fact that the directivity direction can be reversed by changing the Ka value of block 4 is utilized. The microphone A (10A) and the microphone C (10C) form a directivity in the La direction Lc direction, and the microphone A (10A) and the microphone B (10B) form a directivity in the Lb direction Ld direction. This is compared in amplitude at block 60 to detect whether the front of the system is near the sound source direction.
There is also a method to detect that sound waves are coming from the front by comparing the level of the directional microphone signal with the omnidirectional signal, but when using the omnidirectional signal, it is fixed due to the influence of the reflected wave. Although a wave may occur and cannot be detected correctly, this method can detect it correctly.

(Example 3)
The output of the front directional microphone system used for sound source direction tracking and arranged at a distance from the left and right is a signal in which the influence of the reflected wave is reduced, and the sound is easy to hear. At that time, by synthesizing two directional signals formed by 20A and 20B in the block 70, it is possible to make the sound easier to hear. Even in the case of simple addition, quantitatively, in the case of uncorrelated noise, the noise level is doubled by synthesis and the signal level is doubled, so that the signal level / noise level can be improved by 1.4 times (3 dB).
FIG. 14 shows the configuration.

(1) In a video conference system, point the camera at the speaker and show the speaker's face to the other party.
Improve S / N of audio signal by directivity formation.
(2) A human interface that assists humans with the robot's face facing the speaker.
(3) Provide the face image of the speaker to the hearing impaired and recognize the movement of the face and mouth.
(4) Record the sound direction image and sound in the crime prevention system.

1A: Method-oriented omnidirectional microphone A that achieves directivity with two omnidirectional microphones
1B: An omnidirectional microphone B for explaining a method for realizing directivity formation by two omnidirectional microphones
2: Subtractor for directivity formation
3: Integrator for directivity formation
4: Directivity forming coefficient unit (multiply by Ka value)
5: Directivity forming adder 10: Four omnidirectional microphones arranged in a plane
10A: Omnidirectional microphone A
10B: Omnidirectional microphone B
10C: Omnidirectional microphone C
10D: Omnidirectional microphone D
22A: Directivity switching switch A
22B: Directivity switching switch B
30: Sound source direction detection block (broadband phase difference detection)
30A: Binarization processing A for sound source direction detection block, the purpose is to extract only the instantaneous maximum amplitude component
30B: Binarization processing B for sound source direction detection block, the purpose is to extract only the instantaneous maximum amplitude component
40: Subtractor for sound source direction detection block
41: Adder for sound source direction detection block
42: Integrator for sound source direction detection block
43: Multiplier for sound source direction detection block
50: Direction-of-arrival detection averaging unit
51: Motor drive unit for sound source direction tracking
52: Motor for sound source direction tracking
60: Sound source arrival direction detector near the front of the system
61: Directional changeover switch controller

Claims (3)

The front direction of the equipment to follow the sound source direction with broadband frequency component, 2 to determine the broadband the phase difference detection result lead lag due to the time difference between the microphone signal of the system, the more leading direction delays is eliminated in the feedback control In a rotation control system, when the front direction of the device is positioned in the vicinity of the sound source direction by means of roughly detecting whether or not the front direction of the device is positioned in the vicinity of the sound source direction by rotation control, that is, approximately within ± 60 ° with respect to the sound source direction. Performs rotation control using a microphone system signal output with directivity characteristics in which the sound receiving position is located a certain distance away, and if it is not near the sound source direction, the microphone position is omnidirectionally located a certain distance away Sound source direction tracking system that performs rotation control using a signal from a directional microphone. A multi-directional directivity is formed from microphone signals arranged in the front-rear direction and the left-right direction of the plane, and the received signal amplitudes in the multi-directional directions are compared. The sound source direction tracking system according to claim 1, wherein when the amplitude is higher than any of the amplitudes, the device front direction is determined to be in the vicinity of the sound source direction . The sound source direction tracking system according to claim 1, wherein the signal output of a microphone system having directivity characteristics in which sound receiving positions are separated by a certain distance is synthesized to obtain a signal in which reflected waves and noise are suppressed.

Images