JP2011066805A - Sound collection device and sound collection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound collection device and a sound collection method which can input desired voice in an excellent state even when two or more speakers simultaneously utter. <P>SOLUTION: The sound collection device is provided with: a directivity forming section 101 which forms sound collection directivity in two or more directions using two or more microphones MC1-MCm; a voice signal detection section 102 which detects presence/absence of a voice signal to be collected from the two or more directions; and an utterance selection 103 which executes a simultaneous selection function for simultaneously selecting voice signals from the two or more directions when the voice signals are simultaneously detected from the two or more directions. Thus, even when the two or more speakers simultaneously utter, the desired voice is input at the successful state. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a sound collection device and a sound collection method.

  In a system such as a TV / voice conference, a remote lecture, or an IP phone, a voice signal is input using a sound collection device such as a microphone and transmitted to a remote place. By the way, when an omnidirectional microphone is used as a sound collection device, there is a problem that ambient noise, echoes, chats, etc. are received along with the voice of the speaker to be received, making it difficult to hear the desired voice.

  In view of the above problems, Patent Documents 1 and 2 below form a sound collection directivity in two or more directions using two or more microphones, and collect a signal with a maximum level signal among two or more sound collection beams. A technique for limiting the sound collection direction on the assumption that a desired sound source exists in the direction of the sound beam is disclosed.

  Patent Document 1 describes that a direction of a sound collecting beam accompanied by a signal of a maximum level is detected, and a sound signal picked up with a directivity aiming in the direction is input as a speaker's voice. ing.

  Patent Document 2 describes that a sound collecting beam accompanied by a signal of the maximum level and a sound signal collected by a sound collecting beam adjacent to the beam are input as a speaker's sound.

JP 2003-304589 A JP 2007-13400 A

  All of the above methods limit the sound collection direction on the assumption that a speaker is present in the direction of the sound collection beam with the maximum level signal. However, assuming the use situation of a conference system or the like, the number of speakers is not limited to one, and two or more speakers often speak at the same time. Moreover, such a use situation is not limited to a conference system or the like, and is similarly assumed in a system such as a remote lecture or an IP phone.

  In this case, if only one speaker is picked up, the voices of other speakers are not input. On the other hand, if the voice is picked up without limiting the speakers, the voices of a plurality of speakers are mixed. It becomes difficult to hear the desired sound.

  Therefore, the present invention is intended to provide a sound collection device and a sound collection method that can input desired sound in a good state even when two or more speakers speak at the same time.

  According to an embodiment of the present invention, a directivity forming unit that forms sound collection directivity in two or more directions using two or more microphones, and presence / absence of an audio signal collected from two or more directions are determined. A sound collection unit comprising: an audio signal detection unit to detect; and an utterance selection unit that executes a simultaneous selection function of simultaneously selecting audio signals from two or more directions when audio signals are detected simultaneously from two or more directions. An apparatus is provided.

  According to this configuration, even when audio signals are detected simultaneously from two or more directions, audio signals from two or more directions can be selectively input. Thereby, even when two or more speakers speak at the same time, a desired voice can be input in a good state.

  In addition, when the speech selection unit performs the simultaneous selection function, the level ratio of other audio signals based on the audio signal detected at the maximum level among the audio signals detected simultaneously from two or more directions. Is less than a predetermined threshold, the direction of the audio signal detected at a level less than the predetermined threshold may not be selected.

  In addition, when the speech selection unit executes the simultaneous selection function and the audio signal is simultaneously detected from directions exceeding a predetermined number, the utterance selection unit uses omnidirectionality by using at least one of two or more microphones. Sound collection may be performed.

  Further, the utterance selection unit may select and execute either a simultaneous selection function for simultaneously selecting audio signals from two or more directions or a priority selection function for selecting audio signals from one priority direction. Good.

  In addition, when the speech selection unit executes the priority selection function, the direction of the voice signal detected most first among the voice signals detected simultaneously from two or more directions in accordance with a user instruction, One priority direction may be selected.

  In addition, when executing the priority selection function, the utterance selection unit determines the direction of the audio signal detected at the maximum level from among the audio signals detected simultaneously from two or more directions according to a user instruction. May be selected as the preferred direction.

  Further, the utterance selection unit may select either the simultaneous selection function or the priority selection function according to a user instruction.

  According to another embodiment of the present invention, the step of forming sound collection directivity in two or more directions using two or more microphones and the presence or absence of an audio signal collected from two or more directions are determined. And a step of performing a simultaneous selection function of simultaneously selecting audio signals from two or more directions when audio signals are detected simultaneously from two or more directions. .

  According to the present invention, it is possible to provide a sound collection device and a sound collection method capable of inputting desired sound in a good state even when two or more speakers speak at the same time.

It is a block diagram which shows the structure of the sound collection device which concerns on one Embodiment of this invention. It is a figure which shows the basic principle of beam forming. It is a figure (1/2) explaining the detail of a sound collection device. It is a figure (2/2) explaining the detail of a sound-collecting apparatus. It is a flowchart which shows the sound collection method which concerns on one Embodiment of this invention. It is a figure (1/3) explaining the process at the time of a priority selection operation | movement. It is a figure (2/3) explaining the process at the time of a priority selection operation | movement. It is a figure (3/3) explaining the process at the time of a priority selection operation | movement. It is a figure (1/3) explaining the process at the time of simultaneous selection operation | movement. It is a figure (2/3) explaining the process at the time of simultaneous selection operation | movement. It is a figure (3/3) explaining the process at the time of simultaneous selection operation | movement.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[1. Sound collection device]
Hereinafter, a sound collecting apparatus according to an embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing a configuration of a sound collection device according to an embodiment of the present invention. FIG. 2 is a diagram showing the basic principle of beam forming. 3 and 4 are diagrams for explaining the details of the sound collecting device.

  The sound collection device includes a microphone array composed of m microphones 100-1 to 100-m (MC1 to MCm), a directivity forming unit 101, an audio signal detection unit 102, an utterance selection unit 103, an operation unit 104, a selector 105, A mixer 106 is included. The sound collection device performs predetermined signal processing by the directivity forming unit 101 on signals input from two or more microphones MC1 to MCm arranged in a predetermined arrangement, and forms sound collection directivity in an arbitrary direction. To do.

  In this embodiment, in order to detect the direction of the speaker S, the sound collection directivity is formed using the principle of beam forming. If the principle of beam forming is used, the sound collection directivity can be formed in an arbitrary direction using two or more omnidirectional microphones. In the following, it is assumed that the sound collection direction is divided in the J direction.

  Note that the sound collection directivity is obtained by, for example, preparing J directional microphones 100-1 to 100 -J and arranging the microphones 100-1 to 100 -J in the center direction of each arc obtained by equally dividing the circumference into J. You may form by directing and directing.

FIG. 2 shows a case where sound waves (plane waves) arriving from the θ direction are received by two microphones MC1 and MC2 arranged at a distance l. A sound wave coming from the θ direction is received by the microphone MC1, and then propagated by a distance d and received by the microphone MC2. Here, the distance d is expressed by the following equation.
d = lsin θ (1)

Thus, the received signal x2 of the microphone MC2 (t) is different from the received signal _x 1 of the microphone MC1 (t), the time difference required for the propagation waves of the distance d (delay amount) becomes delayed signal in tau. Here, the received signal x ₂ (t) and the delay amount τ are expressed by the following equations.
x ₂ (t) = x ₁ (t−τ) (2)
τ = d / c (c: speed of sound) (3)

Therefore, when the delay amount τ is added to the received signal x ₁ (t) and added to the received signal x ₂ (t), the added signal b (t) obtained by adding the signals having the same phase comes from the specific direction θ. The sound wave (the amplitude) is emphasized. If the sign of θ is reversed, a delay amount τ is added to the received signal x ₂ (t).
b (t) = x ₂ (t) + x ₁ (t−τ) (4)

Directivity formation based on the above principle can be performed in the frequency domain as well as in the time domain. The Fourier transform of the signal with the delay amount τ on the time axis is ^obtained by multiplying the Fourier transform of the original signal by e ^−jωτ . Therefore, if the Fourier transform of the addition signal b (t) and the reception signals x ₁ (t), x ₂ (t) is B (ω), X ₁ (ω), X ₂ (ω), the addition on the time axis The signal b (t) is expressed by the following equation on the frequency axis.
B (ω) = X ₂ (ω) + e ^−jωτ X ₁ (ω) (5)

  Here, on the time axis, when digital processing is performed, the delay amount τ can be selected only at the sampling cycle interval. On the other hand, since the delay amount τ can be arbitrarily selected on the frequency axis, the pointing direction can be arbitrarily set by changing the delay amount τ.

  FIG. 3 shows a configuration of the directivity forming unit 101 that performs directivity formation in the frequency domain. The directivity forming unit 101 includes time-frequency conversion units 301-1 to 301-m (also referred to as FFT (Fast Fourier Transform) units) using a fast Fourier transform, a delay control unit 302, and multiplication units 303-1 to 303. -M, including an adder 304.

Audio signals x ₁ (t) to x _m (t) (see FIG. 4) collected by m microphones MC1 to MCm arranged linearly at a distance of l are A / D (not shown). The signal is converted into a digital signal by the converter and supplied to the directivity forming unit 101. The microphones MC2, MC3,..., MCm have arrival time differences τ, 2τ,..., (M−1) τ with respect to the microphone MC1 due to the sound source direction θ.

Therefore, the arrival time difference τ, 2τ, ···, (m -1) received signal a delay amount corresponding to _{τ x m-1 (t)} , x m-2 (t), ···, x 1 (t ), All signals are in phase. By adding the in-phase signals, only the voice coming from the θ direction is enhanced. And if the signal added in proportion to the increase in the microphone MC increases, the gain in the directivity direction increases.

The directivity forming unit 101 realizes the above principle in the frequency domain. The received signals x ₁ (t), x ₂ (t),..., X _m (t) are converted into spectra X ₁ (ω), X ₂ (ω),. .., X _m (ω), multiplied by a delay coefficient by the multipliers 303-1 to 303-m, and a delay is added.

The delay coefficient is supplied to the multipliers 303-1 to 303-m-1 by the delay controller 302 according to the direction of directivity. Assuming a time difference τ caused by the distance difference d, the delay coefficients are e ^{−jω (m−1) τ} , e ^{−jω (m−2) τ} ,..., E ^−jωτ . If the sign of θ is reversed, since the microphone MCm is closest to the sound source, the maximum delay amount (m−1) τ is added to the spectrum X _m (ω).

The delay control unit 302 controls the delay amount so as to form uniform directivity around the microphone array. Thereby, the collected sound signal spectrums B ₁ (ω), B ₂ (ω),..., B _J (ω) from each direction are obtained.

B ₀ (ω) shown in FIG. 3 is a direct output of a collected signal spectrum of an arbitrary microphone MC, and is a non-directional (omnidirectional) signal having no directivity. In FIG. 3, the spectrum X ₁ (ω) of the received signal x ₁ (t) is output as B ₀ (ω), but the received signal of another microphone MC may be output.

  Returning to FIG. 1, the audio signal detection unit 102 detects the presence or absence of an audio signal from the collected sound signals from each direction. The sound detection is performed using a known method capable of detecting the presence or absence of a sound signal in the collected sound signal. In voice detection, for example, the start of voice input is determined when a received signal of a predetermined level or higher continues for a predetermined time or more with reference to the signal level. The end of is determined. The audio signal detection unit 102 detects the presence or absence of an audio signal for the collected signals in all directions, and supplies the detection result and the signal level (level information) to the utterance selection unit 103.

  The utterance selection unit 103 simultaneously selects an audio signal from two or more directions simultaneously when an audio signal is detected from two or more directions, or a priority selection to select audio from one priority direction. Perform the action. The user can specify the simultaneous selection operation or the priority selection operation as the operation mode. The operation mode is designated by the user via a dip switch or the like of the operation unit 104 and is notified to the utterance selection unit 103.

[2. Sound collection method]
Hereinafter, a sound collection method according to an embodiment of the present invention will be described with reference to FIGS. FIG. 5 is a flowchart showing a sound collection method according to an embodiment of the present invention. FIGS. 6A to 6C and FIGS. 7A to 7C are diagrams for explaining processing of the utterance selection unit 103 during the priority selection operation and the simultaneous selection operation, respectively.

  First, the utterance selection unit 103 evaluates the number of directions in which a voice signal is detected (sound detection number n) (step S301). When the number of detected voices n = 0, a signal having directivity in a specific direction has not been detected, and therefore an omnidirectional (omnidirectional) signal is selected and omnidirectional recording is performed (S309). When the number of detected voices n = 1, the direction in which the voice signal is detected is selected (S302).

  On the other hand, when the number of detected voices n> 1, the utterance selection unit 103 confirms the operation mode designation status (S303), and selects the simultaneous selection operation or the priority selection operation. Then, when the priority selection operation is selected, the selection criterion for the priority direction is selected (S304). As in the operation mode, the selection criterion is designated by the user via the operation unit 104 and notified to the utterance selection unit 103.

  In the first selection criterion, the direction of the audio signal detected most precedingly is selected from the audio signals detected simultaneously from two or more directions. In this case, as long as the voice of a predetermined level or higher is collected, the voice of the preceding priority speaker Sp can be continuously collected regardless of the level of the voice signal of the subsequent speaker S ′. The voice signal of the subsequent speaker S ′ cannot be selected.

  In the second selection criterion, the direction of the audio signal detected at the maximum level is selected from the audio signals detected simultaneously from two or more directions. In this case, if the level is higher than the voice signal of the preceding priority speaker Sp, the voice signal of the subsequent speaker S ′ can be selected, but the voice of the preceding speaker S can be continuously collected. Can not.

  When the first selection criterion is selected, the speaker S in the direction in which the voice signal is detected in advance is set as the priority speaker Sp, and the direction is selected until voice detection is not performed (S305). ). On the other hand, when the second selection criterion is selected, the direction of the voice signal detected at the maximum level is changed with the speaker S in the direction of the voice detected at the maximum level as the priority speaker Sp. This direction is selected until it is determined (S306).

  6A to 6C show processing during the priority selection operation based on the selection criterion. In FIG. 6A, the voice of the speaker S1 (priority speaker Sp) is received by the sound collection beam B1. Here, it is assumed that the speaker S2 starts speaking at a louder volume than the speaker S1. As shown in FIG. 6B, if the first selection criterion is selected, the sound collection of the speech signal of the preceding speaker S1 is given priority with the speaker S1 as the priority speaker Sp, and the speech is collected by the sound collection beam B1. Reception of the voice signal of the person S1 is continued. On the other hand, as shown in FIG. 6C, if the second selection criterion is selected, the voice signal of the louder speaker S2 is picked up with the speaker S2 as the priority speaker Sp instead of the speaker S1. Priority is given and reception of the voice signal of the speaker S2 is started by the sound collection beam B2 instead of the sound collection beam B1.

  When the simultaneous selection operation is selected in the process of step S303, the sound level in each direction in which the sound signal is detected is evaluated, and the number of directions satisfying a predetermined criterion is obtained as the sound detection number n ′ (S307). ). Here, the audio signal in each direction is determined to satisfy a predetermined criterion when accompanied by a level equal to or higher than a predetermined ratio with respect to the audio signal of the maximum level.

Then, it is determined whether the number of detected voices n ′ exceeds a predetermined threshold value n _max (S308). When audio signals are detected simultaneously from a large number of directions, all applicable directions may be selected. However, if the number of directions increases, sound signals from a specific direction are collected by directivity formation. The significance will fade. That is, when the sound collection direction is narrowed, it is easy to hear the utterance, but the atmosphere accompanying the environmental sound other than the sound collection direction is difficult to be transmitted. And if the number of simultaneous utterances increases too much, it will be difficult to hear the utterances and the atmosphere will be difficult to convey. Therefore, when the number of simultaneous utterances increases too much, it is desirable to switch to an omnidirectional microphone so that the atmosphere can be easily transmitted.

For this reason, in this embodiment, the threshold value n _max of the number of directions that can be selected simultaneously is set. When the number of detected voices n ′ exceeds the threshold n _max , a non-directional (omnidirectional) signal is selected without selecting a specific direction, and omnidirectional recording is performed (S309).

On the other hand, when the number of detected voices n ′ is equal to or smaller than the threshold value n _max , directions that satisfy the above-described predetermined criterion are simultaneously selected (S310). This is because when the level difference between detected signals is large, it is desirable not to select the direction of an audio signal with a level less than a predetermined ratio with respect to the audio signal at the maximum level in order to avoid interference. Because.

7A to 7C show processing at the time of the simultaneous selection operation based on the number of detected voices n ′. In FIG. 7A, the voice signals of the speakers S1 to S4 are received by the sound collecting beams B1 to B4. Note that the threshold n _{max for the} number of detected voices n ′ is set to 4. Here, as shown in FIG. 7B, the voice signals of the speakers S2 to S3 are less than a predetermined value based on the voice signal of the speaker S1 detected at the maximum level among the voice signals of the speakers S1 to S4. When the level is accompanied, the voice signal of the speaker S1 is received by the sound collection beam B1 instead of the sound collection beams B1 to B4.

Also, as shown in FIG. 7C, when the speaker S5 starts speaking, the number of detected voices n ′ (= 5) exceeds the threshold value n _max (= 4), so that sound is collected instead of the sound collecting beams B1 to B4. Omnidirectional recording is performed by the beam B0. In FIG. 7C, the sound collection beam B0 is shown only on the front surface of the microphones MC1 to MCm, but it is also formed on the side surface and the back surface.

  Then, the utterance selection unit 103 notifies the selector 105 of the selected directional direction as selection information of the directional direction (S311). The selector 105 extracts a signal in a direction to be selected from the output signal of the directivity forming unit 101 based on the selection information and supplies the extracted signal to the mixer 106. The mixer 106 mixes the supplied signal and outputs it as a transmission audio signal.

[3. Summary]
As described above, according to the sound collection device and the sound collection method according to the above embodiment, even when audio signals are detected simultaneously from two or more directions, audio signals from two or more directions are selectively input. can do. Thereby, even when two or more speakers S speak at the same time, a desired voice can be input in a good state.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 100 Microphone 101 Directionality formation part 102 Audio | voice signal detection part 103 Speech selection part 104 Operation part 105 Selector 106 Mixer 301 Time-frequency conversion part (FFT part)
302 delay control unit 303 multiplication unit 304 addition unit

Claims (8)

A directivity forming unit that forms sound collection directivity in two or more directions using two or more microphones;
An audio signal detector that detects the presence or absence of an audio signal collected from the two or more directions;
An utterance selection unit that executes a simultaneous selection function of simultaneously selecting audio signals from the two or more directions when audio signals are detected simultaneously from the two or more directions;
A sound collecting device.   The speech selection unit, when executing the simultaneous selection function, a level ratio of other audio signals based on an audio signal detected at a maximum level among audio signals detected simultaneously from the two or more directions. The sound collection device according to claim 2, wherein the direction of an audio signal detected at a level less than the predetermined threshold is not selected when is less than the predetermined threshold.   The speech selection unit uses the at least one of the two or more microphones to be omnidirectional when audio signals are simultaneously detected from directions exceeding a predetermined number when executing the simultaneous selection function. The sound collection device according to claim 1, wherein sound collection is performed.   The utterance selection unit selects and executes either a simultaneous selection function for simultaneously selecting voice signals from the two or more directions or a priority selection function for selecting voice signals from one priority direction. The sound collection apparatus in any one of 1-3.   When executing the priority selection function, the utterance selection unit, according to a user instruction, the direction of the audio signal that is detected most first among the audio signals detected simultaneously from the two or more directions, The sound collecting device according to claim 4, wherein the sound collecting device is selected as the one priority direction.   When executing the priority selection function, the utterance selection unit determines the direction of the audio signal detected at the maximum level among the audio signals detected simultaneously from the two or more directions according to a user instruction. The sound collecting device according to claim 4, wherein the sound collecting device is selected as one priority direction.   The sound collection device according to claim 4, wherein the utterance selection unit selects either the simultaneous selection function or the priority selection function in accordance with a user instruction. Forming sound collection directivity in two or more directions using two or more microphones;
Detecting the presence or absence of an audio signal collected from the two or more directions;
Executing a simultaneous selection function of simultaneously selecting audio signals from the two or more directions when audio signals are detected simultaneously from the two or more directions;
Including sound collection method.

Images