JP6361360B2 - Reverberation judgment device and program - Google Patents

Description

  The present invention relates to a reverberation determination device and a program for determining whether or not there is a reverberation component in a sound signal acquired by a microphone (a sound signal or an acoustic signal is collectively referred to as a “sound signal”). The present invention can be applied to a portable voice communication terminal such as a smartphone.

  Because of the portability of portable voice communication terminals, the environment in which voice communication is executed is not unique, and depending on the usage environment, the reverberation component is included in the sound signal obtained by the microphone. There is also. For example, a voice that a speaker utters to a voice communication terminal in a sealed space such as a conference room or a passenger car (hereinafter sometimes referred to as a target sound) not only reaches the microphone linearly from the speaker's mouth, After coming out of the speaker's mouth, it may be reflected one or more times on the wall, ceiling, etc., and reach the microphone, which becomes a reverberant component.

  If there is a reverberation component, the sound quality of the voice signal used for voice recognition and voice communication deteriorates.

  Recently, there are an increasing number of portable voice communication terminals equipped with a voice recognition function and capable of handling voice input. When using a voice recognition function mounted on a portable voice communication terminal in an enclosed space, the voice recognition rate is reduced due to reverberation components.

  By the way, a reverberation removing apparatus that removes a reverberation component included in an audio signal or the like has already been proposed (see, for example, Patent Document 1).

JP 2013-171076 A JP 2013-1206026 A

  As described above, the portable voice communication terminal may be used in an environment where reverberation components do not occur, such as outdoors, if it is used in a sealed space where reverberation components occur such as in conference rooms and passenger cars. If anything, it is more often used in the latter environment.

  Installing a dereverberation device in a portable audio communication terminal used in such various environments is an installation under a situation where the case where the effect is exerted is limited, and the cost performance is low. Can be said to be complicated and expensive. Incidentally, Patent Document 1 intends a dereverberation device mounted on a conference device in which reverberation components are almost always a problem.

  Therefore, instead of mounting the dereverberation device, it is conceivable to determine whether or not there is a reverberation component and use the determination result in the voice processing function. For example, depending on the determination result, it is conceivable that the speech recognition function cannot be executed when a reverberation component occurs, and the speech recognition function can be executed when no reverberation component occurs. Further, for example, it is conceivable to switch the suppression gain of the noise suppression device (see Patent Document 2) depending on the presence or absence of a reverberation component.

  However, conventionally, no device has been proposed that only determines whether there is a reverberation component in the captured sound signal of the microphone.

  The presence or absence of a reverberation component can be determined by comparing the difference between signals before and after dereverberation in a dereverberation apparatus with a threshold or the like, or comparing the magnitude of a removal parameter with a threshold or the like. However, this is the same as mounting a complicated dereverberation device, and the meaning of mounting the dereverberation device is diminished.

  Therefore, a reverberation determination device and a program that can determine whether or not a sound signal has a reverberation component with a simple configuration or simple processing are desired.

  The reverberation determination device of the first aspect of the present invention is (1) generating a sound source azimuth suppression signal having a blind spot in the direction of the target sound source based on an input sound signal obtained by capturing ambient sounds by at least two microphones. And (2) a reverberation determination unit that determines the presence or absence of a reverberation component in the input sound signal by comparing a feature amount reflecting a time change of the generated sound source direction suppression signal with a threshold value. It is characterized by having.

  A reverberation determination program according to a second aspect of the present invention provides a computer that (1) suppresses a sound source direction having a blind spot in the direction of a target sound source based on an input sound signal obtained by capturing ambient sounds by at least two microphones. A sound source azimuth suppression signal generating means for generating a signal, and (2) comparing the feature value reflecting the time change of the generated sound source azimuth suppression signal with a threshold value to determine the presence or absence of a reverberation component in the input sound signal. It functions as a reverberation determination means.

  ADVANTAGE OF THE INVENTION According to this invention, the reverberation determination apparatus and program which can determine with a simple structure or simple process whether a reverberation component exists in a sound signal are realizable.

It is a block diagram which shows the structure of the reverberation determination apparatus which concerns on 1st Embodiment. It is explanatory drawing which shows the directivity of the front suppression signal which the front suppression signal production | generation part in the reverberation determination apparatus of 1st Embodiment produces | generates. It is a block diagram which shows the detailed structure of the reverberation determination part in the reverberation determination apparatus of 1st Embodiment. It is explanatory drawing which shows the behavior of the front suppression signal which the front suppression signal production | generation part in the reverberation determination apparatus of 1st Embodiment produced | generated. It is a flowchart which shows the process of the determination part in the reverberation determination part in the reverberation determination apparatus of 1st Embodiment. It is a block diagram which shows the structure of the reverberation determination apparatus which concerns on 2nd Embodiment.

(A) First Embodiment A reverberation determination apparatus and program according to a first embodiment of the present invention will be described below with reference to the drawings.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram illustrating a configuration of a reverberation determination device according to the first embodiment. Here, the part excluding the pair of microphones m1 and m2 can be configured by hardware, and can also be realized by software executed by the CPU (reverberation determination program) and the CPU. Whichever implementation method is employed, it can be functionally represented in FIG.

  In FIG. 1, the reverberation determination device 10 according to the first embodiment includes a microphone m <b> 1, a microphone m <b> 2, an FFT (Fast Fourier Transform) unit 11, a front suppression signal generation unit 12, and a reverberation determination unit 13.

  The pair of microphones m1 and m2 are arranged apart from each other by a predetermined distance (or an arbitrary distance), and each captures surrounding sounds. Each of the microphones m1 and m2 is omnidirectional (or has a very gentle directivity in the front direction). In the first embodiment, a speaker's mouth (sound source) is located in the front direction (normal direction) of a device (for example, a voice communication terminal) equipped with a reverberation determination device that is equidistant from both microphones m1 and m2. Yes, it is assumed that the target sound comes from the front. The sound signals (input sound signals) captured by the microphones m1 and m2 are converted into digital signals s1 (n) and s2 (n) via corresponding A / D converters (not shown) and given to the FFT unit 11. It is done. Note that n is an index indicating the input order of samples, and is expressed as a positive integer. In the text, it is assumed that the smaller n is the older input sample, and the larger n is the newer input sample.

The FFT unit 11 receives input signal sequences s1 (n) and s2 (n) from the microphones m1 and m2, and performs fast Fourier transform (or discrete Fourier transform) on the input signals s1 and s2. Thereby, the input signals s1 and s2 can be expressed in the frequency domain. In performing the Fast Fourier Transform, analysis frames FRAME1 (K) and FRAME2 (K) composed of predetermined N samples are configured and applied from the input signals s1 (n) and s2 (n). An example of constructing the analysis frame FRAME1 (K) from the input signal s1 (n) is shown in the following equation (1), and the analysis frame FRAME2 (K) is the same.

  K is an index indicating the order of frames and is expressed by a positive integer. In the text, it is assumed that the smaller the K, the older the analysis frame, and the larger, the newer the analysis frame. In the following description, it is assumed that the index representing the latest analysis frame to be analyzed is K unless otherwise specified.

The FFT unit 11 converts the frequency domain signals X1 (f, K) and X2 (f, K) into the frequency domain signals X1 (f, K) by performing a fast Fourier transform process for each analysis frame. And X2 (f, K) are supplied to the front suppression signal generator 12. Note that f is an index representing a frequency. X1 (f, K) is not a single value, but is composed of spectral components of a plurality of frequencies f1 to fm, as shown in equation (2). The same applies to X2 (f, K) and N (f, K) described later.

The front suppression signal generator 12 executes the calculation shown in equation (3) for each frequency to generate the frequency-specific front suppression signal N (f, K), and then executes the calculation shown in equation (4). Thus, the front suppression signal AVE_N (K), which is the average value of the frequency-specific front suppression signals N (f, K), is generated and given to the reverberation determination unit 13. Since the calculation of equation (3) subtracts without delaying any of the two X1 (f, K) and X2 (f, K) obtained from the captured signals of the pair of microphones m1 and m2, FIG. As shown in FIG. 2, this corresponds to a process of forming a directional signal having a blind spot in the front. The calculation formula of the frequency-specific front suppression signal N (f, K) is not limited to the formula (3), and the absolute value of the calculated value of the formula (3) is the frequency-specific front suppression signal N (f, K). You may do it.

  The reverberation determination unit 13 determines the presence or absence of a reverberation component based on the level of the front suppression signal AVE_N (K) and the degree of dispersion, and gives a determination result R (K) to a subsequent signal processing unit (not shown). .

  FIG. 3 is a block diagram illustrating a detailed configuration of the reverberation determination unit 13. The reverberation determination unit 13 includes a front suppression signal reception unit 21, a level / dispersion calculation unit 22, a determination unit 23, and a determination result transmission unit 24.

  The front suppression signal receiving unit 21 receives the front suppression signal AVE_N (K) calculated by the front suppression signal generation unit 12.

  The level / dispersion calculator 22 calculates the level Lv (K) and the variance Vr (K) of the front suppression signal AVE_N (K). For example, the level / dispersion calculation unit 22 may calculate an average value (may be a smoothed value), a square sum, or a square sum square root of the front suppression signals AVE_N (KM) to AVE_N (K) for a predetermined period immediately before. Are calculated as the level Lv (K) of the front suppression signal AVE_N (K). For example, the level / dispersion calculation unit 22 calculates the variance Vr (K) of the front suppression signals AVE_N (KM) to AVE_N (K) for the predetermined period immediately before. The level / variance calculation unit 22 may calculate a statistic representing another degree of variation such as a standard deviation or a variation coefficient instead of the variance.

  The determination unit 23 compares the level Lv (K) of the front suppression signal AVE_N (K) with the level threshold value Θ, compares the variance Vr (K) of the front suppression signal AVE_N (K) with the threshold value Ψ for variance, The presence or absence of a reverberation component is determined. The determination unit 23 determines that the reverberation component is “present” when the level Lv (K) is equal to or greater than the level threshold Θ and the variance Vr (K) of the front suppression signal AVE_N (K) is equal to or greater than the variance threshold Ψ. In other cases, it is determined that there is no reverberation component.

  The determination result transmission unit 24 outputs the determination result R (K) by the determination unit 23 to a subsequent signal processing unit (not shown).

(A-2) Reason for Using Frontal Suppression Signal Next, the reason for using the frontal suppression signal for the presence / absence determination of the reverberation component will be described.

  Since the reverberation component is generated by reflecting the target sound, the characteristics are similar to those of the direct sound directly transmitted from the sound source to the microphone. Therefore, it is quite difficult to analyze and detect a reverberation component based on a signal and its frequency component (whether it is detected by a time domain signal or a frequency domain signal).

  Therefore, from a different viewpoint, let's examine whether reverberation components can be recognized instead of analyzing the signal waveform.

  The reverberation component is a component generated by reflecting the target sound from the sound source. The direct sound that is not reflected comes from the front of the pair of microphones. On the other hand, since the reverberation component is reflected, the arrival direction to the pair of microphones is in all directions. The reverberation component reflected behind the speaker is blocked by the speaker and does not reach the microphone.

  Therefore, it seems that analysis focusing on the direction of arrival can be used.

  There is coherence as a parameter reflecting the correlation between the two directional signals by forming two directional signals having different directivity directions from the signals captured by the pair of microphones (see Patent Document 2). The reverberation component has a small time difference from the target sound. For this reason, the period of the target sound and the period of the reverberation component overlap, and it is difficult to detect the reverberation component based on coherence.

  The first embodiment assumes that the target sound arrives from the front of the microphone. The “frontal suppression signal” having directivity other than the front is one in which the target sound is suppressed even during a period in which there is the target sound uttered by the speaker. On the other hand, the value of the “frontal suppression signal” is large because reverberation from other than the front reaches when the speaker utters and there is reverberation. On the other hand, the value of the “frontal suppression signal” is small in the situation where there is no reverberation where there is no member to reflect even if the speaker utters and the reverberation does not reach. Furthermore, the “frontal suppression signal” naturally takes a small value in a situation where the speaker is not speaking.

  To summarize the above, the “frontal suppression signal” takes a large value only when there is a reverberation component, and when there is no reverberation component, including when no speaker is present, the “frontal suppression signal” Take a small value. That is, the presence or absence of a reverberation component can be detected based on the behavior of the “frontal suppression signal”.

  FIG. 4A shows a change in the front suppression signal generated based on the signal captured by the microphone in an environment where no reverberation occurs as if the speaker uttered in the sound absorbing room. FIG. 4B shows the inside of the passenger car. 6 shows a change in the front suppression signal generated based on the signal captured by the microphone in an environment in which reverberation occurs as if the speaker uttered. As can be seen from FIG. 4, the level and variance of the front suppression signal are significantly increased in an environment where reverberation occurs.

  Therefore, in the first embodiment, the presence / absence of a reverberation component is determined (detected) based on whether the level and variance of the front suppression signal reach a predetermined value.

(A-3) Operation of the First Embodiment Next, the operation of the reverberation determination device 10 of the first embodiment will be described in the order of the overall operation and the operation in the reverberation determination unit 13 with reference to the drawings.

  Signals s1 (n) and s2 (n) input from the pair of microphones m1 and m2 are respectively converted from time domain to frequency domain signals X1 (f, K) and X2 (f, K) by the FFT unit 11. To the front suppression signal generator 12.

  In the front suppression signal generation unit 12, the calculation shown in the equation (3) is executed for each frequency to generate the front suppression signal N (f, K) for each frequency, and further the calculation shown in the equation (4) is executed. Thus, the front suppression signal AVE_N (K) is generated and given to the reverberation determination unit 13.

  The reverberation determination unit 13 calculates the level and variance of the front suppression signal AVE_N (K), and compares the calculated level and variance of the front suppression signal AVE_N (K) with a threshold value. Then, when the two comparison results both indicate the presence of a reverberation component, a determination result R (K) indicating “there is a reverberation component” and in other cases “there is no reverberation component” is formed. To the signal processor.

  For example, if the subsequent signal processing unit is a noise suppression processing unit, the suppression gain to be applied is switched according to the presence or absence of a reverberation component. Further, for example, if the signal processing unit at the subsequent stage is a speech recognition unit, whether to perform speech recognition is switched according to the presence or absence of a reverberation component. Further, if the first speech recognition unit has a large processing load but a high recognition accuracy and the second speech recognition unit has a small processing load but a low recognition accuracy, the case of “with reverberation component” The first speech recognition unit may be operated, and the second speech recognition unit may be operated when “no reverberation component”.

  The following processing is executed inside the reverberation determination unit 13. The front suppression signal AVE_N (K) calculated by the front suppression signal generator 12 is taken in by the front suppression signal receiver 21, and the level / distribution calculator 22 determines the level Lv (K) of the front suppression signal AVE_N (K). ) And variance Vr (K). In the determination unit 23, as shown in FIG. 5, the level Lv (K) of the front suppression signal AVE_N (K) is compared with the level threshold Θ, and the variance Vr (K) of the front suppression signal AVE_N (K). Is compared with the dispersion threshold Ψ (step S101), and when the level Lv (K) is greater than or equal to the level threshold Θ and the variance Vr (K) of the front suppression signal AVE_N (K) is greater than or equal to the dispersion threshold Ψ, “ A determination result R (K) of “with reverberation component” is formed (step S102). In other cases, a determination result R (K) of “without reverberation component” is formed (step S103).

  Then, the determination result R (K) formed in this way is output from the determination result transmission unit 24 to a signal processing unit not shown in the figure.

(A-4) Effects of the First Embodiment As described above, according to the first embodiment, a frontal suppression signal is formed from a sound signal obtained by a pair of microphones, and the level and dispersion thereof. Therefore, the presence / absence of the reverberation component is determined based on the above, so it is possible to determine whether or not the sound signal has the reverberation component with a simple configuration or simple processing.

  By controlling the subsequent processing unit according to the presence or absence of the reverberation component, it is possible to cause the subsequent processing unit to execute an appropriate operation. Thereby, the improvement in the performance in the smart phone which applied the reverberation determination apparatus or program of 1st Embodiment, the audio conference apparatus, and the speech recognition apparatus can be expected.

(B) Second Embodiment Next, a second embodiment of the reverberation determination device and program according to the present invention will be described with reference to the drawings.

  The first embodiment is configured on the assumption that the voice uttered by the speaker comes from the front of the apparatus. However, there may be a time difference between the pair of microphones m1 and m2 capturing the sound at the same time of utterance, such as uttering while holding the smartphone slightly tilted. The second embodiment learns the sound source direction, forms a suppression signal (sound source direction suppression signal) in which the learned sound source direction component (target sound) is suppressed, and determines the presence or absence of a reverberation component based on the suppression signal It is what I tried.

  FIG. 6 is a block diagram illustrating a configuration of a reverberation determination device according to the second embodiment. The same or corresponding parts as those in FIG. 1 according to the first embodiment are indicated by the same reference numerals. Yes.

  In FIG. 6, the reverberation determination device 10 </ b> A of the second embodiment includes a microphone m <b> 1, a microphone m <b> 2, an FFT unit 11, a sound source direction suppression signal generation unit 12 </ b> A, a reverberation determination unit 13, and a sound source direction learning unit 14. Since only the sound source azimuth suppression signal generation unit 12A and the sound source azimuth learning unit 14 are different from those of the first embodiment, the sound source azimuth suppression signal generation unit 12A and the sound source azimuth learning unit 14 will be described below.

  The sound source direction learning unit 14 learns the sound source direction in the learning mode, for example. For example, a device equipped with the reverberation determination device is provided with a learning mode for a sound source direction, and the user is instructed to activate the learning mode and emit a sound in an environment where no reverberation occurs in a manual or the like. In such a case, the sound source direction is learned based on the signals s1 (n) and s2 (n) acquired by the microphones m1 and m2. As a method for detecting the sound source direction, an existing method using coherence as described in JP 2009-042552 A (hereinafter referred to as Reference Document 1) can be applied. Further, the time difference with the largest correlation between the signals s1 (n) and s2 (n) may be detected as the sound source direction information.

  The sound source azimuth suppression signal generator 12 </ b> A generates a sound source azimuth suppression signal having a blind spot in the learned sound source azimuth and gives it to the reverberation determination unit 13. Reference 1 describes a method for suppressing a non-target sound whose azimuth is known, and this suppression method can be used as a method for forming a sound source azimuth suppression signal. Further, if the time difference with the largest correlation between the signals s1 (n) and s2 (n) is used as the sound source direction information, one signal s1 (n) or s2 (n) is converted by the amount corresponding to the time difference. After delaying and aligning the time axes of the signals s1 (n) and s2 (n), the FFT unit 11 and the front direction suppression signal generation unit 12 described in the first embodiment are caused to function, thereby generating a sound source direction suppression signal. It can function as the generation unit 12A.

  Except for the above points, the second embodiment is the same as the first embodiment.

  According to the second embodiment, even if the orientation of the speaker (sound source) is not the front of the apparatus, whether or not there is a reverberation component in the sound signal acquired by the pair of microphones It can be determined by simple processing.

(C) Other Embodiments In the description of each of the above-described embodiments, various modified embodiments have been referred to. However, modified embodiments as exemplified below can be given.

  In each of the above embodiments, the front suppression signal AVE_N (K) is input to the reverberation determination unit 13, but the front suppression signal obtained by averaging the front suppression signal AVE_N (K) of about a few frames is determined as the reverberation. It may be possible to input to the unit 13, and in this case, it is possible to make it less susceptible to sudden fluctuations.

  In each of the above-described embodiments, the determination is made by using both the level and variation of the front suppression signal AVE_N (K) to determine whether or not there is a reverberation component. However, the level and variation of the front suppression signal AVE_N (K) have been described. It may be determined whether there is a reverberation component based on one.

  In the second embodiment, the sound source azimuth is learned in the learning mode. However, the sound source azimuth is detected in parallel with the reverberation component determination operation and processed in the same manner as in the second embodiment. good.

  In each of the above embodiments, the case where there are two microphones has been shown, but the number of microphones is not limited to two, and there may be three or more.

  In each of the above embodiments, an apparatus or a program for immediately processing a signal captured by a pair of microphones has been shown. However, the present invention is also applicable to a case where a signal captured by a pair of microphones is recorded on a recording medium and reproduced. Can be applied.

  DESCRIPTION OF SYMBOLS 10, 10A ... Reverberation determination apparatus, m1, m2 ... Microphone, 11 ... FFT (Fast Fourier Transform) unit, 12 ... Front suppression signal generation unit, 12A ... Sound source direction suppression signal generation unit, 13 ... Reverberation determination unit, 14 ... Sound source Orientation learning unit, 21... Front suppression signal receiving unit, 22... Level / dispersion calculation unit, 23.

Claims (8)

Sound source azimuth suppression signal generating means for generating a sound source azimuth suppression signal having a blind spot in the direction of the target sound source based on an input sound signal obtained by capturing ambient sounds by at least two microphones;
A reverberation determination device comprising: a reverberation determination unit that determines a presence / absence of a reverberation component in the input sound signal by comparing a feature amount reflecting a time change of the generated sound source direction suppression signal with a threshold value.   2. The reverberation determination device according to claim 1, wherein the sound source direction suppression signal generation unit generates a sound source direction suppression signal with the front of the apparatus as a sound source direction. Learning means for learning the direction of the target sound source,
The reverberation determination device according to claim 1, wherein the sound source direction suppression signal generation unit generates a sound source direction suppression signal having a blind spot in the direction of the learned target sound source. The reverberation determining means is
A level calculation unit for calculating the level of the sound source direction suppression signal from the time series of the sound source direction suppression signal;
A determination unit that determines that the reverberation component is present when the level of the sound source azimuth suppression signal is equal to or greater than the level threshold value, and that otherwise determines that there is no reverberation component. The reverberation determination device according to claim 1. The reverberation determining means is
A variance calculation unit for calculating the variance of the sound source direction suppression signal from the time series of the sound source direction suppression signal,
A determination unit that determines that the reverberation component is present when the variance of the sound source azimuth suppression signal is equal to or greater than the dispersion threshold, and otherwise determines that there is no reverberation component. The reverberation determination device according to claim 1. The reverberation determining means is
A level / dispersion calculation unit for calculating the level and variance of the sound source direction suppression signal from the time series of the sound source direction suppression signal;
A determination unit that determines that the reverberation component is present when the level of the sound source direction suppression signal is equal to or greater than the level threshold and the variance of the sound source direction suppression signal is equal to or greater than the dispersion threshold; otherwise, the determination unit determines that there is no reverberation component The reverberation determination device according to any one of claims 1 to 3.   7. The sound source azimuth suppression signal generating means generates a sound source azimuth suppression signal to be given to the reverberation determining means by averaging the sound source azimuth suppression signals once obtained by calculation for a plurality of frames. The reverberation determination device according to any one of the above. Computer
Sound source azimuth suppression signal generating means for generating a sound source azimuth suppression signal having a blind spot in the direction of the target sound source based on an input sound signal obtained by capturing ambient sounds by at least two microphones;
A reverberation determination characterized in that the reverberation determination means functions as a reverberation determination means for determining the presence or absence of a reverberation component in the input sound signal by comparing a feature amount reflecting a time change of the generated sound source direction suppression signal with a threshold value. program.

Images