JP6711205B2 - Acoustic signal processing device, program and method - Google Patents

Description

The present invention relates to an acoustic signal processing device, a program, and a method, and can be applied to, for example, a communication device or communication software used in a telephone, a videophone, or the like, or acoustic signal processing used in preprocessing of voice recognition processing. is there.

In recent years, devices equipped with various voice processing functions such as a voice call function and a voice recognition function have become widespread, such as smartphones and car navigation systems. However, with the spread of these devices, the voice processing function has come to be used in a more harsh noise environment than before, such as in a crowded city or inside a running vehicle. Therefore, there is an increasing demand for signal processing technology capable of maintaining call quality and speech recognition performance even in a noisy environment.

JP, 2014-68052, A

Kazuyuki Hiraoka, Gen Hori, “Probability Statistics for Programming”, published by Ohmsha, Ltd., October 23, 2009, p. 178-p. 179

In recent years, acoustic signal processing technology using a multi-channel microphone has been realized, but even microphones of the same model number have different sensitivities, and accurate acoustic feature quantities cannot be calculated without calibrating the sensitivity differences. Until now, methods such as measuring the microphone sensitivity in advance and setting the correction gain according to the difference in sensitivity, or comparing the input levels for each channel and automatically setting the correction gain that matches the average value, etc. Is being dealt with. However, the former takes time and the latter fills not only the difference in the sensitivity of the microphones but also the difference in the acquired input signals, so that there is a problem that the accuracy of the acoustic feature quantity calculated in the latter stage cannot be guaranteed.

One of the methods for improving this problem is to calculate the calibration gain by comparing the input levels only in the section of the input signal that comes from the front of the microphone. This is because if the signal comes from the front, the distance between each microphone and the sound source is equal, so the difference in the acoustic characteristics of the signal components reaching the microphone is very small, and the only characteristic difference that occurs between them is the microphone sensitivity. It is supposed to be expected. One of the solutions based on this is the method described in Patent Document 1. This is to note that the magnitude of the feature quantity called coherence varies depending on whether or not the voice of the target speaker comes from the front, and calculates the microphone sensitivity difference calibration gain in the signal section where the voice comes from the front. It is a technology. Even if there is a difference in sensitivity between the microphones, the behavior that the magnitude changes depending on whether or not the voice comes from the front is maintained, and thus the sensitivity difference can be calibrated by this method. (Supplement: For the calculation method of coherence, refer to Formula 7 of Patent Document 1)

However, in the method of Patent Document 1, the coherence takes a large value even when the target voice coming from the front of the microphone array and the speaking voice (interfering sound) of another speaker from the left and right simultaneously arrive, so the method comes from the front. The uncorrected voice component is also reflected in the calibration gain. Further, since the microphone sensitivity difference is random for each microphone array, it is difficult to optimize the threshold value for detecting the signal section coming from the front, and the target voice section may be erroneously determined.

Therefore, in order to improve the above two problems, there is a demand for a sensitivity calibration gain calculation method that can accurately calculate the sensitivity calibration gain even if there is an interfering sound and that can set the threshold value more easily. ..

In order to solve the above problems, an acoustic signal processing device according to a first aspect of the present invention is an acoustic signal processing device that calibrates a difference in microphone sensitivity among a plurality of input acoustic signals. Each of the first frequency domain signal obtained by converting the time domain into the frequency domain and the second frequency domain signal obtained by converting the second input acoustic signal from the time domain into the frequency domain are obtained for each frequency component. A frontal suppression signal generator that averages the values of the frequency components to generate a frontal suppression signal having a blind spot in the frontal direction, and (2) a front surface based on the first frequency domain signal and the second frequency domain signal. A directivity signal having a strong directivity characteristic in a first direction different from the direction and a directivity in a second direction different from the front direction and different from the first direction. A coherence calculation unit that calculates coherence using the second directivity signal to which a strong directivity characteristic is added, and (3) a feature amount calculation unit that calculates a correlation value indicating the relationship between the front suppression signal and coherence. And (4) a calibration gain calculator that calculates each calibration gain for the first and second input acoustic signals depending on whether the correlation value is a positive value or a negative value, and (5) each calibration The calibration gain calculation unit is provided with a calibration unit that calibrates each corresponding input acoustic signal with a gain , and when the correlation value is a positive value, the target sound section that is not affected by the interfering sound and arrives from the front is obtained. A value that reflects the microphone sensitivity of each input acoustic signal is calculated using the first and second input acoustic signals in the target sound section, and the target is calculated from the values that reflect the calculated microphone sensitivity. Calculate the sensitivity, calculate each calibration gain for each input acoustic signal based on the value that reflects each microphone sensitivity and the target sensitivity, and when the correlation value is a negative value, set each calibration gain for each input acoustic signal. It is characterized in that the initial value or the latest calibration gain of the target sound section that is not affected by the interfering sound stored in the calibration gain calculation unit is used.

An acoustic signal processing program according to a second aspect of the present invention is an acoustic signal processing program for calibrating a difference in microphone sensitivity between a plurality of input acoustic signals, wherein the computer uses (1) the first input acoustic signal from the time domain to the frequency domain. The value of each frequency component obtained by taking the difference for each frequency component of the first frequency domain signal converted to and the second frequency domain signal obtained by converting the second input acoustic signal from the time domain to the frequency domain. On average , a front suppression signal generator that generates a front suppression signal having a blind spot in the front direction, and (2) a first frequency domain signal and a second frequency domain signal based on The first directional signal having a directivity characteristic with a strong directivity in the direction 1 and the directivity characteristic with a strong directivity in a second direction different from the front direction and different from the first direction. A coherence calculation unit that calculates coherence using the added second directional signal ; (3) a feature amount calculation unit that calculates a correlation value representing the relationship between the front suppression signal and coherence; and (4) Depending on whether the correlation value is a positive value or a negative value, a calibration gain calculation unit that calculates each calibration gain for the first and second input acoustic signals, and (5) each calibration gain When functioning as a calibration unit that calibrates each input acoustic signal , the calibration gain calculation unit detects a target sound section that arrives from the front and is not affected by the interfering sound when the correlation value is a positive value, and determines the purpose. Using the first and second input acoustic signals in the sound section, a value reflecting the microphone sensitivity of each input acoustic signal is calculated, and the target sensitivity is calculated from the calculated values reflecting the plurality of microphone sensitivities. Based on the value reflecting the microphone sensitivity and the target sensitivity, calculate each calibration gain for each input acoustic signal, when the correlation value is a negative value, the initial value of each calibration gain for each input acoustic signal, or, calibration gain is stored in the calculation unit, characterized by the latest child and each calibration gain of the target sound section non affected disturbing sound.

An acoustic signal processing method according to a third aspect of the present invention is the acoustic signal processing method for calibrating the difference in microphone sensitivity between a plurality of input acoustic signals, wherein (1) the front suppression signal generation unit outputs the first input acoustic signal over time. Each frequency obtained by taking a difference for each frequency component of a first frequency domain signal converted from the domain to the frequency domain and a second frequency domain signal converted from the second input acoustic signal from the time domain to the frequency domain The values of the components are averaged to generate a frontal suppression signal having a blind spot in the frontal direction, and (2) the coherence calculator calculates the frontal direction based on the first frequency domain signal and the second frequency domain signal. Is a directivity signal having a directivity characteristic having a strong directivity in a different first direction, and a directivity having a strong directivity in a second direction different from the front direction and different from the first direction. The coherence is calculated using the second directivity signal to which the sexual characteristic is added, and (3) the feature amount calculation unit calculates a correlation value indicating the relationship between the front suppression signal and the coherence, and (4) The calibration gain calculation unit calculates each calibration gain for the first and second input acoustic signals depending on whether the correlation value is a positive value or a negative value, and (5) the calibration unit calculates each calibration gain. The gain is calibrated for each corresponding input acoustic signal, and when the correlation gain value is a positive value, the target sound section arriving from the front that is not affected by the interfering sound is detected and Using the first and second input acoustic signals in the sound section, a value reflecting the microphone sensitivity of each input acoustic signal is calculated, and the target sensitivity is calculated from the calculated values reflecting the plurality of microphone sensitivities. Based on the value reflecting the microphone sensitivity and the target sensitivity, calculate each calibration gain for each input acoustic signal, when the correlation value is a negative value, the initial value of each calibration gain for each input acoustic signal, or The calibration gain calculation unit stores the latest calibration gain of the target sound section that is not affected by the interfering sound .

According to the present invention, the sensitivity calibration gain can be accurately calculated even if there is an interfering sound, and the threshold value can be set more easily.

It is a block diagram showing the whole audio signal processing device composition concerning an embodiment. It is explanatory drawing which shows the characteristic of the directivity formed in the front suppression signal generation part which concerns on embodiment. It is a block diagram which shows the structure of the correlation calculation part which concerns on embodiment. It is a block diagram which shows the structure of the calibration gain calculation part which concerns on embodiment. 6 is a flowchart showing a processing operation in a calibration gain calculation unit according to the embodiment.

(A) Main Embodiment Hereinafter, embodiments of an acoustic signal processing device, a program, and a method according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of Embodiment FIG. 1 is a block diagram showing the overall configuration of an acoustic signal processing device 10 according to this embodiment.

In FIG. 1, the acoustic signal processing device 10 includes a plurality of microphones m_1 and m_2 (two cases are illustrated in FIG. 1), an FFT unit 11, a front suppression signal generation unit 12, a coherence calculation unit 13, and a correlation calculation. It has a unit 14, a calibration gain calculation unit 15, a first calibration gain multiplication unit 16, and a second calibration gain multiplication unit 17.

The “feature amount calculation unit” described in the claims includes the correlation calculation unit 14. The “calibration gain calculation unit” includes the calibration gain calculation unit 15. Further, the “calibration unit” includes the first calibration gain multiplication unit 16 and the second calibration gain multiplication unit 17.

In the acoustic signal processing device 10 illustrated in FIG. 1, the constituent elements other than the microphones m_1 and m_2 can be realized as software (acoustic signal processing program) executed by the CPU, and the function of the acoustic signal processing program is shown in FIG. Can be expressed as

The microphone m_1 and the microphone m_2 are arranged apart from each other by a predetermined distance (or an arbitrary distance), and each of the microphone m_1 and the microphone m_2 captures ambient sound. The acoustic signals (input signals) captured by the microphones m_1 and m_2 are converted into an analog/digital (A/D) converter (not shown), and the input signals s1(n) and s2(n) are respectively converted. , FFT unit 11, calibration gain calculation unit 15, first calibration gain multiplication unit 16 and second calibration gain multiplication unit 17. Note that n is an index indicating the input order of the samples, and is expressed by a positive integer. In the text, it is assumed that the smaller the value of n, the older the input sample and the larger the value of n, the newer the input sample.

The FFT unit 11 receives the input signals s1(n) and s2(n) from the microphones m_1 and m_2, and performs fast Fourier transform (or discrete Fourier transform) on the input signals s1(n) and s2(n). is there. This allows the input signals s1(n) and s2(n) to be transformed from the time domain to the frequency domain. When performing the fast Fourier transform, the FFT unit 11 includes a predetermined number N (N is an arbitrary integer) of samples of the input signals s1(n) and s2(n), and the analysis frames FRAME1(K) and It is assumed that FRAME2(K) is configured.

An example of constructing FRAME1 from the input signal s1 is illustrated by the equation (1). In the following equation (1), K is an index indicating the order of frames and is represented by a positive integer. Below, it is assumed that the smaller the value of K, the older the analysis frame, and the larger the value of K, the newer the analysis frame. Further, in the following explanation of the operation, unless otherwise specified, it is assumed that the index K represents the latest analysis frame to be analyzed.

The FFT unit 11 performs a fast Fourier transform process on each analysis frame to input the frequency domain signal X1(f,K) obtained by Fourier transforming the analysis frame FRAME1(K) configured from the input signal s1. A frequency domain signal X2(f,X) obtained by Fourier transforming an analysis frame FRAME2(K) composed of the signal s2 is acquired. The FFT unit 11 supplies the frequency domain signal X1(f, K) and the frequency domain signal X2(f, X) to the front suppression signal generation unit 12 and also supplies them to the coherence calculation unit 13.

Here, f is an index showing a frequency. Further, the frequency domain signals X1(f,K) and X2(f,K) are not a single value, but m (m is an arbitrary number) of a plurality of frequencies f1 to fm as shown in the following equation (2). It is assumed to be composed of (integer) components (spectral components).

In the above equation (2), X1(f,K) is a complex number, and is composed of a real part and an imaginary part. The same applies to X2(f,K) and the front suppression signal N(f,K) that appears in the front suppression signal generation unit 12 described below.

The front suppression signal generation unit 12 performs processing for suppressing the signal component coming from the front direction for each frequency component of the signal from the FFT unit. In other words, the front suppression signal generation unit 12 functions as a directional filter that suppresses a component in the front direction.

For example, as shown in FIG. 2, the front suppression signal generation unit 12 suppresses the front component from the signal from the FFT unit 11 by using a bidirectional filter having a figure 8 shape and having a blind spot in the front direction. To form a directional filter.

Specifically, the frontal suppression signal generation unit 12 performs a calculation as in Expression (3) based on the signals X1(f,K) and X2(f,K) from the FFT unit 11 to calculate the frequency component. A front suppression signal N(f,N) is generated for each. The calculation of the following equation (3) corresponds to the process of forming an 8-shaped bidirectional filter having a blind spot in the front direction as shown in FIG.
N(f,K)=X1(f,K)-X2(f,K) (3)

As described above, the frontal suppression signal generation unit 12 acquires each frequency component of the frequencies f1 to fm (power for one frame in each frequency band).

Further, the frontal suppression signal generation unit 12 calculates an average frontal suppression signal AVE_N(K) by averaging the frontal suppression signals N(f,K) over all frequencies f1 to fm according to the equation (4). To do.

The coherence calculation unit 13 calculates a coherence COH(K) by forming a signal having strong directivity in a specific direction included in the frequency domain signals X1(f,K) and X2(f,K) from the FFT unit 11. ..

Here, the process of calculating the coherence COH(K) in the coherence calculator 13 will be described.

The coherence calculation unit 13 outputs the signal B1(f,K) processed by the filter having strong directivity in the first direction (for example, the left direction) from the frequency domain signals X1(f,K) and X2(f,K). Further, the coherence calculator 13 forms the signal B2(f) processed by the filter having strong directivity from the frequency domain signals X1(f,K) and X2(f,K) in the second direction (for example, rightward direction). , K). As a method of forming the signals B1(f) and B2(f) having a strong directivity in a specific direction, an existing method can be applied. Here, the following equation (5) is applied to apply the signals to the first direction. A case where the signal B1 having a strong directivity is formed and the following formula (6) is applied to form the signal B2 having a strong directivity in the second direction will be exemplified.

In the above equations (5) and (6), S is a sampling frequency, N is an FFT analysis frame length, τ is a sound wave arrival time difference between the microphones m_1 and m_2, i is an imaginary unit, and f is a frequency. ..

Next, the coherence calculation unit 13 performs the calculation shown in the following equations (7) and (8) on the signals B1(f) and B2(f) obtained as described above. Obtain coherence COH(K). Here, B2(f,K) ^* in the equation (7) is a conjugate complex number of B2(f,K).

coef(f,K) is a coherence in a component of an arbitrary frequency f (any one of frequencies f1 to fm) forming the frame (analysis frames FRAME1(K) and FRAME2(K)) with an index of arbitrary index K. Is represented.

When determining coef(f,K), if the directivity direction of the signal B1(f) and the directivity direction of the signal B(f) are different, the signal B1(f) and the signal B2( The directivity directions according to f) may be arbitrary directions other than the front direction. Further, the method for calculating coef(f,K) is not limited to the above calculation method, and for example, the calculation method described in Patent Document 1 can be applied.

The correlation calculation unit 14 acquires the average frontal suppression signal AVE_N(K) from the frontal suppression signal generation unit 12, the coherence COH(K) from the coherence calculation unit 13, and the average frontal suppression signal AVE_N(K) and the coherence COH. A correlation coefficient cor(K) with (K) is calculated.

The significance of calculating the correlation coefficient between the coherence and the front suppression signal (average front suppression signal) having directivity in directions other than the front direction will be described.

Here, it is assumed that the sound source that emits the target sound exists in the front direction of the microphones m_1 and m_2, and the disturbing sound comes from a direction other than the front direction (for example, the left-right direction of the microphones m_1 and m_2). ..

For example, when “no interfering voice exists” and “target sound exists”, the front suppression signal has a signal value proportional to the magnitude of the target sound component. However, as shown in FIG. 2, since the gain in the front direction is smaller than the gain in the lateral direction, the gain is smaller than that in the case where the disturbing sound exists.

Further, the coherence COH(K) is a feature quantity having a deep relationship with the arrival direction of the input signal, and can be paraphrased as a correlation between two signal components. This is because the formula (6) is a formula for calculating the correlation for a certain frequency component, and the formula (7) is a formula for calculating the average of the correlation values of all the frequency components. Therefore, when the coherence COH(K) is small, it means that the correlation between the two signal components is small, and conversely, when the coherence COH(K) is large, it means that the correlation between the two signal components is large. You can The input signal when the coherence COH(K) is small has a direction of arrival largely deviated to the right or left, and can be said to be a signal coming from a direction other than the front direction. On the other hand, the input signal when the coherence COH(K) is large can be said to be a signal arriving from the front direction with little deviation in the direction of arrival.

Then, when "no interfering sound exists" and "the target sound exists", the coherence COH(K) becomes a large value, and "the interfering sound exists" and "the target sound exists" , Coherence COH(K) has a small value.

When the above behaviors are organized by focusing on the presence or absence of disturbing sounds, the following relationships are established.
When "no interfering sound exists" and "the target sound exists", the coherence COH(K) has a large value, and the front suppression signal has a value proportional to the size of the target sound component. When there is sound”, the coherence COH(K) has a small value and the front suppression signal has a large value.

By the way, in the case of the above behavior, the following can be said when the correlation coefficient between the front suppression signal and the coherence COH(K) is introduced.
・If "interfering sound does not exist", the correlation coefficient has a positive value. ・If "interfering sound exists", the correlation coefficient has a negative value.
Therefore, it is possible to determine the presence or absence of the interfering sound only by observing whether the correlation coefficient between the front suppression signal and the coherence is positive or negative. Then, by using this behavior, when the value of the correlation coefficient between the front suppression signal and the coherence is “positive”, it can be determined that the section is only the target sound from the front direction, and thus is not affected by the interfering sound, A calibration gain for the sensitivity difference between microphones m_1 and m_2 can be calculated. Further, since the target voice section can be detected simply by observing whether the value of the correlation coefficient is positive or negative, the threshold value setting becomes easy unlike the prior art.

Hereinafter, the calculation process of the correlation coefficient between the front suppression signal and the coherence in the correlation calculation unit 14 will be described in detail with reference to the drawings.

FIG. 3 is a block diagram showing the configuration of the correlation calculation unit 14 according to the embodiment.

In FIG. 3, the correlation calculation unit 14 includes a front suppression signal/coherence acquisition unit 31, a correlation coefficient calculation unit 32, and a correlation coefficient output unit 33.

The frontal suppression signal/coherence acquisition unit 31 acquires the average frontal suppression signal AVE_N(K) and the coherence COH(K), and the correlation coefficient calculation unit 32 causes the average frontal suppression signal AVE_N(K) and the coherence COH(K). ) And the correlation coefficient cor(K) is calculated. Then, the correlation coefficient output unit 33 outputs the calculated correlation coefficient cor(K) to the calibration gain calculation unit 15.

Here, the calculation method of the correlation coefficient cor(K) is not limited, but for example, the calculation method described in Non-Patent Document 1 can be applied. For example, the correlation coefficient cor(K) is calculated for each frame using the following equation (9). In the equation (9) below, Cov[AVE_N(K), COH(K)] represents the covariance of the average frontal suppression signal AVE_N(K) and the coherence COH(K). Further, in the following equation (9), σAVE_N(K) represents the standard deviation of the average frontal suppression signal AVE_N(K), and σCOH(K) represents the standard deviation of the coherence COH(K). The correlation coefficient cor(K) thus obtained takes a value of -1.0 to 1.0.

The calibration gain calculation unit 15 acquires the correlation coefficient cor(K) from the correlation calculation unit 14, observes whether the correlation coefficient cor(K) is positive or negative, and determines whether the correlation coefficient cor(K) is “positive”. The calibration gains of the microphone m_1 and the microphone m_2 are calculated using only the input signal.

FIG. 4 is a block diagram showing the configuration of the calibration gain calculation unit 15 according to the embodiment.

4, the calibration gain calculation unit 15 includes a correlation coefficient and input signal acquisition unit 41, a calibration gain calculation execution determination unit 42, a calibration gain calculation unit 43, a calibration gain storage unit 44, and a calibration gain output unit 45.

The correlation coefficient and input signal acquisition unit 41 acquires the correlation coefficient cor(K) and the FRAME1(K) and FRAME2(K) analysis frames of the input signal from the correlation calculation unit 14.

The calibration gain calculation execution determination unit 42 determines whether the value of the correlation coefficient cor(K) is “positive” or “negative” in order to determine whether to execute the calibration gain calculation. .. That is, when the value of the correlation coefficient cor(K) is “positive”, the calibration gain calculation execution determination unit 42 determines that it is the target sound section in which the input signal does not include the interfering sound, and calculates the calibration gain. It is determined that the section is to be executed. On the other hand, when the value of the correlation coefficient cor(K) is "negative", the calibration gain calculation execution determination unit 42 determines that the input signal includes a disturbing sound, and does not execute the calibration gain calculation. It is determined to be a section.

The calibration gain calculator 43 calculates the calibration gains LEVEL_GAIN_1CH and LEVEL_GAIN_2CH for the sensitivity difference between the microphones m_1 and m_2 according to the determination result of the calibration gain calculation execution determiner 42.

When the calibration gain calculation execution determination unit 42 determines that the correlation coefficient cor(K) is “positive”, the calibration gain calculation unit 43 calculates the calibration gains LEVEL_GAIN_1CH and LEVEL_GAIN_2CH. On the other hand, when the calibration gain calculation execution determination unit 42 determines that the correlation coefficient cor(K) is “negative”, the calibration gain calculation unit 43 does not calculate the calibration gain and stores it in the calibration gain storage unit 44. Set the specified value as the calibration gain.

Here, a method of calculating the calibration gains CALIB_GAIN_1CH and CALIB_GAIN_2CH by the calibration gain calculator 43 will be described.

The calibration gain calculator 43 calibrates the input signal s1 by using the following equations (10, 1), (10, 2), (11), (12, 1) and (12, 2). The gain CALIB_GAIN_1CH and the calibration gain CALIB_GAIN_2CH for the input signal s2 are calculated.

The equation (10, 1) is for calculating the average LEVEL_1CH of the absolute values of all the constituent elements of the current frame (Kth frame) of the input signal s1(n) captured by the microphone m_1. The value LEVEL_1CH that has been set can be regarded as a value that reflects the sensitivity of the microphone m_1. The equation (10, 2) is for calculating the average LEVEL_2CH of the absolute values of all the constituent elements of the current frame (Kth frame) of the input signal s2(n) captured by the microphone m_2. The value LEVEL_2CH can be regarded as a value that reflects the sensitivity of the microphone m_2.

Note that, for example, the total sum of absolute values of the constituent elements of each frame in a predetermined number of frames may be used as the values LEVEL_1CH and LEVEL_2CH that reflect the microphone sensitivity. Further, for example, the average of the absolute values of all the elements (signal components) forming the latest P (P≦K) frames whose correlation coefficient cor(K) is “positive” is a value that reflects the microphone sensitivity. You may make it use as LEVEL_1CH and LEVEL_2CH. In the latter case, the sum of absolute values of the components of the latest P-1 frames whose correlation coefficient cor(K) is "positive" is stored to save the current frame (K-th frame). ) When the information of FRAME1(K) and FRAME2(K) is given, the values LEVEL_1CH and LEVEL_2CH reflecting the microphone sensitivity can be easily calculated. As described above, by calculating the average or sum of the absolute values of the signal components for a long period of time, it is possible to suppress the influence of the instantaneous fluctuation of the input signal and calculate a value that reflects the microphone sensitivity.

The equations (10, 1) and (10, 2) are examples of the equations for calculating the value that reflects the microphone sensitivity, and various other equations can be applied as described above. However, the calculation formula of the value LEVEL_1CH that reflects the microphone sensitivity of the microphone m_1 and the calculation formula of the value LEVEL_2CH that reflects the microphone sensitivity of the microphone m_2 need to be the same.

Formula (11) calculates the average AVE_LEVEL of the sensitivities LEVEL_1CH and LEVEL_2CH of the two microphones m_1 and m_2 as the target sensitivity of the calibrated microphones m_1 and m_2. The target sensitivity may be the larger value or the smaller value of the sensitivities LEVEL_1CH and LEVEL_2CH of the two microphones m_1 and m_2.

In the equation (12, 1), as can be understood by considering an equation in which the denominator LEVEL_1CH on the right side of the equation is transferred to the left side, calibration is performed so that the value obtained by multiplying the sensitivity LEVEL_1CH of the microphone m_1 by the calibration gain CALIB_GAIN_1CH becomes the target sensitivity AVE_LEVEL. It is an expression that determines the gain CALIB_GAIN_1CH. Similarly, in the expression (12, 2), as can be understood by considering an expression in which the denominator LEVEL_2CH on the right side is transferred to the left side, the value obtained by multiplying the sensitivity LEVEL_2CH of the microphone m_2 by the calibration gain CALIB_GAIN_2CH becomes the target sensitivity AVE_LEVEL. In addition, it is an expression that determines the calibration gain CALIB_GAIN_2CH.

The calibration gain storage unit 44 stores the calibration gains CALIB_GAIN_1CH (=INIT_GAIN_1CH) and CALIB_GAIN_2CH (=INIT_GAIN_2CH) that are applied when the calibration gain calculation unit 43 does not calculate the calibration gain. As such calibration gains INIT_GAIN_1CH and INIT_GAIN_2CH, a value 1.0 that is not calibrated may be applied, or the latest value calculated by the calibration gain calculator 43 may be applied.

The calibration gain output unit 45 corresponds to the calibration gains CALIB_GAIN_1CH and CALIB_GAIN_2CH obtained by the calculation by the calibration gain calculation unit 43, or the calibration gains INIT_GAIN_1CH and INIT_GAIN_2CH read from the storage unit 24, respectively. To give to.

The first calibration gain multiplication unit 16 outputs the post-calibration signal y1(n) obtained by multiplying the input signal s1(n) from the microphone m_1 by the calibration gain CALIB_GAIN_1CH.

The second calibration gain multiplication unit 17 outputs the post-calibration signal y2(n) obtained by multiplying the input signal s2(n) from the microphone m_2 by the calibration gain CALIB_GAIN_2CH.

(A-2) Operation of Embodiment Next, the operation of the overall processing and the calculation processing of the calibration gain in the acoustic signal processing device 10 according to the embodiment will be described in detail with reference to the drawings.

The input signals s1(n) and s2(n) for one frame are input to the FFT unit 11 from the microphones m_1 and m_2 through an AD converter (not shown).

The FFT unit 11 performs Fourier transform on the analysis frames FRAME1(K) and FRAME2(K) based on the input signals s1(n) and s2(n) for one frame, and the signal X1(f,K) shown in the frequency domain. And X2(f,K). The signals X1(f,K) and X2(f,K) generated by the FFT unit 11 are given to the front suppression signal generation unit 12 and the coherence calculation unit 13.

The frontal suppression signal generation unit 12 calculates a frontal suppression signal N(f,K) having directivity in a direction other than the frontal direction based on the signals X1(f,K) and X2(f,K). Then, the frontal suppression signal generation unit 12 generates an averaged frontal suppression signal AVE_N(f,K) by averaging the frontal suppression signals N(f,K) over all frequencies, and this average frontal suppression signal AVE_N(K ) Is given to the correlation calculation unit 14.

On the other hand, the coherence calculation unit 13 calculates the coherence COH based on the signals X1(f,K) and X2(f,K), and supplies the coherence COH to the correlation calculation unit 14.

The correlation calculation unit 14 acquires the average frontal suppression signal AVE_N(f,K) and the coherence COH, and calculates the correlation coefficient cor(K) between the average frontal suppression signal AVE_N(f,K) and the coherence COH, The correlation coefficient cor(K) is given to the calibration gain calculator 15.

The calibration gain calculation unit 15 acquires the correlation coefficient cor(K), observes whether the correlation coefficient cor(K) is positive or negative, and determines the signals s1(n) and s2(n) according to the determination result. Calculate the calibration gain for. Further, the calibration gain calculation unit 15 outputs the calibration gain CALIB_GAIN_1CH for the signal s1(n) to the first calibration gain multiplication unit 16, and outputs the calibration gain CALIB_GAIN_2CH for the signal s2(n) to the second calibration gain multiplication unit 17. ..

FIG. 5 is a flowchart showing the processing operation in the calibration gain calculation unit 15.

The correlation coefficient and input signal acquisition unit 41 acquires the correlation coefficient cor(K) from the correlation coefficient unit 14 and outputs the FRAME1(K) and FRAME2(K) of the input signals s1(n) and s2(n). It is acquired (S51).

Then, the calibration gain calculation execution determination unit 42 determines whether the value of the correlation coefficient cor(K) is positive or negative (S52).

When the correlation coefficient cor(K) is positive, there is no interfering sound coming from directions other than the front direction, and it is regarded as the target sound section from the front direction, and the calibration gain calculation unit 43 determines that the correlation coefficient cor(K) ), FRAME1(K), and FRAME2(K), according to equations (10,1), (10,2), (11), (12,1), and (12,2), Calibration gains CALIB_GAIN_1CH and CALIB_GAIN_2CH for s1(n) and signal s2(n) are calculated (S53). At this time, the calibration gain calculator 43 stores the calculated calibration gains CALIB_GAIN_1CH and CALIB_GAIN_2CH in the calibration gain storage 44, respectively, and updates the calibration gains stored in the calibration gain storage 44.

When the correlation coefficient cor(K) is negative, it is considered that there is an interfering sound coming from a direction other than the front direction, and the calibration gain calculation unit 43 sets the value stored in the calibration gain storage unit 44 to the calibration gain CALIB_GAIN_1CH and It is set to CALIB_GAIN_2CH (S54).

That is, when the calibration gain storage unit 44 stores the initial values INIT_GAIN_1CH and INIT_GAIN_2CH of the calibration gain, INIT_GAIN_1CH is set to CALIB_GAIN_1CH and INIT_GAIN_2CH is set to CALIB_GAIN_2CH. Alternatively, when the latest calibration gain is stored in the calibration gain storage unit 44, the latest calibration gains stored in the calibration gain storage unit 44 are set as the current calibration gains CALIB_GAIN_1CH and CALIB_GAIN_2CH.

Then, the calibration gain output unit 45 outputs the calibration gain CALIB_GAIN_1CH to the first calibration gain multiplication unit 16 and outputs the calibration gain CALIB_GAIN_2CH to the second calibration gain multiplication unit 17 (S55). Then, the calibration gain calculation unit 15 updates the index K (S56), moves to S51, and performs a calibration gain calculation process for the next index.

Here, since the calibration gain calculation unit 15 does not change the calibration gain after once calculating the calibration gain, constantly updating the calibration gain wastes the amount of calculation. May be stopped. That is, in an environment where the acoustic signal processing device 10 having the microphones m_1 and m_2 is used, it is not necessary to constantly update the calibration gain after acquiring the calibration gains for the microphones m_1 and m_2 in the initial stage. The calibration gain may be calculated when necessary.

Then, the first calibration gain multiplication unit 16 multiplies the signal s1(n) by the calibration gain CALIB_GAIN_1CH and outputs the post-calibration signal y1(n), and the second calibration gain multiplication unit 17 outputs the signal s2(n). The calibration gain CALIB_GAIN_2CH is multiplied, and the post-calibration signal y2(n) is output.

(A-3) Effects of the Embodiment As described above, according to this embodiment, when there is an interfering sound arriving from a direction other than the front direction, the correlation coefficient between the front suppression signal and COH is negative. By using the characteristic behavior that the correlation coefficient between the front suppression signal and COH becomes positive when there is no interfering sound, the threshold setting for the designer is not affected by the interfering sound. An easy microphone sensitivity calibration method can be realized.

Accordingly, by applying the process of calculating the calibration gain for the microphones m_1 and m_2 to the pre-processing of various signal processing methods using the microphone array, it is expected that the subsequent audio processing performance will be improved.

(B) Other Embodiments Although various modified embodiments have been mentioned in the above-described embodiments, the present invention can be applied to the following modified embodiments.

(B-1) In the above-described embodiment, the case where the correlation calculation unit calculates the correlation coefficient as the feature amount of the frontal suppression signal and the coherence is illustrated, but the covariance is calculated as the feature amount of the frontal suppression signal and the coherence. Even if the value of is calculated, the same effect as that of the above-described embodiment can be obtained.

(B-2) In the above-described embodiment, the acoustic signal processing device according to the present invention is applicable to various devices as long as the device has a voice processing function (for example, voice recognition processing) including a plurality of microphones. It can be widely applied to, for example, smartphones, tablet terminals, video conference terminals, car navigation systems, call center terminals, robots, devices that use sound signals as sensor signals, and the like.

Further, for example, the acoustic signal processing device of the present invention may be mounted on a device having a communication function, and the device may transmit the post-calibration signal to a server having a predetermined voice processing function via a network.

Furthermore, for example, a device having a communication function, which includes a plurality of microphones, may transmit the input signal of each microphone to a server equipped with the acoustic signal processing device of the present invention through a network. In this case, the server equipped with the acoustic signal processing device can calculate the calibration gain for each input signal according to the correlation coefficient between the front suppression signal and the coherence, as in the above-described embodiment.

(B-3) In the above-described embodiment, the case where the number of microphones is two is illustrated, but the present invention can be applied to an apparatus that acquires an input signal from each of three or more microphones.

10... Acoustic signal processing device, m_1 and m_2... Microphone, 11... FTT section, 12... Front suppression signal generation section, 13... Coherence calculation section,
14... Correlation calculation unit, 31... Front suppression signal, coherence acquisition unit, 32... Correlation coefficient calculation unit, 33... Correlation coefficient output unit,
15... Calibration gain calculation unit, 41... Correlation coefficient and input signal acquisition unit, 42... Calibration gain calculation execution determination unit, 43... Calibration gain calculation unit, 44... Calibration gain storage unit, 45... Calibration gain output unit,
16... 1st calibration gain multiplication part, 17... 2nd calibration gain multiplication part.

Claims (6)

In an acoustic signal processing device for calibrating the difference in microphone sensitivity between a plurality of input acoustic signals,
For each frequency component, a first frequency domain signal obtained by converting the first input acoustic signal from the time domain into the frequency domain and a second frequency domain signal obtained by converting the second input acoustic signal from the time domain into the frequency domain A value of each frequency component obtained by taking the difference is averaged, a front suppression signal generation unit that generates a front suppression signal having a blind spot in the front direction,
A first directivity signal having a directivity characteristic with strong directivity in a first direction different from the front direction based on the first frequency domain signal and the second frequency domain signal; A coherence calculation unit that calculates coherence by using a second directional signal that is different from the first direction and has a directivity characteristic with a strong directivity in a second direction different from the first direction ;
A feature amount calculation unit that calculates a correlation value representing the relationship between the front suppression signal and the coherence,
A calibration gain calculation unit that calculates each calibration gain for the first and second input acoustic signals depending on whether the correlation value is a positive value or a negative value ,
A calibration unit that calibrates each corresponding input acoustic signal with each calibration gain described above,
The calibration gain calculation unit is
When the correlation value is a positive value, a target sound section coming from the front, which is not affected by the interfering sound, is detected, and the first and second input acoustic signals in the target sound section are used, A value reflecting the microphone sensitivity of each input acoustic signal is calculated, a target sensitivity is obtained from a value reflecting a plurality of calculated microphone sensitivities, and based on the target sensitivity and the value reflecting each microphone sensitivity, Calculate each calibration gain for each input acoustic signal,
When the correlation value is a negative value, the initial value of each calibration gain for each of the input acoustic signals, or stored in the calibration gain calculation unit, the latest of the target sound section not affected by the interfering sound An acoustic signal processing device, wherein each of the calibration gains is used. The calibration gain calculation unit is
When the correlation value is a positive value, it is determined that the target sound section that is not affected by the interfering sound is a section in which the calculation of each calibration gain is executed, and when the correlation value is a negative value, the effect of the interfering sound is determined. A calibration gain calculation execution determination unit that determines that the calculation of each calibration gain is not performed as a target sound section
A calibration gain storage unit that stores each calibration gain,
Based on the determination by the calibration gain calculation execution determination unit, in the target sound section that is not affected by the interfering sound, while performing the calculation of each of the calibration gain, the calculated calibration gain in the calibration gain storage unit And a calibration gain calculation unit that outputs the latest calibration gains stored in the calibration gain storage unit in a target sound section that is stored and is affected by an interfering sound.
Audio signal processing apparatus according to claim 1, characterized in that it comprises a. The calibration gain calculation unit calculates, for each of the input acoustic signals, an average value of absolute values of a plurality of signal components in the input acoustic signal as a value that reflects the microphone sensitivity of the input acoustic signal. The acoustic signal processing device according to claim 1 or 2 . The calibration gain calculation unit determines the average value of the values reflecting the calculated plurality of microphone sensitivities as the target sensitivity, and the determined target sensitivity reflects the microphone sensitivity associated with each of the input acoustic signals. The acoustic signal processing device according to claim 1, wherein a calibration gain for each of the input acoustic signals is calculated by dividing by a value . In an acoustic signal processing program that calibrates the difference in microphone sensitivity between multiple input acoustic signals,
Computer,
For each frequency component, a first frequency domain signal obtained by converting the first input acoustic signal from the time domain into the frequency domain and a second frequency domain signal obtained by converting the second input acoustic signal from the time domain into the frequency domain A value of each frequency component obtained by taking the difference is averaged, a front suppression signal generation unit that generates a front suppression signal having a blind spot in the front direction,
A first directivity signal having a directivity characteristic with strong directivity in a first direction different from the front direction based on the first frequency domain signal and the second frequency domain signal; A coherence calculation unit that calculates coherence by using a second directional signal that is different from the first direction and has a directivity characteristic with a strong directivity in a second direction different from the first direction ;
A feature amount calculation unit that calculates a correlation value representing the relationship between the front suppression signal and the coherence,
A calibration gain calculation unit that calculates each calibration gain for the first and second input acoustic signals depending on whether the correlation value is a positive value or a negative value ,
With each of the above calibration gains, function as a calibration unit that calibrates each corresponding input acoustic signal ,
The calibration gain calculation unit is
When the correlation value is a positive value, a target sound section coming from the front, which is not affected by the interfering sound, is detected, and the first and second input acoustic signals in the target sound section are used, A value reflecting the microphone sensitivity of each input acoustic signal is calculated, a target sensitivity is obtained from a value reflecting a plurality of calculated microphone sensitivities, and based on the target sensitivity and the value reflecting each microphone sensitivity, Calculate each calibration gain for each input acoustic signal,
When the correlation value is a negative value, the initial value of each calibration gain for each of the input acoustic signals, or stored in the calibration gain calculation unit, the latest of the target sound section not affected by the interfering sound Set each calibration gain
Acoustic signal processing program characterized and this. In the acoustic signal processing method for calibrating the difference in microphone sensitivity between a plurality of input acoustic signals,
A front suppression signal generation unit converts a first input acoustic signal from a time domain to a frequency domain in a first frequency domain signal and a second input acoustic signal from a time domain to a frequency domain in a second frequency domain signal. By averaging the values of each frequency component obtained by taking the difference with the signal for each frequency component, to generate a frontal suppression signal having a blind spot in the frontal direction,
A first directivity in which the coherence calculation unit imparts a strong directivity characteristic in a first direction different from the front direction based on the first frequency domain signal and the second frequency domain signal. Calculating coherence using a signal and a second directional signal that is different from the front direction and has a directional characteristic having a strong directivity in a second direction different from the first direction ;
The feature amount calculation unit calculates a correlation value representing the relationship between the front suppression signal and the coherence,
The calibration gain calculator calculates each calibration gain for the first and second input acoustic signals depending on whether the correlation value is a positive value or a negative value ,
The calibration unit calibrates each of the corresponding input acoustic signals with each of the above calibration gains ,
The calibration gain calculation unit is
When the correlation value is a positive value, a target sound section coming from the front, which is not affected by the interfering sound, is detected, and the first and second input acoustic signals in the target sound section are used, A value reflecting the microphone sensitivity of each input acoustic signal is calculated, a target sensitivity is obtained from a value reflecting a plurality of calculated microphone sensitivities, and based on the target sensitivity and the value reflecting each microphone sensitivity, Calculate each calibration gain for each input acoustic signal,
When the correlation value is a negative value, the initial value of each calibration gain for each of the input acoustic signals, or stored in the calibration gain calculation unit, the latest of the target sound section not affected by the interfering sound An acoustic signal processing method, wherein each calibration gain is used.

Images