CN102831898A - Microphone array voice enhancement device with sound source direction tracking function and method thereof - Google Patents

Abstract

The invention discloses a microphone array speech enhancement device with sound source direction tracking function and its method, relating to speech signal processing. The device is provided with a microphone array, an adjustable parallel beamformer group module, a fixed coefficient FIR filter module, a fixed coefficient signal blocking module, an adaptive noise canceller module, and a sound source direction update module. The method includes the steps of initialization, adjustable beam forming, fixed coefficient filtering, signal blocking, adaptive noise elimination, sound source direction update and the like. An adjustable parallel beamformer group combined with a sidelobe cancellation structure is provided to realize real-time tracking of the target sound source direction, and the sound source direction tracking function is directly embedded in the generalized sidelobe canceller structure, which can realize sound source direction tracking, The simultaneous speech enhancement can overcome the sensitivity of the algorithm performance to the DOA estimation error.

Description

Microphone array speech enhancement device with sound source direction tracking function and method thereof

technical field

The invention relates to a voice signal processing, in particular to a microphone array voice enhancement device with sound source direction tracking function and its method.

Background technique

The research and application based on microphone array is a new field of speech signal processing. In the field of speech signal processing such as speech recognition, speech control, and speech synthesis, the speech signal received by the microphone is greatly affected by environmental noise and interference, which seriously affects the processing quality of the speech signal. The general speech enhancement system based on a single microphone is difficult Get a better enhancement effect. Since the microphone array utilizes the spatial information of the target signal, noise and interference, the speech enhancement system based on the microphone array can provide better enhancement effect.

Array microphones can form multiple microphones into an array according to the designed topology, so that the collected signals can add a space domain on the basis of the time-frequency domain, and can perform space-time diversity processing on the collected multipath signals. The array can form different responses to signals in different directions, that is, the spatial pointing characteristics of the array, so that the array microphone has functions such as sound source localization and tracking, voice extraction and separation, and denoising, thereby improving the quality of voice signals in complex backgrounds , to make up for the defect that isolated microphones cannot acquire and utilize spatial information.

In 1982, Griffiths and Jim (1, L.J.Griffiths, C.W.Jim. An alternative approach to linearly constrained adaptive beamforming. IEEE Transactions on Antennas and Propagation. January, 1982, 30, 27-34) proposed a modified linear beamformer, that is, a generalized side lobe Eliminator (Generalized Sidelobe Canceller, GSC for short). The generalized sidelobe canceller allows extensive control over the response of the beamformer, especially the linearly constrained restricted problem can be transformed into an unrestricted adaptive solution, so it has more general significance, in the microphone array speech enhancement processing It has been extensively researched and applied. In response to the shortcomings of the classic GSC algorithm in practical applications, Gannot et al. (2, Sharon Gannot, Israel Cohen. Speech Enhancement Based on the General TransferFunction GSC and Postfiltering. IEEE Transactions on Speech and Audio Processing. 2004, 12 (6)) based on the classic Based on the generalized sidelobe canceller algorithm, a generalized sidelobe canceller based on the non-stationary acoustic transfer function of the useful signal is proposed. Abad et al. (3. A Abad, J Hernando. Speech Enhancement and recognition by Integrating Adaptive Beamforming and Wiener Filtering. IEEE Sensor Array and Multichannel Signal Processing Workshop, SAM, Sitges, 2004) proposed to introduce Wiener filtering into the non-adaptive branch of GSC way to further improve the effect of sidelobe cancellation.

It should be pointed out that obtaining the accurate direction of the sound source is the premise of using the generalized sidelobe canceller technology for microphone array speech enhancement processing. Only after obtaining the target sound source direction can the microphone be trained by an adaptive algorithm under the criterion of minimizing the output variance. The array forms beams in the direction of the desired sound source for speech enhancement.

Chinese patent ZL 200510105526.7 discloses a multi-channel adaptive speech signal processing method using noise reduction, which improves the signal-to-noise ratio of the signal channel by adding an adaptive processor to the fixed beam path of the GSC. However, this method still needs to compensate the time delay of each channel with the help of time delay estimation in the frequency domain, so that the beam is aligned in the direction of the desired sound source.

However, the classic generalized sidelobe canceller algorithm always keeps the beam aligned to the desired channel unchanged during the speech enhancement process, and the expected signal direction aligned by the algorithm can be set again only after the positioning algorithm recalculates the target direction. In the actual application process of microphone array speech enhancement such as video conferencing, speech recognition, and speaker recognition, there are often occasions when the speaker moves during the speaking process. At this time, from the perspective of microphone array speech enhancement, it is required that the directional beam of the microphone array can always be aimed at the moving target speaker to obtain the best speech enhancement effect. However, according to the traditional generalized sidelobe canceller algorithm structure, the sound source localization calculation needs to be performed first, and then the GSC microphone array speech enhancement processing is performed according to the obtained target sound source azimuth, which will cause the expected sound source azimuth aligned by the GSC to be different from the actual sound source. There is a lag between source bearings, which affects speech enhancement.

At the same time, since the classic generalized sidelobe canceller needs to obtain the target sound source direction as the precondition for the algorithm to run, when there is a certain error between the target sound source direction and the actual sound source direction, the DOA (Direction of arrival) mismatch will It affects the blocking effect of the blocking matrix on the desired signal in the sound source direction, that is, part of the signal of the non-adaptive branch leaks to the input of the noise canceller of the adaptive branch, causing the speech signal to be weakened, thus affecting the speech enhancement effect.

Contents of the invention

The first object of the present invention is to provide a microphone array speech enhancement device with sound source direction tracking function.

The second object of the present invention is to provide a microphone array speech enhancement method using the microphone array speech enhancement device with sound source direction tracking function.

The microphone array speech enhancement device with sound source direction tracking function is provided with:

Microphone array for multi-channel acquisition, pre-processing and analog-to-digital conversion of voice signals;

Adjustable parallel beamformer group module, used for real-time tracking of sound source direction by adjusting the time delay of each channel;

A fixed-coefficient FIR filter module is used to form expected frequency characteristics in the non-adaptive branch to obtain noisy signals;

The fixed coefficient signal blocking module is used to filter out the signal contained in the direction of the sound source to obtain the noise signal;

The adaptive noise canceller module is used to eliminate the noise contained in the noisy signal with the noise signal as a reference according to the principle of adaptive noise cancellation, and output the speech enhancement signal;

The sound source direction update module is used to select the current optimal sound source direction according to the current expected sound source direction and the current expected sound source left and right offset direction corresponding to the speech enhancement signal variance output by sidelobe cancellation, and obtain the current optimal sound source direction. The source direction is input to the adjustable parallel beamformer group module to update the current sound source direction and the left and right offset direction of the current sound source direction;

The voice signal output terminals of each channel in the microphone array pass through the preamplifier circuit and the analog-to-digital converter in turn, and are directly connected to the signal input terminals of the adjustable parallel beamformer group module through the data line;

The adjustable parallel beamformer group module is provided with 3 parallel adjustable beamformers, and the 3 parallel adjustable beamformers are respectively provided with signal input terminals and sound source direction input terminals, and each signal input terminal is connected to the module Each channel voice signal output end of the digital converter, and each sound source direction input end is connected with the current optimal sound source direction output end of the sound source direction update module; the delay adjustment of the three parallel adjustable beamformers After each channel voice signal output terminal is connected to the input end of the fixed coefficient signal blocking module, the superposition output of each channel voice signal after time delay adjustment is then connected with the input end of the fixed coefficient FIR filter module;

The adaptive noise eliminator module is provided with a reference noise input end and a noisy speech input end, and the fixed coefficient signal blocking module blocks the input signal after the output is connected to the reference noise input end of the adaptive noise eliminator module, and the fixed coefficient The output after the FIR filter module filters the input signal is connected to the noise-containing speech input end of the adaptive noise canceller module;

The voice enhancement signal output terminal of the adaptive noise eliminator module is connected to the sound source direction update module to select the current optimal sound source direction, and the adaptive noise eliminator module is provided with the current optimal sound source direction output end and the voice enhancement The signal output terminal, the output terminal of the current optimal sound source direction is connected to the adjustable parallel beamformer group module, and the voice enhancement signal output terminal outputs the voice enhancement signal corresponding to the current optimal sound source direction.

The microphone array may be an equidistant linear array composed of 5-element microphones.

The microphone array voice enhancement method adopts the microphone array voice enhancement device with sound source direction tracking function, and the method comprises the following steps:

1 initialization step: In the initialization stage, set the front as the default current sound direction, or input the sound source direction obtained by the microphone array positioning algorithm as the initial sound source direction, as the sound source direction parameter of the adjustable beamformer group module;

1 adjustable beamforming step: adjust the step size according to the set sound source direction, and adjust the step size by adding and subtracting the current sound source direction to generate the left and right offset directions of the current sound source direction and calculate the signal of each channel of the microphone array Corresponding time delay compensation value, and carry out time delay compensation in the corresponding direction for the output signals of each channel of the microphone array in the three parallel beamformers, so that they are respectively aligned with the direction of the sound source and the direction of the left and right offset;

1 fixed coefficient filtering step: use fixed coefficients to perform FIR filtering on the outputs of the 3 parallel beamformers to form the required frequency response in the current sound source direction and the 3 beam alignment directions in the left and right offset directions;

1 signal blocking step: In the adaptive branch, the signals output by the 3 adjustable beamformers are respectively input into 3 identical signal blocking matrices, which are used to filter out the current sound source direction and align the 3 beams in the left and right offset directions Speech signals contained within directions;

1 adaptive noise elimination step: In the adaptive branch, the blocking matrix output signals corresponding to the current sound source direction and the three beams in the left and right offset directions are used as the reference noise signal, and the output signals corresponding to the three beams in the fixed branch Adaptive noise cancellation processing as a noisy input signal;

1 sound source direction update step: obtain the difference between the output of the non-adaptive branch signal of the 3 beam directions and the output of the corresponding adaptive noise canceller, and select the beam direction with the smallest variance in the observation window as the current sound source direction for update, After updating the current sound source direction, add and subtract the adjustment steps respectively to generate the left and right offset directions of the current sound source direction, and calculate the corresponding delay compensation value of each channel signal of the microphone array to perform algorithm iteration again.

The problem to be solved by the present invention is to provide a microphone array speech enhancement device with sound source direction tracking function based on the traditional generalized sidelobe canceller algorithm. For speaker recognition, voice recognition and other situations where the speaker may move during microphone array speech enhancement, the present invention provides an adjustable parallel beamformer group combined with a sidelobe cancellation structure to realize real-time tracking of the direction of the target sound source , the sound source direction tracking function is directly embedded in the generalized sidelobe canceller structure, which can realize sound source direction tracking and speech enhancement at the same time, thereby overcoming the sensitivity of algorithm performance to DOA estimation error.

The technical solution of the invention is to add the sound source direction tracking function on the basis of the traditional generalized side lobe canceller to enhance the processing of the voice signal.

The concrete train of thought that the microphone array voice enhancement device with sound source direction tracking function realizes the microphone array voice enhancement with sound source direction tracking function is as follows: first, three parallel beams are set in the non-adaptive signal branch of the side lobe canceller The beamformers form a beamformer group, and each beamformer is sequentially aligned with the current sound source direction and the left and right offset orientation of the current sound source direction (the left and right offset directions are obtained by adjusting the step size by adding or subtracting one direction to the current sound source direction respectively. ), calculate the output of the adaptive filter of the sidelobe canceller under the condition that the microphone array beams are respectively aligned with the above three expected sound source directions, and perform a performance comparison in an observation window, and select The current best sound source direction; after obtaining the current best sound source direction, set it as the current sound source direction, and get the new sound source direction to drift left and right, and perform the direction iteration of the parallel beamformer group again; through the above An iterative process for microphone array speech enhancement with sound source direction tracking. Therefore, the basic structure of the speech enhancement with sound source direction tracking microphone array disclosed in the present invention is actually three sidelobe cancellers working in parallel and with adjustable directions, wherein each sidelobe canceller corresponds to the current sound source direction and the current direction respectively. The left and right offset direction of the sound source direction, and the iterative optimization selection of the current sound source direction through the output signal variance of each sidelobe canceller, and the current sound source direction tracking and speech enhancement are performed simultaneously through the adaptive algorithm in the iterative process.

Compared with the existing microphone array positioning and speech enhancement methods, the microphone array speech enhancement device with sound source azimuth tracking function of the present invention has three outstanding advantages: first, due to the adjustable parallel Beamformer group, so it can realize sound source target direction tracking and microphone array voice enhancement at the same time, without relying on additional microphone array positioning algorithm for target sound source direction calculation; second, due to the built-in The parallel beamformer group is always running simultaneously with the speech enhancement in the algorithm flow, so it can ensure the real-time tracking of the direction of the target sound source while the speech enhancement is in progress, so as to achieve fast and real-time response to the tracking of the moving target sound source; 3. When there is an error between the acquired sound source direction and the actual sound source direction, the sound source azimuth tracking through the adjustable parallel beamformer group can eliminate the mismatch of the direction of arrival (DOA) of the speech signal, thereby improving the adaptive channel. signal leakage and improve speech enhancement performance.

Description of drawings

FIG. 1 is a block diagram of the structure of an embodiment of a microphone array speech enhancement device with sound source direction tracking function according to the present invention.

FIG. 2 is a circuit diagram of a 5-element microphone array and its connection with a microprocessor according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of data flow and control flow connection of each signal processing module in the embodiment of the present invention.

Fig. 4 is a schematic structural diagram of an adjustable beamformer according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of calculation of time delay compensation values of each channel of ultrasonic-assisted microphone speech enhancement according to an embodiment of the present invention.

Detailed ways

In order to make the technical content, features and advantages of the present invention more comprehensible, the following embodiments will further illustrate the present invention in conjunction with the accompanying drawings.

As shown in Figure 1, the embodiment of the microphone array speech enhancement device with sound source direction tracking function is provided with a microphone array 1, an adjustable parallel beamformer group module 2, a fixed coefficient FIR filter module 3, a fixed coefficient signal blocking Module 4, Adaptive Noise Canceller Module 5 and Sound Source Direction Update Module 6.

Microphone array 1 is used for multi-channel acquisition, pre-processing and analog-to-digital conversion of voice signals; adjustable parallel beamformer group module 2 is used for real-time tracking of sound source direction by adjusting the time delay of each channel; fixed coefficient FIR filter module 3 is used to form the expected frequency characteristics in the non-adaptive branch to obtain the noisy signal; the fixed coefficient signal blocking module 4 is used to filter the signal contained in the direction of the sound source to obtain the noise signal; the adaptive noise canceller module 5 is used to obtain the noise signal according to The principle of adaptive noise cancellation uses the noise signal as a reference to eliminate the noise contained in the noisy signal, and outputs a speech enhancement signal; the sound source direction update module 6 is used to correspond to the current expected sound source direction and the current expected sound source left and right offset direction The variance of the speech enhancement signal output by sidelobe cancellation selects the current optimal sound source direction, and inputs the obtained current optimal sound source direction to the adjustable parallel beamformer group module to update the current sound source direction and the left and right sides of the current sound source direction offset direction.

The voice signal output terminals of each channel in the microphone array 1 pass through the preamplifier circuit and the analog-to-digital converter 7 in turn, and are directly connected to the signal input terminals of the adjustable parallel beamformer group module 2 through data lines.

The adjustable parallel beamformer group module 2 is provided with 3 parallel adjustable beamformers, and the 3 parallel adjustable beamformers are respectively provided with signal input terminals and sound source direction input terminals, and each signal input terminal is connected to Each channel speech signal output end of the analog-to-digital converter, each sound source direction input end is connected with the current optimal sound source direction output end of the sound source direction update module 6; the time-lapse of the three parallel adjustable beamformers The output terminal of each channel voice signal after delay adjustment is connected to the input terminal of the fixed coefficient signal blocking module 4 , and the superimposed output of each channel voice signal after delay adjustment is connected to the input terminal of the fixed coefficient FIR filter module 3 .

The adaptive noise eliminator module 5 is provided with a reference noise input end and a noisy speech input end, and the fixed coefficient signal blocking module 4 blocks the input signal after the output is connected to the reference noise input end of the adaptive noise eliminator module 5 The output of the fixed coefficient FIR filter module 3 after filtering the input signal is connected to the noisy speech input terminal of the adaptive noise canceller module 5 .

The voice enhancement signal output terminal of the adaptive noise eliminator module 5 is connected to the sound source direction updating module 6 to carry out the selection of the current optimal sound source direction, and the adaptive noise eliminator module 5 is provided with the current optimal sound source direction output terminal and the voice enhancement signal output terminal, the output terminal of the current optimal sound source direction is connected to the adjustable parallel beamformer group module 2, and the voice enhancement signal output terminal outputs the voice enhancement signal corresponding to the current optimal sound source direction.

The microphone array 1 may be an equidistant linear array composed of 5-element microphones.

In the embodiment of the microphone array voice enhancement device with sound source direction tracking function, the microphone array is composed of 5 equidistantly arranged microphones (m0, m1, ... m4) to form a microphone line array, and the voice signals obtained by each microphone in the array are sent into Built-in sidelobe canceller with 3 parallel adjustable beamformer groups for sound source direction tracking and speech enhancement.

The microphone array is composed of microphones and hardware circuits. The microphone array is composed of small size, simple structure, and good electroacoustic performance. An analog-to-digital conversion chip is formed (as shown in FIG. 2 ). In this embodiment, the distance between microphones is d=10cm.

The adjustable parallel beamformer group module, the fixed coefficient FIR filter module, the fixed coefficient signal blocking module, the adaptive noise canceller module, the sound source direction update module and other components all belong to the digital signal processing module. ARM9 S3C2440 microprocessor is used for software programming.

The connection between the microphone array and the microprocessor is as follows: the output signals of the five microphones in the microphone array are amplified by the 2-stage pre-amplification circuit composed of the operational amplifier shown in Figure 2, and then input to the multi-channel analog-to-digital conversion chip MAX118, and the S3C2440 microprocessor passes through IO ports GPB2, 3, 4 control the input channel terminals A1, A2, and A3 of MAX118, and control the read/write ports WR, RD of MAX118 through the timer output pins TOUT0, TOUT1 to perform analog-to-digital conversion with a sampling frequency of 16ksps. Lines DATA0 to DATA7 transmit the 8bit analog-to-digital conversion results to the S3C2440 microprocessor.

After the multi-channel voice signal analog-to-digital conversion in the microphone array voice enhancement device with sound source direction tracking function enters the microprocessor, the data and control flow connection mode between the digital signal processing modules running in the form of software are shown in Figure 3 , the specific description is as follows:

The adjustable beamformer group introduced by the microphone array speech enhancement device with sound source direction tracking function is 3 parallel beamformers with the same structure, and two methods can be used to carry out the module parameters of the adjustable beamformer group in the device Initialization: First, the front can be set as the default current sound direction, and the algorithm can gradually converge to the real current sound source direction after starting the tracking function; second, the sound source direction obtained by the microphone array positioning algorithm known in the art can be used as the current sound direction. Sound source direction, the algorithm will track the current sound source direction after starting the tracking function.

After the adjustable beamformer group module obtains the current sound source direction information in the initialization stage, it adjusts the step size Δ according to the set sound source direction to form a left and right offset direction of the sound source direction with a left and right offset of Δ, and three parallel beams are formed The beamformers are respectively aligned to the sound source direction and the left-right offset direction, that is, in each parallel beamformer, the input signals x _i (k )i=0,1,2,…,5 time delay compensation value τ _i (θ _j )i=0,1,2,…,5, j=0,1,2, thus obtained after delay compensation Each channel signal x _{i, j} (k), where i represents the channel number of the microphone array, θ _j represents the current sound source direction of the three adjustable beamformers, and the current sound source is offset left and right direction. In this embodiment, θ ₁ is the current sound source direction, θ ₀ and θ ₂ are respectively the left and right offset directions of the current sound source direction, and Δ is the beam adjustment step size. That is:

θ ₀ ＝θ ₁ -Δ, θ ₂ ＝θ ₁ +Δ

After delay compensation-addition beamforming, three-way microphone array beamforming signals y _j (k)j=0,1,2 respectively aligned to the current sound source direction and the left and right offset directions can be obtained

y _j (k) = A ^T X _i,j (k)

In this embodiment, simple delay compensation-addition beamforming is adopted, and the fixed coefficient A=[0.2, 0.2, 0.2, 0.2, 0.2].

The three parallel adjustable beamformers in the adjustable parallel beamformer group module are shown in Figure 4, each of which has two input terminals for signal input and sound source direction input, as well as voice signal output of each channel after delay adjustment, Beamforming outputs two outputs. Each adjustable beamformer can be used to form a voice beam to align with the input sound source direction, and 3 parallel adjustable beamformers can respectively align with the current sound source direction and its left and right offset directions.

As shown in Fig. 3, the beamforming signals output by the three parallel adjustable beamformers are respectively input into three same fixed-coefficient FIR filters. The coefficients of the FIR filters are obtained by calculating the expected frequency response in the direction of the sound source. The required frequency response is formed in the three beam alignment directions of the current sound source direction and the left and right offset directions.

y' _j (k) = F ^T Y (k)

Also as shown in Figure 3, the delay-adjusted speech signals of each channel output by the three parallel adjustable beamformers are respectively input into three identical signal blocking matrices, which are used to filter out the current sound source direction, left and right offset direction 3 Speech signals contained in beam-aligned directions. In this embodiment, the signal blocking matrix B is configured according to the following formula:

$\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccccc}\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

After being processed by the blocking matrix, the signal blocking output corresponding to the three beam directions is:

U _j (k) = B ^T X _i,j (k)

The three adaptive noise canceller modules shown in Figure 3 are equipped with two input terminals of reference noise and noise-containing signal, and take the output signals of the blocking matrix corresponding to the current sound source direction and the three beams in the left and right offset directions as reference Noise input signal, the output signal of fixed coefficient FIR filter corresponding 3 beams is used as noisy input signal, adopts LMS (minimum mean square error algorithm) adaptive algorithm known in the art to adjust the weight of adaptive noise canceller in the present embodiment The coefficients W _j,k are subjected to adaptive noise cancellation processing.

At this time, the output of the adaptive noise canceller corresponding to the three beam directions is:

n _j (k)=W _j,k ^T U _j (k)

After noise elimination processing, the system speech enhancement output corresponding to the three beam directions is the difference between the non-adaptive branch signal output of each beam and the output of the adaptive noise canceller:

s _j (k)=y' _j (k)-n _j (k)

After obtaining the system speech enhancement output s _j (k) corresponding to the three beam directions, the direction update module shown in Figure 3 uses the length L as the observation window length to compare the variance V _j of the speech enhancement output signals corresponding to the three beam directions, And select the beam direction with the smallest variance as the new current sound source direction to update the algorithm to minimize the residual noise in the speech signal. In this embodiment, L is 200, that is, the cost represented by the output signal variance is calculated once every 200 algorithm iterations The function performs an update of the current sound source direction.

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; min</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; min</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}\underset{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; k</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

At the same time, the direction update module can output the speech enhancement output s _J (k) of this method through the best sound source direction, and obtain the new current sound source direction θ _1new =θ _J .

As shown in Figure 3, after obtaining the new current sound source direction θ _1new , update the time delay compensation value τ _i (θ _jnew )i=0 of each channel according to the sound source direction and its left and right offset directions θ _0new , θ _2new ,1,2,...,5, j=0,1,2, continue algorithm iteration. In the iterative process of the algorithm, the tracking of the sound source direction and the speech enhancement are realized simultaneously through three adaptive noise cancellation operations aligned with the beam direction.

In Figure 3, the adjustable parallel beamformer group module consists of three parallel adjustable beamformers. The specific structure of each adjustable beamformer is shown in Figure 4. Each adjustable beamformer is equipped with two input terminals for signal input and sound source direction input, and output of each channel voice signal after delay adjustment, Beamforming outputs two outputs. After the voice data input from channels 0 to 4 is adjusted by the delay of each channel, one channel directly outputs the delay-adjusted data of each channel, and the other channel is superimposed on each channel to obtain the beamforming output. The channel adjustment delays of the three parallel adjustable beamformers are respectively obtained from the input sound source direction, and the input sound source direction plus or minus the beam adjustment step. Therefore, each adjustable beamformer can be used to form a voice beam to align with the input sound source direction, and the three parallel adjustable beamformers can respectively align with the current sound source direction and its left-right offset direction.

In the above working process, it is necessary to calculate the time delay compensation value of each channel of the microphone array according to the aligned sound source direction _θj . The principle is described below in conjunction with Figure 5:

As shown in Figure 5, in the embodiment of the present invention: the horizontal line where the 5-element microphone line array is located is the X-axis, and the position of the microphone m2 in the middle of the line array is used as the coordinate origin to establish a positioning coordinate system, and the distance between each array element of the line array is d, Then when the target azimuth angle is _θj , considering that the sound source a is in the far field range in the embodiment, the voice signal sent by it can be considered as a plane incident wave when it reaches the microphone line array, then the central array of the line array in this embodiment The element microphone m2 is used as a reference to calculate the corresponding delay compensation value, that is, the voice signal received by m2 is not compensated for delay, and the voice signal x _i received by each channel microphone in the line array can be compensated as follows according to the azimuth angle θ _j Calculation (as shown in Figure 5):

x _i,j (k) = x _i (k')

${\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; round</span>}<span\; class="notranslate">\; [</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; cos</span>}{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0,1,2,3,4</span>}$

Where i is the number of each channel in the line array, C is the speed of sound in the air (340m/s in this example), θ _j (j=0,1,2) represents the corresponding direction of the three adjustable beamformers The direction of the current sound source, the direction of the current sound source's left offset and right offset, f _s is the sampling frequency of the voice signal of the microphone array (the unit is Hz, 16000Hz in this embodiment), and round() represents the rounding operation. The voice signals of each channel are weighted and superimposed after the delay compensation corresponding to the three beams, which can be aligned with the current sound source direction and the left-right offset direction of the current sound.

The biggest feature of the microphone array speech enhancement device with sound source direction tracking and its method disclosed in the present invention is that the sound source direction can be adjusted simultaneously with the speech enhancement processing by means of three parallel adjustable beamformers combined with an adaptive noise canceller. Tracking, while allowing a certain error in the input sound source orientation during initialization, due to the tracking function of the sound source direction, it can avoid signal leakage caused by the mismatch of sound source orientation (DOA).

Claims (3)

1. be with the microphone array speech sound enhancement device of Sounnd source direction following function, it is characterized in that being provided with:

Microphone array is used for voice signal multichannel collecting, pre-process and analog to digital conversion;

Adjustable parallel Beam-former pack module is used for through adjusting the real-time follow-up that each channel time delay carries out Sounnd source direction;

Fixed coefficient FIR filter module is used for forming desired frequency characteristics at the non-self-adapting branch road and obtains signals and associated noises;

Fixed coefficient signal jam module is used for the signal that the filtering Sounnd source direction comprised and obtains noise signal;

Adaptive noise canceller module is used for offseting the noise that principle is eliminated signals and associated noises as a reference and comprised with noise signal according to adaptive noise, output voice enhancing signal;

The Sounnd source direction update module; Be used for carrying out the voice enhancing signal variance selection current optimal Sounnd source direction that secondary lobe is eliminated output, and the current optimal Sounnd source direction that obtains is inputed to the offset direction, the left and right sides that adjustable parallel Beam-former pack module upgrades current Sounnd source direction and current Sounnd source direction according to current expection Sounnd source direction and offset direction, current expection sound source left and right sides correspondence;

Each passage voice signal output terminal behind pre-amplification circuit and analog to digital converter, directly is connected with the signal input part of adjustable parallel Beam-former pack module through data line successively in the said microphone array;

Said adjustable parallel Beam-former pack module is provided with 3 parallel adjustable Beam-formers; Said 3 parallel adjustable Beam-formers are respectively equipped with signal input part and Sounnd source direction input end; Each signal input part connects each passage voice signal output terminal of analog to digital converter, and each Sounnd source direction input end then is connected with the current optimal Sounnd source direction output terminal of Sounnd source direction update module; The input end through adjusted each the passage voice signal output termination fixed coefficient signal jam module of time delay of said 3 parallel adjustable Beam-formers, the stack output of each passage voice signal of time delay adjustment back then is connected with the input end of fixed coefficient FIR filter module;

Said Adaptive Noise Canceler module is provided with reference noise input end and noisy speech input end; Fixed coefficient signal jam module is blocked the reference noise input end that output after the processing connects the Adaptive Noise Canceler module to input signal, and fixed coefficient FIR filter module carries out the noisy speech input end that output after the Filtering Processing connects the Adaptive Noise Canceler module to input signal;

The voice enhancing signal output termination Sounnd source direction update module of said Adaptive Noise Canceler module is to carry out the selection of current optimal Sounnd source direction; The Adaptive Noise Canceler module is provided with current optimal Sounnd source direction output terminal and voice enhancing signal output terminal; Current optimal Sounnd source direction output termination adjustable parallel Beam-former pack module, the corresponding voice enhancing signal of voice enhancing signal output terminal output current optimal Sounnd source direction.

2. the microphone array speech sound enhancement device of band Sounnd source direction following function as claimed in claim 1 is characterized in that said microphone array adopts the equidistant linear array of being made up of 5 yuan of microphones.

3. microphone array voice enhancement method is characterized in that adopting according to claim 1 with the microphone array speech sound enhancement device of Sounnd source direction following function, said method comprising the steps of:

1 initialization step: at initial phase the dead ahead is set and is the current audio direction of acquiescence, or import the Sounnd source direction that obtains by the microphone array location algorithm, as the initialization Sounnd source direction, as the Sounnd source direction parameter of adjustable Beam-former pack module;

1 adjustable wave beam forms step: according to the Sounnd source direction adjustment step-length of setting; Adding, subtract the adjustment step-length respectively through current Sounnd source direction produces the left and right offset direction of current Sounnd source direction and calculates the corresponding delay compensation value of each channel signal of microphone array; And to 3 parallel beams form each passage output signal of microphone array in the device carry out the counterparty to delay compensation, make it aim at Sounnd source direction and offset direction, the left and right sides respectively;

1 fixed coefficient filter step: utilize fixed coefficient respectively FIR filtering to be carried out in the output of 3 parallel beams formation devices, be used for forming the frequency response of needs at current Sounnd source direction, 3 the wave beam aligning directions in offset direction, the left and right sides;

1 signal jam step: at the self-adaptation branch road, the signal of 3 adjustable Beam-former outputs is imported 3 identical signal jam matrixes respectively, is used for the voice signal that comprises in the current Sounnd source direction of filtering, 3 the wave beam aligning directions in offset direction, the left and right sides;

1 adaptive noise removal process: at the self-adaptation branch road; With the corresponding blocking matrix output signal of 3 wave beams of current Sounnd source direction, offset direction, left and right sides noise signal as a reference, fixedly the output signal of corresponding 3 wave beams carries out adaptive noise and eliminates processing as the band input signal of making an uproar in the branch road;

1 Sounnd source direction step of updating: output poor that obtains the output of 3 beam direction non-self-adapting tributary signals and corresponding Adaptive Noise Canceler; And select the minimum beam direction of variance to upgrade in the view window as current Sounnd source direction, upgrade and add, subtract the adjustment step-length behind the current Sounnd source direction respectively and produce the left and right offset direction of current Sounnd source direction and calculate the corresponding delay compensation value of each channel signal of microphone array and carry out algorithm iteration once more.

Images