CN115866483A - Beam forming method and device for audio signal - Google Patents
Beam forming method and device for audio signal Download PDFInfo
- Publication number
- CN115866483A CN115866483A CN202211283723.8A CN202211283723A CN115866483A CN 115866483 A CN115866483 A CN 115866483A CN 202211283723 A CN202211283723 A CN 202211283723A CN 115866483 A CN115866483 A CN 115866483A
- Authority
- CN
- China
- Prior art keywords
- array
- audio
- radio
- signal
- sound receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Circuit For Audible Band Transducer (AREA)
Abstract
The invention discloses a method and equipment for forming beams of audio signals, which are used for improving the robustness and the pointing performance of a beam former at low frequency. The method comprises the following steps: determining an audio array signal output by an original audio signal through a sound receiving array according to the directional characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, the sound receiving devices have directional characteristics for collecting sound in a specific area, the directional characteristic is determined according to the respective directional characteristics of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directional characteristic; and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
Description
Technical Field
The present invention relates to the field of audio processing technologies, and in particular, to a method and an apparatus for forming a beam of an audio signal.
Background
The radio array is a group of radio devices arranged according to a specific shape structure, and common array types are a linear array and a circular array. The sound receiving devices used in the array are classified into omnidirectional sound receiving devices for picking up sound in all directions and directional sound receiving devices for picking up sound in a specific area according to the directivity of the sound receiving devices, and most of the research on the sound receiving array currently focuses on the omnidirectional sound receiving devices.
The audio signals are broadband signals, the existing beamformer for the audio signals is usually based on an omnidirectional radio array, spatial filtering is realized only by designing the weight of the beamformer, and the robust performance and the directivity of the beamformer obtained by the method at low frequency are obviously attenuated and need to be further improved.
Disclosure of Invention
The invention provides a beam forming method and device of audio signals, which are used for carrying out beam forming on the received audio signals by taking the directivity of a sound receiving array as a reference factor and improving the robustness performance and the pointing performance of a beam forming device at low frequency.
In a first aspect, an embodiment of the present invention provides a method for forming a beam of an audio signal, where the method includes:
determining an audio array signal output by an original audio signal through a sound receiving array according to the directional characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, the sound receiving devices have directional characteristics for collecting sound in a specific area, the directional characteristic is determined according to the respective directional characteristics of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directional characteristic;
and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
In the beamforming method provided in this embodiment, the directivity characteristic of the sound receiving array and the array structure are used to determine the output audio array signal, and the directivity characteristic of the sound receiving array is used as a reference factor, so that when beamforming is finally performed, beamforming is performed on the audio array signal including the directivity characteristic.
In a second aspect, an embodiment of the present invention further provides an apparatus for beamforming an audio signal, where the apparatus includes a processor and a memory, where the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and execute the following steps:
determining an audio array signal output by an original audio signal through a sound receiving array according to the directional characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, the sound receiving devices have directional characteristics for collecting sound in a specific area, the directional characteristic is determined according to the respective directional characteristics of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directional characteristic;
and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
In a third aspect, an embodiment of the present invention further provides a beam forming apparatus for an audio signal, including:
the sound receiving output unit is used for determining an audio array signal of an original audio signal output by the sound receiving array according to the directivity characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, the sound receiving devices have the directional characteristic of picking up sound in a specific area, the directivity characteristic is determined according to the directional characteristic of each of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directivity characteristic;
and the beam forming unit is used for performing beam forming on the audio array signal according to the weight vector and the filtering parameter vector of the sound receiving array, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range of filtering the audio array signal by using the filtering parameter.
In a fourth aspect, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, where the computer program is used to implement the steps of the method in the first aspect when the computer program is executed by a processor.
These and other aspects of the present application will be more readily apparent from the following description of the embodiments.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required to be used in the description of the embodiments will be briefly introduced below, and it is apparent that the drawings in the description below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings may be obtained according to the drawings without inventive labor.
Fig. 1 is a flowchart illustrating an implementation of a method for forming a beam of an audio signal according to an embodiment of the present invention;
fig. 2 is a flowchart illustrating an overall method for forming a beam of an audio signal according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an audio signal beam forming apparatus according to an embodiment of the present invention;
fig. 4 is a schematic diagram of an audio signal beam forming apparatus according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be described in further detail with reference to the accompanying drawings, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the embodiment of the present invention, the term "and/or" describes an association relationship of an associated object, and indicates that three relationships may exist, for example, a and/or B, and may indicate: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
The application scenario described in the embodiment of the present invention is for more clearly illustrating the technical solution of the embodiment of the present invention, and does not form a limitation on the technical solution provided in the embodiment of the present invention, and it can be known by a person skilled in the art that with the occurrence of a new application scenario, the technical solution provided in the embodiment of the present invention is also applicable to similar technical problems. In the description of the present invention, the term "plurality" means two or more unless otherwise specified.
Embodiment 1, the sound receiving array is a group of sound receiving devices arranged according to a specific shape structure, and common array types are a linear array and a circular array. The sound receiving devices used in the array are classified into omnidirectional sound receiving devices for picking up sound in all directions and directional sound receiving devices for picking up sound in a specific area according to the directivity of the sound receiving devices, and most of the research on the sound receiving array currently focuses on the omnidirectional sound receiving devices. The audio signals are broadband signals, the existing beamformer for the audio signals is usually based on an omnidirectional radio array, spatial filtering is realized only by designing the weight of the beamformer, and the robust performance and the directivity of the beamformer obtained by the method at low frequency are obviously attenuated and need to be further improved.
For example, a sound receiving array is taken as an example of a microphone array, and the microphone array is a system which is a group of microphones arranged according to a specific shape structure and is used for sampling and processing the spatial characteristics of a sound field. Common array types are linear and circular arrays. The directivity of a microphone means that the microphone picks up sound from different directions. In a field setting, it is most important to identify the type of microphone used in order to reduce the feedback of sound and to determine the best place to place the listening depending on the use of directivity. Microphones used in the array are classified into an omnidirectional microphone for picking up sound in all directions and a directional microphone for picking up sound in a specific area according to the directivity of the microphones. An omni-directional microphone has the same sensitivity for all angles, which means that it can pick up sound equally from all directions. An omni-directional microphone is a special form of directional microphone.
Currently, most of the research on microphone arrays focuses on omni-directional microphones. Compared with a single microphone, the microphone array has better spatial resolution capability and anti-interference capability, can obtain higher signal gain, and is widely applied to the fields of voice enhancement and the like. The beam forming based on the microphone array can form a beam in a specific direction by utilizing spatial information acquired by the array, so that a signal incident in the direction is received, interference in other directions is inhibited, and filtering in a spatial domain is realized. The audio signal is a broadband signal, a common time domain broadband beam former is a filter summation structure beam former, and the weight of the beam former is designed according to a certain criterion on the basis of predetermining the target sound source direction, so that the actual beam response approaches to a desired value, and the weight is not changed along with the change of data after the weight design is finished. The existing broadband beam former design is usually based on an omnidirectional microphone array, and only the weight of the beam former is designed to realize spatial filtering and improve frequency invariance, but the directivity of a directional microphone is not utilized. The beam former obtained by the method has obvious attenuation on the robustness and the directivity at low frequency, and needs to be further improved.
The core idea of the method for forming the sound wave beam provided by the embodiment of the invention is to determine the output sound wave array signal by using the directivity and the array structure of the sound receiving array, and to take the directivity of the sound receiving array as a reference factor, so that when the sound wave beam is formed finally, the sound wave beam forming is performed on the audio wave array signal containing the directivity, the spatial filtering performance of the sound receiving array is further improved, and the robustness and the pointing performance of the wave beam forming device at low frequency can be improved.
As shown in fig. 1, the implementation flow of the method for forming a beam of an audio signal provided in this embodiment is as follows:
in some embodiments, the sound receiving array may be an array of acoustic sensors, the sound receiving devices being acoustic sensors, including but not limited to microphones, for sampling and processing spatial characteristics of the sound field. Optionally, the sound receiving array is a microphone array, and the structure of the microphone array includes, but is not limited to, a linear array and a circular array. The preset structure in this embodiment may be a linear array or a circular array, which is not limited in this embodiment.
It should be noted that, the sound receiving array includes a plurality of sound receiving devices, and when receiving an original audio signal, each sound receiving device can receive the original audio signal, and because each sound receiving device has a directional characteristic of collecting sound to a specific area, the directional characteristic of the sound receiving device can be utilized to improve the directional performance of the beamformer.
In some embodiments, after receiving the original audio signal, the original audio signal may be further preprocessed to facilitate beamforming, specifically performing the following steps:
performing frame windowing processing and short-time Fourier transform on the original audio signals received by each radio device to obtain original frequency domain audio signals of each original audio signal;
the purpose of framing the original audio data is to divide a number of audio samples into a frame within which the characteristics of the audio data can be considered stable. The length of one frame should be less than that of one phoneme, the duration of the next phoneme at normal speech speed is about 50ms, the length of the general frame is about 10-40 ms, in order to make the frame have a smooth transition, there is a certain overlap between the general frame and the frame, and the time difference between the start positions of two adjacent frames is called frame shift. Spectral leakage is generally generated by non-periodic truncation of a signal (such as audio data), in order to avoid spectral leakage, window functions are introduced, different window functions have different mitigation degrees on the spectral leakage, and the total leakage is measured by equivalent noise bandwidth. The short-time stable characteristic of the signal is ensured by carrying out frame-dividing windowing processing on the original audio data, and the leakage of frequency spectrum energy is prevented.
In the implementation, taking the linear microphone array composed of K first-order directional microphones as an example, it is assumed that the original audio signal x (n) incident from the far field is composed of the target signal s0 (n) and/or the interference signal si (n), and the arrival direction of the target signal (i.e. the target direction) is known as θ 0 The direction of arrival of the interfering signal is theta i And theta i ≠θ 0 . The original audio signal received by the kth array element (i.e. the kth microphone) is represented as:
wherein x is k (n) denotes the original audio signal received at the kth array element (i.e. the kth microphone), s 0，k (n) is the target signal, s, received by the kth array element after the target signal is transmitted i，k And (n) is an interference signal received by the kth array element after the interference signal is transmitted. The received original audio signal is subjected to framing and windowing, and then is transformed into an original frequency domain audio signal by short-time Fourier transform, as shown in the following formula:
wherein m represents a frame index of a multi-frame audio signal obtained by framing an original audio signal, f represents a frequency of the audio signal, and X k (m, f) is x k (n) frequency domain audio signal obtained by short time Fourier transform, S 0，k (m, f) S0, k (n) a frequency domain target signal obtained by short-time Fourier transform, S i，k (m, f) is s i，k (n) performing short-time Fourier transform to obtain a frequency domain interference signal.
After the original frequency domain audio signals of the original audio signals are obtained through the steps, the frequency domain audio signals output by the corresponding radio equipment through the original frequency domain audio signals can be determined, so that the frequency domain expression of the audio array signals is determined according to the frequency domain audio signals output by the radio equipment, and the frequency domain audio array signals are obtained and are used for calculation of subsequent beam forming.
In implementation, since the sound receiving device in this embodiment has a directional characteristic, the sound receiving array composed of the sound receiving devices also has a directional characteristic, and the audio array signal output by the sound receiving array from the original audio signal also has a directional characteristic and can change with the change of the directional characteristic. Because the radio equipment has directivity, the radio equipment in the radio array can receive original audio signals from different directions, when the directivity characteristic of the radio array is changed, namely the directivity of the radio equipment is changed, the direction of the radio equipment for receiving the original audio signals is changed, and different receiving directions can cause that the interference signals and the target signals in the received original audio signals are different, and finally, the audio array signals output by the radio array are changed.
In some embodiments, the audio array signal output by the sound receiving array is determined by:
step a, determining a directional diagram and a transfer function of sound receiving equipment in the sound receiving array, wherein the directional diagram represents the directional characteristic of the sound receiving equipment, and the transfer function represents the arrangement position of the sound receiving equipment in the sound receiving array;
optionally, the directional diagram and the transfer function of the radio devices in the radio array are determined as follows:
a1 According to the directional characteristic of the radio equipment in the radio array and the angle of the radio equipment deviating from the symmetry axis of the radio array, determining the directional diagram of the radio equipment;
in the implementation, a linear microphone array composed of K first-order directional microphones is taken as an example, and the directional patterns of the microphones in the microphone array are represented as follows:
u (θ) = α + (1- α) cos (θ - Φ) formula (3);
wherein u (theta) represents a directional diagram, alpha represents microphone characteristics, alpha is different with different directional characteristics of the microphone, phi is the directional direction of the microphone, and theta is an off-axis angle, wherein the off-axis angle represents the angle of the microphone deviating from the symmetry axis of the microphone array.
Note that, when the directional characteristic of the microphone is omnidirectional, the directional pattern of the microphone is a fixed value of 1.
a2 According to the directional diagram and the omnidirectional transfer function of the radio equipment, determining the transfer function of the radio equipment, wherein the omnidirectional transfer function is calculated under the condition that the directional characteristic of the radio equipment is omnidirectional.
In the implementation, taking a linear microphone array composed of K first-order directional microphones as an example, for a far-field linear microphone array, when the directional direction of the microphone is the target direction (Φ = θ o), the transfer function of the kth microphone is expressed as:
wherein the content of the first and second substances,
representing the transfer function of the kth microphone, h k (f, theta) represents the omnidirectional transfer function when the pointing characteristic of the kth microphone is omnidirectional, namely the omnidirectional microphone, and when the microphone array is determined, the omnidirectional transfer function of the microphone is also determinedIt is determined that when the microphone array is known, the omnidirectional transfer function of the microphone as an omnidirectional microphone is also known;
j represents an imaginary number, d k The distance from the kth array element to a reference point, wherein the reference point is any point in a predefined sound receiving array. c is sound velocity, w is off-axis angle, f represents frequency of audio signal, u (theta) represents directional diagram, alpha represents microphone characteristic, theta 0 Is the direction of arrival of the target signal, i.e. the target direction.
In implementation, the transfer function vector pointing to the microphone array may be expressed as:
wherein the content of the first and second substances,
representing a transfer function vector directed at the microphone array, u (theta) = [ u 0 (θ)，...，u K-1 (θ)] T A directional pattern vector representing the microphone array, h (f, θ) = [ h = 0 (f，θ)，...，h K-1 (f，θ)]T,. Indicates a dot product. u. of 0 (theta) represents the directional diagram of the 1 st microphone, u K-1 (θ) represents a directivity pattern of the kth microphone; h0 (f, theta) represents the omnidirectional transfer function of the 1 st microphone, h K-1 (f, θ) represents the omnidirectional transfer function of the Kth microphone, θ is the off-axis angle, and f represents the frequency of the audio signal.
It should be noted that, when the orientation of the sound receiving device is changed, the expression of u (θ) may be changed accordingly, and when the geometric structure of the sound receiving array is changed, the expression of h (f, θ) may be changed accordingly.
B, determining an audio signal output by the original audio signal through the radio equipment according to the directional diagram and the transfer function of the radio equipment;
in the implementation, the product value of the directional diagram and the transfer function of the sound receiving equipment is determined as the audio signal of the original audio signal output by the sound receiving equipment.
And c, determining the audio array signals according to the audio signals output by the radio equipment in the radio array.
And determining the sum value obtained by adding the audio signals output by the radio equipment as an audio array signal. The audio array signal in this embodiment includes the audio signal output by each radio.
The purpose of beamforming is to satisfy the enhanced output signal Y (m, f)
S 0，ref (m, f) represents the received signal at a reference point, which is any point in a predetermined array of sound receptions. In implementation, the weight groups and the filtering parameter groups of each radio device in the radio array are utilized to filter audio signals received by each radio device in different frequency ranges, so that interference signals in the audio signals can be effectively filtered, and target signals in the audio signals are enhanced.
In implementation, in order to perform frequency domain calculation and perform beamforming on the audio array signal, the audio array signal is converted into a frequency domain audio array signal in a frequency domain form, then the frequency domain audio array signal is subjected to weighted summation by using a weight vector and a filtering parameter vector of the radio array to obtain a frequency domain audio beam signal, and then the frequency domain audio beam signal is subjected to inverse transformation to obtain a time domain audio beam signal.
In some embodiments, the audio array signal is beamformed by:
the process 1) determines a filtering weight vector according to the weight vector and the filtering parameter vector of the radio receiving array;
in implementation, the weight vector includes a weight group corresponding to each radio apparatus, each radio apparatus corresponds to one weight group, each weight group includes a plurality of weights, the filter parameter vector includes a filter parameter group corresponding to each radio apparatus, each radio apparatus corresponds to one filter parameter group, each filter parameter group includes a plurality of filter parameters, therefore, the filter weight group corresponding to each radio apparatus is determined by using the product of the weight group corresponding to each radio apparatus and the filter parameter group, the filter weight group of each radio apparatus forms a filter weight vector, that is, the filter weight vector includes a filter weight group corresponding to each radio apparatus, each radio apparatus corresponds to one filter weight group, and each filter weight group includes a plurality of filter weights.
And 2) carrying out weighted summation on the audio array signals according to the filtering weight vector to obtain audio beam signals, and outputting the audio beam signals.
In implementation, the filtering weight vector includes a filtering weight set corresponding to each radio device, each radio device corresponds to a filtering weight set, and the audio array signal includes audio signals output by each radio device; the audio signal output by each sound receiving device corresponds to a filtering weight set of the sound receiving device.
Optionally, the audio beam signal is determined specifically through the following procedure:
step 2a, determining a filtering weight set corresponding to each radio device according to the weight set corresponding to each radio device and the filtering parameter;
optionally, the product value of the weight set of each radio receiver and the filter parameter set may be used as the filter weight set of the radio receiver; when the weight group and the filter parameter group are multiplied, the multiplication is performed according to the arrangement position of the elements in each group, for example, when the weight group is [ weight 1, weight 2], and the filter parameter group is [ filter parameter 1, filter parameter 2], the product value obtained by multiplying the weight group and the filter parameter group is [ (weight 1) × (filter parameter 1), (weight 2) × (filter parameter 2) ].
And 2b, carrying out weighted summation on the audio signals output by each radio equipment according to the filtering weight set corresponding to each radio equipment to obtain the audio wave beam signals.
In implementation, the original audio signals received by each radio equipment are subjected to framing and windowing processing to obtain processed audio signals corresponding to each radio equipment;
carrying out short-time Fourier transform on the processed audio signals corresponding to the radio equipment to obtain frequency domain audio signals corresponding to the radio equipment;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the frequency domain audio signals corresponding to each radio receiving device to obtain the audio wave beam signals.
Taking a linear microphone array composed of K first-order directional microphones as an example, the beam forming method in this embodiment may be used as a beam former, and after receiving an original audio signal, an output beam response is an audio beam signal, which is expressed as follows:
where P' (f, θ) is the audio beam signal, w k，l Set of weights for the kth radio receiver, e l (f)＝exp{-j2πfl/f s As the filter parameters, take FIR filter as an example, e l (f) Representing the FIR filter delay factor, L being the length of the FIR filter, f s Representing the sampling frequency of the audio signal, theta is an off-axis angle, f represents the frequency of the audio signal, and j represents an imaginary number;
u (θ) denotes the directional diagram, h k (f, theta) represents that the pointing characteristic of the kth microphone is an omnidirectional transfer function when the omnidirectional microphone is an omnidirectional microphone; l represents an integer from 0 to L-1, K represents an integer from 0 to K-1, and K represents the number of sound-receiving devices.
Expression (6) is expressed in vector form as follows:
P(f，θ)＝w T [v(θ)⊙g(f，θ)]formula (7);
wherein P (f, θ) represents a vector of the audio beam signal P' (f, θ), i.e., an audio beam vector;
w＝[w 0，0 ，w 0，1 ，...，w K-1，L-1 ] T is the weight vector of the radio array;
e(f)＝[e 0 (f)，...，e L-1 (f)] T denotes a filtered parameter vector, where e l (f)＝exp{-j2πfl/f s L is the length of the FIR filter, f s Denotes a sampling frequency for the audio signal, f denotes a frequency of the audio signal, L denotes an integer from 0 to L-1, and j denotes an imaginary number.
In some embodiments, the present embodiment determines the weight vector of the sound receiving array by:
step 1) determining an initial weight vector of the radio array;
step 2) according to the initial weight vector and the filtering parameter vector, carrying out beam forming on the audio array signal, and determining an audio beam signal after beam forming;
step 3) optimizing the initial weight vector based on a preset constraint condition of the audio wave beam signal to obtain an optimal weight vector;
optionally, determining a constraint condition in a real number domain based on a constraint on the audio beam signal that is invariant in frequency; and optimizing the initial weight vector based on the constraint condition of the real number domain to obtain an optimal weight vector, wherein the optimal weight vector is the weight vector of the radio array with the directivity characteristic.
And 4) determining the optimal weight vector as the weight vector of the radio array.
In some embodiments, to achieve a comparable enhancement of signals at different frequencies, existing frequency invariant beamformer designs based on spatial response variation constraints are considered for application to directional microphone arrays. Taking an example of a linear microphone array composed of K first-order directional microphones, based on a constraint condition that the frequency of an initial audio beam signal is not changed, the constraint condition is expressed as:
wherein, s.t.P (f) r ，θ 0 ) =1 represents P (f) is satisfied r ，θ 0 ) Condition of = 1;
f r for a frequency invariant constrained reference frequency, Ω is the frequency range, Θ = [0, π]Angle of arrival range, Θ SL For the set beam side lobe range, β is the weight of the balanced frequency invariance and the side lobe level; theta.theta. 0 The arrival direction of the target signal is shown, and theta is an off-axis angle;
p (f, θ) represents a vector of the audio beam signal P' (f, θ), i.e., an audio beam vector; p (f) r And θ) represents a reference beam vector of the reference frequency.
Taking the formula (8) as a preset constraint condition for the audio beam vector P' (f, θ), optimizing the initial weight vector, and solving a final optimal weight vector, wherein in the implementation, the complex domain optimization problem of the formula (8) is rewritten into a real domain optimization problem, and can be obtained:
in the formula (9), W represents an initial weight vector to be optimized;
in the formulae (10) and (11), U (θ) = v (θ) v T (θ)，
1 L×2 Is an L x 2 dimensional all 1 matrix. />
Directional pattern vector, u (theta), representing microphone array 0 ) A directional pattern vector representing a directional direction of the microphone array toward the target direction;
matrices in equations (10) and (11):
all are the corresponding matrixes obtained under the condition that the sound receiving matrix is an omnidirectional microphone matrix, wherein,
denotes a steering vector of the beamformer when the microphone is an omni-directional microphone, h (f, θ) = [ h0 (f, θ) ], h K-1 (f，θ)] T Representing the transfer function vector, h, of the microphone array when the directivity of the microphone array is omni-directional 0 (f, theta) represents the omnidirectional transfer function of the 1 st microphone, h K-1 (f, theta) represents the omnidirectional transfer function of the Kth microphone, theta is an off-axis angle, and f represents the frequency of the audio signal;
e(f)＝[e 0 (f)，...，e L-1 (f)] T denotes a filtered parameter vector, where e l (f)＝exp{-j2πfl/f s L is the length of the FIR filter, f s Denotes a sampling frequency for the audio signal, f denotes a frequency of the audio signal, L denotes an integer from 0 to L-1, and j denotes an imaginary number;
f r reference frequency, g (f), which is a frequency invariant constraint r θ) represents a reference steering vector of the beamformer when the microphone is omni-directional at the reference frequency;
wherein the content of the first and second substances,
represents a real part, <' > based on>
Representing the imaginary part.
The closed-form solution of equation (9) can be found from the lagrange multiplier method as:
for the explanation of the variables in equation (12), reference may be made to the descriptions of equations (9) to (11), which are not described herein again.
Taking a linear microphone array consisting of K first-order directional microphones and using the frequency invariant design of a filtering and summing structure beam former as an example, the optimal weight vector and the filtering parameter vector obtained by solving in the formula (12) are utilized to carry out weighted summation on the audio signal output by each radio equipment, and the audio beam signal is output, specifically as follows:
wherein, Y (m, f) represents the audio beam signal output after the beam forming is carried out on the original audio signal received by the sound receiving array, and w k，l Represents the weight group corresponding to the kth radio equipment, e l (f) Denotes the set of filter parameters, X, corresponding to the k-th radio device k (m, f) represents a frequency domain audio signal of an original audio signal output via a k-th sound receiving apparatus.
As shown in fig. 2, the present embodiment provides an overall flowchart of a method for forming a beam of an audio signal, and the specific flowchart is as follows:
the weight vector comprises a weight group corresponding to each radio device, the filtering parameter vector comprises a filtering parameter group corresponding to each radio device, and the filtering weight vector comprises a filtering weight group corresponding to each radio device.
And step 207, performing weighted summation on the frequency domain audio signals output by each radio equipment according to the filtering weight vector to obtain audio beam signals, and outputting the audio beam signals.
Determining a filtering weight set corresponding to each radio equipment according to the weight set corresponding to each radio equipment and the filtering parameter; and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the frequency domain audio signals output by each radio receiving device to obtain the audio wave beam signals.
The steps 200 to 203 and the steps 204 to 206 may be executed in parallel or sequentially, and the specific order of execution is not limited in this embodiment.
The beamforming method provided by this embodiment uses a directional radio device with directivity to further improve the spatial filtering performance of the array, and may improve the robustness and the directional performance of the filter-sum structure beamformer at low frequencies. The frequency invariance design of the filter and sum structure beam former by using the directional microphone array further improves the frequency invariance, and meanwhile, the weight value does not need to be changed once the design is finished. The beamforming method provided by the present embodiment is applicable to any directional microphone and any array geometry.
Embodiment 2, based on the same inventive concept, an embodiment of the present invention further provides a beam forming device for an audio signal, and as the device is a device in the method in the embodiment of the present invention, and the principle of the device to solve the problem is similar to that of the method, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 3, the apparatus comprises a processor 300 and a memory 301, wherein the memory 301 is used for storing programs executable by the processor 300, and the processor 300 is used for reading the programs in the memory 301 and executing the following steps:
determining an audio array signal output by an original audio signal through a sound receiving array according to the directional characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, the sound receiving devices have directional characteristics for collecting sound in a specific area, the directional characteristic is determined according to the respective directional characteristics of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directional characteristic;
and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
As an alternative implementation, the processor 300 is specifically configured to perform:
determining a directional diagram and a transfer function of the sound receiving equipment in the sound receiving array, wherein the directional diagram represents the directional characteristic of the sound receiving equipment, and the transfer function represents the arrangement position of the sound receiving equipment in the sound receiving array;
determining an audio signal output by the original audio signal through the radio equipment according to the directional diagram and the transfer function of the radio equipment;
and determining the audio array signals according to the audio signals output by each radio device in the radio array.
As an alternative implementation, the processor 300 is specifically configured to perform:
determining a directional diagram of the radio equipment according to the directional characteristic of the radio equipment in the radio array and the angle of the radio equipment deviating from the symmetry axis of the radio array;
and determining the transfer function of the radio equipment according to the directional diagram and the omnidirectional transfer function of the radio equipment, wherein the omnidirectional transfer function is obtained by calculation under the condition that the directional characteristic of the radio equipment is omnidirectional.
As an alternative embodiment, the processor 300 is specifically configured to perform:
determining a filtering weight vector according to the weight vector and the filtering parameter vector of the radio receiving array;
and according to the filtering weight vector, carrying out weighted summation on the audio array signal to obtain an audio wave beam signal, and outputting the audio wave beam signal.
As an optional implementation manner, the weight vector includes a weight set corresponding to each radio apparatus, the filter parameter vector includes a filter parameter set corresponding to each radio apparatus, the filter weight vector includes a filter weight set corresponding to each radio apparatus, and the audio array signal includes an audio signal output by each radio apparatus;
the processor 300 is specifically configured to perform:
determining a filtering weight set corresponding to each radio equipment according to the weight set corresponding to each radio equipment and the filtering parameter set;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the audio signals output by each radio receiving device to obtain the audio wave beam signals.
As an alternative embodiment, the processor 300 is specifically configured to perform:
performing frame windowing processing and short-time Fourier transform on the original audio signals received by each radio device to obtain original frequency domain audio signals of each original audio signal;
determining a frequency domain audio signal output by the original frequency domain audio signal through corresponding radio equipment;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the frequency domain audio signals output by each radio receiving device to obtain the audio wave beam signals.
As an optional implementation, the processor 300 is specifically configured to determine the weight vector of the sound reception array by:
determining an initial weight vector of the radio receiving array;
according to the initial weight vector and the filtering parameter vector, carrying out beam forming on the audio array signal, and determining an audio beam signal after beam forming;
optimizing the initial weight vector based on a preset constraint condition of the audio wave beam signal to obtain an optimal weight vector;
and determining the optimal weight vector as the weight vector of the radio array.
As an alternative implementation, the processor 300 is specifically configured to perform:
determining a real number domain constraint condition based on the frequency invariant constraint of the audio wave beam signals;
and optimizing the initial weight vector based on the constraint condition of the real number domain to obtain an optimal weight vector, wherein the optimal weight vector is the weight vector of the radio array with the directivity characteristic.
Embodiment 3, based on the same inventive concept, an embodiment of the present invention further provides a beam forming apparatus for an audio signal, and since the apparatus is an apparatus in the method in the embodiment of the present invention, and the principle of the apparatus to solve the problem is similar to the method, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not repeated.
As shown in fig. 4, the apparatus includes:
the sound receiving output unit 400 is configured to determine, according to a directivity characteristic and an array structure of a sound receiving array, an audio array signal output by the sound receiving array through an original audio signal, where the sound receiving array includes a plurality of sound receiving devices arranged in a preset structure, the sound receiving devices have a directional characteristic of picking up sound in a specific area, the directivity characteristic is determined according to the directional characteristic of each of the plurality of sound receiving devices, and the audio array signal changes with a change of the directivity characteristic;
and a beam forming unit 401, configured to perform beam forming on the audio array signal according to the weight vector and the filtering parameter vector of the radio receiving array, and determine and output a beam-formed audio beam signal, where the weight vector represents a frequency range in which the audio array signal is filtered by using the filtering parameter.
As an optional implementation manner, the sound reception output unit 400 is specifically configured to:
determining a directional diagram and a transfer function of sound receiving equipment in the sound receiving array, wherein the directional diagram represents the directional characteristic of the sound receiving equipment, and the transfer function represents the arrangement position of the sound receiving equipment in the sound receiving array;
determining an audio signal output by the original audio signal through the radio equipment according to the directional diagram and the transfer function of the radio equipment;
and determining the audio array signals according to the audio signals output by each radio device in the radio array.
As an optional implementation manner, the sound reception output unit 400 is specifically configured to:
determining a directional diagram of the radio equipment according to the directional characteristic of the radio equipment in the radio array and the angle of the radio equipment deviating from the symmetry axis of the radio array;
and determining the transfer function of the radio equipment according to the directional diagram and the omnidirectional transfer function of the radio equipment, wherein the omnidirectional transfer function is obtained by calculation under the condition that the directional characteristic of the radio equipment is omnidirectional.
As an optional implementation manner, the beam forming unit 401 is specifically configured to:
determining a filtering weight vector according to the weight vector and the filtering parameter vector of the radio receiving array;
and according to the filtering weight vector, carrying out weighted summation on the audio array signal to obtain an audio wave beam signal, and outputting the audio wave beam signal.
As an optional implementation manner, the weight vector includes a weight set corresponding to each radio apparatus, the filter parameter vector includes a filter parameter set corresponding to each radio apparatus, the filter weight vector includes a filter weight set corresponding to each radio apparatus, and the audio array signal includes an audio signal output by each radio apparatus;
the beam forming unit 401 is specifically configured to:
determining a filtering weight set corresponding to each radio equipment according to the weight set corresponding to each radio equipment and the filtering parameter set;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the audio signals output by each radio receiving device to obtain the audio wave beam signals.
As an optional implementation manner, the beam forming unit 401 is specifically configured to:
performing frame windowing processing and short-time Fourier transform on the original audio signals received by each radio device to obtain original frequency domain audio signals of each original audio signal;
determining a frequency domain audio signal output by the original frequency domain audio signal through corresponding radio equipment;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the frequency domain audio signals output by each radio receiving device to obtain the audio wave beam signals.
As an optional implementation manner, the beam forming unit 401 specifically determines the weight vector of the radio reception array by:
determining an initial weight vector of the radio receiving array;
according to the initial weight vector and the filtering parameter vector, carrying out beam forming on the audio array signal, and determining an audio beam signal after beam forming;
optimizing the initial weight vector based on a preset constraint condition of the audio wave beam signal to obtain an optimal weight vector;
and determining the optimal weight vector as the weight vector of the radio array.
As an optional implementation manner, the beam forming unit 401 is specifically configured to:
determining a real number domain constraint condition based on the frequency invariant constraint of the audio wave beam signals;
and optimizing the initial weight vector based on the constraint condition of the real number domain to obtain an optimal weight vector, wherein the optimal weight vector is the weight vector of the radio array with the directivity characteristic.
Based on the same inventive concept, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, which when executed by a processor implements the following steps:
receiving an original audio signal through a sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices which are arranged according to a preset structure, and the sound receiving devices have a directional characteristic of picking up sound in a specific area;
determining an audio array signal of the original audio signal output by the sound receiving array according to the directivity and the array structure of the sound receiving array, wherein the directivity of the sound receiving array is determined according to the respective directional characteristics of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directivity;
and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. A method of beamforming an audio signal, the method comprising:
determining an audio array signal output by an original audio signal through a sound receiving array according to the directivity characteristic and the array structure of the sound receiving array, wherein the sound receiving array comprises a plurality of sound receiving devices arranged according to a preset structure, the sound receiving devices have the directional characteristic of picking up sound in a specific area, the directivity characteristic is determined according to the respective directional characteristic of the plurality of sound receiving devices, and the audio array signal changes along with the change of the directivity characteristic;
and according to the weight vector and the filtering parameter vector of the sound receiving array, carrying out beam forming on the audio array signal, and determining and outputting the audio beam signal after beam forming, wherein the weight vector represents the frequency range for filtering the audio array signal by using the filtering parameter.
2. The method as claimed in claim 1, wherein the determining the audio array signal of the original audio signal output through the sound receiving array according to the directivity characteristic and the array structure of the sound receiving array comprises:
determining a directional diagram and a transfer function of the sound receiving equipment in the sound receiving array, wherein the directional diagram represents the directional characteristic of the sound receiving equipment, and the transfer function represents the arrangement position of the sound receiving equipment in the sound receiving array;
determining an audio signal output by the original audio signal through the radio equipment according to the directional diagram and the transfer function of the radio equipment;
and determining the audio array signal according to the audio signals output by the radio equipment in the radio array.
3. The method of claim 2, wherein the determining the directivity pattern and transfer function of the sound receiving devices in the sound receiving array comprises:
determining a directional diagram of the radio equipment according to the directional characteristic of the radio equipment in the radio array and the angle of the radio equipment deviating from the symmetry axis of the radio array;
and determining a transfer function of the radio equipment according to the directional diagram and the omnidirectional transfer function of the radio equipment, wherein the omnidirectional transfer function is obtained by calculation under the condition that the directional characteristic of the radio equipment is omnidirectional.
4. The method according to claim 1, wherein the beamforming the audio array signal according to the weight vector and the filtering parameter vector of the radio reception array, and determining and outputting a beamformed audio beam signal comprises:
determining a filtering weight vector according to the weight vector and the filtering parameter vector of the radio receiving array;
and according to the filtering weight vector, carrying out weighted summation on the audio array signal to obtain an audio wave beam signal, and outputting the audio wave beam signal.
5. The method of claim 4, wherein the weight vector comprises a set of weights corresponding to each radio, the filter parameter vector comprises a set of filter parameters corresponding to each radio, the filter weight vector comprises a set of filter weights corresponding to each radio, and the audio array signal comprises audio signals output by each radio;
determining a filtering weight set corresponding to each radio equipment according to the weight set corresponding to each radio equipment and the filtering parameter set;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the audio signals output by each radio receiving device to obtain the audio wave beam signals.
6. The method of claim 5, wherein the weighted summation of the audio signals output by the sound receivers according to the corresponding filtering weight set of each sound receiver to obtain the audio beam signal comprises:
performing frame windowing processing and short-time Fourier transform on the original audio signals received by each radio device to obtain original frequency domain audio signals of each original audio signal;
determining a frequency domain audio signal output by the original frequency domain audio signal through corresponding radio equipment;
and according to the filtering weight set corresponding to each radio receiving device, carrying out weighted summation on the frequency domain audio signals output by each radio receiving device to obtain the audio wave beam signals.
7. The method of claim 1, wherein the weight vector of the radio array is determined by:
determining an initial weight vector of the radio receiving array;
according to the initial weight vector and the filtering parameter vector, carrying out beam forming on the audio array signal, and determining an audio beam signal after beam forming;
optimizing the initial weight vector based on a preset constraint condition of the audio wave beam signal to obtain an optimal weight vector;
and determining the optimal weight vector as the weight vector of the radio array.
8. The method according to claim 7, wherein the optimizing the initial weight vector based on the preset constraint condition on the initial audio beam signal to obtain an optimal weight vector comprises:
determining a real number domain constraint condition based on the frequency invariant constraint of the audio wave beam signals;
and optimizing the initial weight vector based on the constraint condition of the real number domain to obtain an optimal weight vector, wherein the optimal weight vector is the weight vector of the radio array with the directivity characteristic.
9. An apparatus for beamforming an audio signal, the apparatus comprising a processor and a memory, the memory storing a program executable by the processor, the processor being configured to read the program from the memory and to perform the steps of the method according to any of claims 1 to 8.
10. A computer storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211283723.8A CN115866483A (en) | 2022-10-20 | 2022-10-20 | Beam forming method and device for audio signal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211283723.8A CN115866483A (en) | 2022-10-20 | 2022-10-20 | Beam forming method and device for audio signal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115866483A true CN115866483A (en) | 2023-03-28 |
Family
ID=85661653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211283723.8A Pending CN115866483A (en) | 2022-10-20 | 2022-10-20 | Beam forming method and device for audio signal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115866483A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117935835A (en) * | 2024-03-22 | 2024-04-26 | 浙江华创视讯科技有限公司 | Audio noise reduction method, electronic device and storage medium |
-
2022
- 2022-10-20 CN CN202211283723.8A patent/CN115866483A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117935835A (en) * | 2024-03-22 | 2024-04-26 | 浙江华创视讯科技有限公司 | Audio noise reduction method, electronic device and storage medium |
| CN117935835B (en) * | 2024-03-22 | 2024-06-07 | 浙江华创视讯科技有限公司 | Audio noise reduction method, electronic device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109102822B (en) | Filtering method and device based on fixed beam forming | |
| US11539846B1 (en) | Conferencing device with microphone beamforming and echo cancellation | |
| US9591404B1 (en) | Beamformer design using constrained convex optimization in three-dimensional space | |
| US7366662B2 (en) | Separation of target acoustic signals in a multi-transducer arrangement | |
| CN104936091B (en) | Intelligent interactive method and system based on circular microphone array | |
| US8965002B2 (en) | Apparatus and method for enhancing audio quality using non-uniform configuration of microphones | |
| EP2647221B1 (en) | Apparatus and method for spatially selective sound acquisition by acoustic triangulation | |
| US8098844B2 (en) | Dual-microphone spatial noise suppression | |
| US9100734B2 (en) | Systems, methods, apparatus, and computer-readable media for far-field multi-source tracking and separation | |
| CN109616136B (en) | Adaptive beam forming method, device and system | |
| CN111025233A (en) | Sound source direction positioning method and device, voice equipment and system | |
| EP1571875A2 (en) | A system and method for beamforming using a microphone array | |
| US9521486B1 (en) | Frequency based beamforming | |
| CN108447499B (en) | Double-layer circular-ring microphone array speech enhancement method | |
| Derkx et al. | Theoretical analysis of a first-order azimuth-steerable superdirective microphone array | |
| CN115866483A (en) | Beam forming method and device for audio signal | |
| Benesty et al. | Array beamforming with linear difference equations | |
| Frank et al. | Constant-beamwidth kronecker product beamforming with nonuniform planar arrays | |
| CN114073106B (en) | Binaural beamforming microphone array | |
| Leng et al. | A new method to design steerable first-order differential beamformers | |
| Markovich‐Golan et al. | Spatial filtering | |
| CN113491137B (en) | Flexible differential microphone array with fractional order | |
| US11902755B2 (en) | Linear differential directional microphone array | |
| CN115547354A (en) | Beam forming method, device and equipment | |
| Atkins et al. | Robust superdirective beamformer with optimal regularization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |