CN112714383A - Microphone array setting method, signal processing device, system and storage medium - Google Patents
Microphone array setting method, signal processing device, system and storage medium Download PDFInfo
- Publication number
- CN112714383A CN112714383A CN202011615054.0A CN202011615054A CN112714383A CN 112714383 A CN112714383 A CN 112714383A CN 202011615054 A CN202011615054 A CN 202011615054A CN 112714383 A CN112714383 A CN 112714383A
- Authority
- CN
- China
- Prior art keywords
- microphone
- point
- free
- vector
- microphone unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
The application discloses a method for setting a microphone array, a signal processing device, a system and a storage medium, wherein the microphone array comprises a plurality of microphone units which are arranged in an array mode and a filter which is connected with each microphone, and the method for setting the microphone array comprises the following steps: acquiring non-free field guide vectors of each microphone unit in multiple directions under the action of an acoustic structure at the periphery of the microphone array; calculating filter coefficients corresponding to the microphone units based on the non-free field oriented vector and the target direction; and filtering the filter corresponding to each microphone unit based on the filter coefficient. The method is used for achieving the purpose of improving the filtering effect and further enhancing the target signal.
Description
Technical Field
The present disclosure relates to the field of array signal processing technologies, and in particular, to a method for setting a microphone array, a signal processing apparatus, a system, and a storage medium.
Background
With the development of voice communication and human-computer interaction technologies, the requirement of far-field sound pickup is increasingly urgent, and the microphone array technology is also increasingly paid attention by related researchers as an important means for realizing high-quality far-field sound pickup. The microphone array technology mainly relates to a beam forming algorithm and microphone array hardware equipment which are mutually dependent to form a microphone array system.
The beam forming algorithm essentially filters signals collected by each microphone unit through a group of filters, so that sound waves in a target direction are superposed in phase, the amplitude is enhanced, sound waves in a non-target direction are offset in opposite phase, and the amplitude is suppressed, thereby achieving the purposes of eliminating noise and interference and enhancing target signals. Because of the spacing between the microphone units of the array, the time delay of the incident sound waves arriving at the microphone units in different directions is different, and the sound wave signals received by each microphone unit are phase-different in frequency domain.
In the current beam design method (i.e., the filter coefficient setting method), a free-field far-field plane wave is mostly used as a discussion premise, and it is only assumed that the sound wave propagates in a straight line, and is not attenuated, blocked, reflected, scattered, and the like. Therefore, the phase difference between the microphone units can be simply expressed by mathematical analysis according to the relative position relation of the microphone units, and the subsequent beam design process is introduced.
However, the microphone array is generally disposed inside a product, and an acoustic structure formed by product components on the periphery of the microphone array affects the propagation of sound waves, so that filter coefficients set based on the existing beam design method cannot obtain satisfactory filtering results.
Disclosure of Invention
The application mainly solves the problem of providing a microphone array setting method, a signal processing device, a system and a storage medium. The method is used for achieving the purpose of improving the filtering effect and further enhancing the target signal.
In order to solve the above technical problem, the first technical solution adopted by the present application is: there is provided a microphone array placement method including a plurality of microphone units arranged in an array, the method including: acquiring non-free field guide vectors of each microphone unit in multiple directions under the action of an acoustic structure at the periphery of the microphone array; and calculating filter coefficients corresponding to the microphone units based on the non-free field guide vector and the target direction, and filtering the filters corresponding to the microphone units based on the filter coefficients.
Wherein the non-free field steering vectors of each microphone unit in a plurality of directions are calculated by: three-dimensionally modeling an acoustic structure to obtain an acoustic model; based on the acoustic model, calculating the transmission function of sound waves from multiple directions to each microphone unit under the action of the acoustic structure; non-free-field steering vectors in a plurality of directions for each microphone unit are calculated based on the transfer function.
The step of calculating the transfer function of the sound wave reaching each microphone unit from a plurality of directions under the action of the acoustic structure based on the acoustic model comprises the following steps: determining the relative position relationship between each microphone unit and the acoustic structure; setting a plurality of first coordinate points at different positions on the periphery of the acoustic model, and setting a plurality of second coordinate points corresponding to the microphone units in the acoustic model based on the relative positional relationship; setting a sound source point on one of the first coordinate point and the second coordinate point, and setting a receiving point on the other one of the first coordinate point and the second coordinate point; and calculating frequency response data of sound waves generated by each sound source point reaching each receiving point, and further obtaining the transmission function reaching each microphone unit from multiple directions, wherein the frequency response data of each receiving point comprises response characterization values of the receiving points to the sound waves at multiple frequency sampling points.
The sound source point is arranged on the second coordinate point, and the receiving point is arranged on the first coordinate point.
Wherein the step of calculating non-free-field steering vectors for each microphone unit in a plurality of directions based on the transfer function comprises: selecting a reference microphone unit from the microphone units; calculating the ratio of the response characteristic value of each microphone unit under the same frequency sampling point and the response characteristic value of the reference microphone unit aiming at each direction; and respectively taking the ratios of a plurality of frequency sampling points in the same direction corresponding to each microphone unit as vector elements of the non-free field oriented vector in the corresponding direction.
Wherein the ratio is a ratio between the response characteristic value of each microphone unit and a summation result of the response characteristic value of the reference microphone unit and a non-zero constant.
Wherein the step of calculating non-free-field steering vectors for the microphone units in the plurality of directions based on the transfer function further comprises: and carrying out smooth correction on the non-free field guide vector.
Wherein the step of smoothly modifying the non-free field steering vector comprises: arranging a sliding window sliding along the frequency; the vector elements within the sliding window are averaged to serve as the vector element for the center point of the sliding window.
Wherein the step of calculating the filter coefficients corresponding to each microphone unit based on the non-free-field steering vector and the target direction comprises: aiming at a specified frequency point in frequency sampling points, substituting a guide vector matrix corresponding to the specified frequency point and a target direction into an optimization function and a constraint function, wherein matrix elements in the guide vector matrix are vector elements corresponding to the target direction and the specified frequency point in a non-free field guide vector of each microphone unit; and taking the optimization function as a target in a maximized mode, taking the constraint function as a constraint condition, and calculating filter coefficients corresponding to the microphone units in an iterative mode, so that the power ratio of a target signal corresponding to a target direction and the extended field noise output by the microphone array is maximized after the filter coefficients act.
Wherein the optimization function comprises:
wherein L (ω) is the optimization function, ω is the designated frequency point, W (ω) is a filter coefficient matrix at the designated frequency point, wherein matrix elements in the filter coefficient matrix are filter coefficients of each of the microphone units at the designated frequency point, W (ω)H(ω) is the conjugate transpose of W (ω),
for said steering vector matrix, θ0Gamma is a preset diffusion field noise coherent matrix for the target direction;
the constraint function includes:
where 1 is the unit vector, σthrIs a white noise gain lower limit constraint.
The method comprises the following steps of obtaining non-free field guide vectors of each microphone unit in multiple directions under the action of an acoustic structure on the periphery of a microphone array, wherein the step of obtaining the non-free field guide vectors of each microphone unit in the multiple directions comprises the following steps: pre-stored non-free-field steering vectors are retrieved from memory.
Before the step of calculating the filter coefficient corresponding to each microphone unit based on the non-free-field oriented vector and the target direction, the method further includes: and tracking the target to be acquired, and generating the target direction in real time according to the tracking result.
In order to solve the above technical problem, the second technical solution adopted by the present application is: there is provided a signal processing apparatus comprising a memory and a processor coupled to each other, the memory having stored therein program instructions, the processor being configured to execute the program instructions to implement the method of any of the above.
In order to solve the above technical problem, the third technical solution adopted by the present application is: a signal processing system is provided, which comprises a microphone array and the signal processing device.
In order to solve the above technical problem, a fourth technical solution adopted by the present application is: there is provided a storage medium storing program instructions executable by a processor for implementing any of the above methods.
The beneficial effect of this application is: according to the method and the device, on the premise that the incident microphone array is a non-free field influenced by a peripheral acoustic structure, non-free field guide vectors of all microphone units in multiple directions are calculated in advance, filter coefficients corresponding to all microphone units are further calculated in combination with a target direction, the filtering effect is improved, and then a target signal is enhanced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts. Wherein:
fig. 1 is a schematic flow chart illustrating a method for setting a microphone array according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating an embodiment of step S11 in FIG. 1;
FIG. 3a is a schematic diagram of a microphone array;
FIG. 3b is a schematic diagram of a steering vector matrix corresponding to the microphone array shown in FIG. 3 a;
FIG. 3c is a schematic diagram of filter coefficients of filters in the microphone array of FIG. 3 a;
FIG. 4 is a flowchart illustrating an embodiment of step S112 in FIG. 2;
FIG. 5 is a flowchart illustrating an embodiment of step S113 in FIG. 2;
FIG. 6 is a schematic structural diagram of a signal processing apparatus according to an embodiment of the present invention;
FIG. 7 is a block diagram of a signal processing system according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a storage medium according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The application provides a method for setting a microphone array, a signal processing unit, a system and a storage medium, wherein the microphone array comprises a plurality of microphone units which are arranged in an array mode. In one embodiment, the plurality of microphone units may be arranged in a circular array, for example, along the contour of a circle of successively decreasing radius. In another embodiment, the plurality of microphone units may also be arranged in a rectangular array, for example, in an outline of a rectangle whose side lengths decrease in order. Each microphone unit is correspondingly connected with one filter, and the arrangement mode of the filter corresponding to each microphone unit is the same as that of the corresponding microphone unit.
Referring to fig. 1, fig. 1 is a schematic flow chart of an embodiment of a method for arranging a microphone array according to the present application, the method including:
step S11: non-free field steering vectors of the microphone units in multiple directions under the action of the acoustic structures on the periphery of the microphone array are acquired.
An acoustic structure is a structure that affects the transmission of sound waves to a microphone array. Taking a mobile phone as an example, a microphone array in the mobile phone is all arranged in a mobile phone shell, and when sound waves outside the mobile phone are transmitted to an internal microphone, the sound waves need to pass through the mobile phone shell, the mobile phone shell can affect the sound waves transmitted to the microphone array, and the mobile phone shell is an acoustic structure. According to the method, non-free field steering vectors of each microphone unit in multiple directions under the action of an acoustic structure on the periphery of a microphone array are calculated in a finite element simulation calculation mode. Specifically, referring to fig. 2, step S11 specifically includes:
step S111: the acoustic structure is modeled three-dimensionally to obtain an acoustic model.
In this embodiment, the acoustic structure is three-dimensionally modeled, and an acoustic model is obtained. Specifically, taking a mobile phone shell as an example, an acoustic structure such as a mobile phone shell is three-dimensionally modeled according to a preset proportion. In a specific embodiment, the acoustic result may be modeled in a 1:1 ratio by using a 3D scanning or 3D modeling manner, so as to obtain an acoustic model.
Step S112: based on the acoustic model, the transfer functions of the sound waves arriving at the microphone units from multiple directions under the influence of the acoustic structure are calculated.
And calculating the transfer functions of the sound waves from multiple directions to each microphone array under the action of the acoustic structure based on the constructed acoustic model. Assuming that there are n directions, the transfer function of the sound wave arriving at each microphone unit from the n direction under the action of the acoustic structure needs to be calculated one by one. As shown in fig. 3a, the microphone array comprises 5 × 5 microphone units, and it is necessary to calculate the transfer function of the sound waves arriving at each of the 5 × 5 microphone units in n directions.
After the acoustic model is built, performing acoustic simulation on the basis of the acoustic model, and further calculating a transfer function, specifically referring to fig. 4, step S112 specifically includes:
step S1121: the relative positional relationship of each microphone unit to the acoustic structure is determined.
Specifically, after the acoustic model is established, the relative position relationship between each microphone unit and the acoustic structure may be further determined, and the coordinate position of each microphone unit relative to the acoustic structure may be obtained through the relative position relationship.
Step S1122: a plurality of first coordinate points are set at different positions on the periphery of the acoustic model, and a plurality of second coordinate points corresponding to the respective microphone units are set inside the acoustic model based on the relative positional relationship.
A plurality of first coordinate points are set at different positions on the periphery of the acoustic model, and a plurality of second coordinate points corresponding to the respective microphone units are set inside the acoustic model based on the relative positional relationship.
In a specific embodiment, a plurality of first coordinate points may be disposed at equal angular intervals on the periphery of the acoustic model. For example, one first coordinate point is set every 60 ° at the periphery of the acoustic model; or, a first coordinate point is set at every 45 ° on the periphery of the acoustic model, which is not limited specifically.
Since no model of the microphone units is built in the acoustic model, the coordinate positions of the microphone units with respect to the acoustic structure can be obtained by determining the relative positional relationship of the microphone units with respect to the acoustic structure in step S1121. Therefore, a plurality of second coordinate points corresponding to the microphone units may be set inside the acoustic model according to the coordinate positions of the respective microphone units with respect to the acoustic structure. Specifically, the second coordinate point is a coordinate position of the microphone unit relative to the acoustic structure.
Specifically, in the present application, a finite element numerical simulation calculation mode, such as finite element simulation software such as LMS, may be adopted to perform the simulation calculation. For example, the well-established acoustic model is imported into software, and the position of the origin of coordinates is set, for example, the center position of the acoustic model may be set as the position of the origin of coordinates. Then, a plurality of first coordinate points are arranged on a circle with the origin as the center of a circle and the radius r, and sound source points or receiving points are arranged on the first coordinate points. Wherein the radius r is typically more than 10 times the maximum length of the acoustic model itself. Setting the sound source intensity of the sound source point to be 1 and the phase to be 0, and then setting a receiving point or a sound source point corresponding to the position of the microphone array in the model.
Step S1123: an acoustic source point is set on one of the first coordinate point and the second coordinate point, and a reception point is set on the other of the first coordinate point and the second coordinate point.
Specifically, a sound source point is set at one of the first coordinate point and the second coordinate point, and a receiving point is set at the other coordinate point, where the receiving point is the structure of the analog microphone unit.
In a specific embodiment, the sound source point may be set at a first coordinate point and the receiving point may be set at a second coordinate point. In one embodiment, because the simulation software can generally set only one sound source point, but can set a plurality of receiving points, according to the acoustic reciprocity principle, in the same sound field, the positions of the sound source and the receiving points are interchanged, and the response of the whole transmission path is not influenced. Therefore, the receiving point may be set at the first coordinate point and the sound source point may be set at the second coordinate point. Therefore, the original multiple sound sources with one receiving point can be subjected to position exchange to become multiple receiving points and one sound source, so that the frequency response data from all directions to the same receiving point can be calculated at one time, and the calculation amount is further reduced.
Step S1124: and calculating frequency response data of sound waves generated by each sound source point reaching each receiving point, and further obtaining the transmission functions reaching each microphone unit from multiple directions.
After the sound source point and the receiving point are set, frequency response data of sound waves generated by the sound source point and passing through the acoustic structure to the receiving point are calculated, and then transfer functions reaching all the microphone units from multiple directions are obtained. Wherein the frequency response data for each of the receive points comprises a representation of the response of the receive point to the acoustic wave at the plurality of frequency sample points. In one embodiment, the response-characterizing value is used to characterize the amplitude and phase of the sound wave received at the receiving point, typically in the form of a complex number of a + bi, where the modulus of the complex number is
For representing the amplitude of the sound wave, arctan
For indicating the phase of the sound wave. When the ratio of the frequency response data of other sound source points to the frequency response data of the reference sound source point is calculated subsequently, the ratio is also calculated through the complex number a + bi.
In one embodiment, frequency response data for multiple directions to arrive at the same microphone unit may be calculated simultaneously. Referring to FIG. 3a, assuming that the microphone unit J11 has n directions, the frequency response data arriving at the microphone unit J11 from the n directions can be calculated at the same time and recorded as Hi(ω, θ), where i represents the number of the microphone unit, e.g., if microphone unit J11 is the first microphone unit, i is 0; ω represents the frequency corresponding to the frequency sampling point; θ represents the incident direction of the sound source.
After frequency response data of sound waves generated by each sound source point reaching each receiving point is calculated, the transmission function of each microphone unit is calculated based on the frequency response data. In one embodiment, one sound source point is selected from the plurality of sound source points as the reference sound source point. And calculating the transmission function according to the ratio of the frequency response data of other sound source points to the frequency response data of the reference sound source point. In a specific embodiment, the ratio of the frequency response data of other sound source points to the sum of the frequency response data of the reference sound source point and the nonzero constant can be further calculated, and then the transmission function is calculated. The specific calculation method is as follows:
wherein, RTFiThe transfer function of the ith microphone unit in the frequency omega and the incidence direction theta; hi(ω, θ) is frequency response data of the ith microphone unit at the frequency ω and the incident direction θ; h0(ω, θ) is frequency response data of the reference microphone unit at frequency ω, incident direction θ; δ is a non-zero constant.
In the microphone array shown in fig. 3a, taking J11 as the reference microphone element, the transfer functions of the remaining microphone elements except the microphone element J11 in multiple directions can be calculated by using the above formula (1).
Step S113: non-free-field steering vectors in a plurality of directions for each microphone unit are calculated based on the transfer function.
Specifically, after the transfer function of each microphone unit is calculated, the non-free-field steering vector of each microphone unit in a plurality of directions is calculated based on the transfer function. Referring to fig. 4, step S113 specifically includes:
step S1131: a reference microphone unit is selected from the microphone units.
In the microphone array shown in fig. 3a, J11 may be selected as the reference microphone unit, or J55 may be selected as the reference microphone unit, which is not limited specifically.
Step S1132: for each direction, a ratio between the response characterizing value of each microphone unit at the same frequency sampling point and the response characterizing value of the reference microphone unit is calculated.
And calculating the ratio of the response characteristic value of each microphone unit and the response characteristic value of the reference microphone unit in each direction and at the same frequency sampling point.
In one embodiment, assuming that the incident direction θ is calculated to be 30 °, the frequency ω of the frequency sampling point is calculated to be 31.25HZ, and the response characteristic value of the i-th microphone unit is Hi(31.25, 30 °), the response characteristic value of the reference microphone unit is H0(31.25, 30 °), the ratio of the response characteristic of the i-th microphone unit to the response characteristic of the reference microphone unit is:
specifically, there are a plurality of frequency sampling points in one direction, and in another embodiment, assuming that the incident direction θ is calculated to be 30 °, the frequency ω of the frequency sampling points is 62.5HZ, and the response characteristic value of the i-th microphone unit is Hi(62.5, 30 °), the response characteristic value of the reference microphone unit is H0(62.5, 30 °), the ratio of the response characteristic of the i-th microphone unit to the response characteristic of the reference microphone unit is:
in another embodiment, the ratio of the response characteristic value of the i-th microphone unit to the response characteristic value of the reference microphone unit is the ratio between the response characteristic value of each microphone unit and the result of summing the response characteristic value of the reference microphone unit and a non-zero constant. For example, the incident direction θ is 30 °, the frequency ω of the frequency sampling point is 31.25HZ, and the response characteristic value of the i-th microphone unit is Hi(31.25, 30 °), the response characteristic value of the reference microphone unit is H0(31.25, 30 °), the ratio of the response characteristic of the i-th microphone unit to the response characteristic of the reference microphone unitComprises the following steps:
step S1133: and respectively taking the ratios of a plurality of frequency sampling points in the same direction corresponding to each microphone unit as vector elements of the non-free field oriented vector in the corresponding direction.
The ratios of the microphones at a plurality of frequency sampling points in one direction are calculated through step S1132, and the ratios are respectively used as vector elements of the non-free field steering vectors in the corresponding directions.
Specifically, referring to fig. 3a and fig. 3b, the ratio of the response characteristic value of the microphone unit J12 at the frequency ω in the incident direction θ to the response characteristic value of the reference microphone unit J11 at the incident direction θ is the vector element W12 of the non-free-field-oriented vector of the microphone unit J12 at the incident direction θ. The ratio of the response characteristic value of the microphone unit J13 at the frequency ω in the incident direction θ to the response characteristic value of the reference microphone unit J11 at the incident direction θ is the vector element W13 of the non-free-field-oriented vector of the microphone unit J13 at the incident direction θ. The ratio of the response characteristic value of the microphone unit J45 at the frequency ω in the incident direction θ to the response characteristic value of the reference microphone unit J11 at the incident direction θ is the vector element W13 of the non-free-field-oriented vector of the microphone unit J45 at the incident direction θ.
As shown in fig. 3a and 3b, each microphone unit has vector elements of the non-free-field steering vector at a frequency ω in the incident direction θ, and all microphone units form a steering vector matrix at the incident direction θ of the vector elements of the non-free-field steering vector at the frequency ω. The microphone array shown in fig. 3a corresponds to a matrix of non-free-field steering vectors at frequency ω in the direction of incidence θ as shown in fig. 3 b.
Step S12: and calculating the filter coefficient corresponding to each microphone unit based on the non-free field oriented vector and the target direction.
After the vector elements of the non-free-field-oriented vector of each microphone unit in the incident direction θ and at the frequency ω are calculated by the above method, the filter coefficients corresponding to the respective microphone units may be calculated based on the non-free-field-oriented vector and the target direction. In an embodiment, the target to be acquired can be tracked, and the target direction can be generated in real time according to the tracking result.
The matrix elements in the steering vector matrix are the vector elements in the non-free-field steering vector of each microphone unit that correspond to the target direction and the specified frequency point. Specifically, assuming that the target direction is θ, the frequency of the specified frequency point is ω, and all microphone units are in the incident direction θ, the vector elements of the non-free-field steering vector at the frequency ω constitute the steering vector matrix. For example, the microphone array shown in fig. 3a corresponds to a matrix of non-free-field steering vectors at frequency ω in the direction of incidence θ as shown in fig. 3 b.
In a specific embodiment, the calculated non-free field oriented vector is further subjected to a smoothing correction. Specifically, because the reflection phenomenon of the acoustic wave is considered in the finite element simulation, the sound pressure may be 0 or approach to 0 due to the superposition of the opposite-phase acoustic waves with the same frequency, so that the maximum value or the minimum value, even 0 value, of the obtained non-free field guide vector (i.e., the transfer function) occurs, and the filter is unstable in the design process. This problem can be solved by smoothly modifying the resulting non-free-field steering vector.
In a specific embodiment, the smooth modification of the non-free-field steering vector comprises: arranging a sliding window sliding along the frequency; the vector elements within the sliding window are averaged to serve as the vector element for the center point of the sliding window. Specifically, the vector elements after smoothing are calculated using the following formula:
wherein the content of the first and second substances,
to smooth and correctThe latter steering vector matrix, AθAnd (ω i) is a guide vector matrix before smoothing correction, and N is the number of vector elements in the sliding window.
The guide vector matrix is corrected through the formula (2), so that the amplitude frequency response curve is clearer and smoother, and no performance abnormal value exists.
Specifically, for a specified frequency point in the frequency sampling points, the specified frequency point and a steering vector matrix corresponding to the target direction are substituted into the optimization function and the constraint function.
Wherein the optimization function is:
wherein L (omega) is an optimization function, omega is a specified frequency point, W (omega) is a filter coefficient matrix under the specified frequency point, matrix elements in the filter coefficient matrix are filter coefficients of each microphone unit under the specified frequency point, WH(ω) is the conjugate transpose of W (ω), Bθ0(ω) is a steering vector matrix, θ0Γ is a preset diffusion field noise coherence matrix for the target direction.
The constraint function is:
where 1 is the unit vector, σthAnd r is a white noise gain lower limit constraint.
And taking the optimization function (3) as a target, taking the constraint functions (4) and (5) as constraint conditions, and calculating the filter coefficients corresponding to the microphone units in an iterative mode, so that the power ratio of a target signal corresponding to the target direction and the extended field noise output by the microphone array is maximized after the filter coefficients act.
Specifically, referring to fig. 3a, fig. 3b and fig. 3c, specifically, the filter coefficient corresponding to each microphone array unit in fig. 3a may be calculated by using the steering vector matrix formed in fig. 3 b. Wherein the filter coefficient H11 corresponds to the filter coefficient of the microphone unit J11; the filter coefficient H12 corresponds to the filter coefficient of the microphone unit J12; the filter coefficient H14 corresponds to the filter coefficient of the microphone unit J14.
In one embodiment, after non-free field steering vectors of each microphone unit in multiple directions are obtained through calculation in a finite element simulation mode, the non-free field steering vectors are stored in a memory, when a microphone array enters an application after being manufactured, after a target direction is determined, a non-free field steering vector matrix formed by steering vector elements of each microphone unit at corresponding frequencies in the target direction is retrieved from the memory, and filter coefficients corresponding to each microphone unit in the target direction are calculated by using the non-free field steering vector matrix obtained in the memory.
Step S12: and filtering the filter corresponding to each microphone unit based on the filter coefficient.
By the method, after the filter coefficient corresponding to each microphone unit is obtained through calculation, the corresponding filter is filtered by the filter coefficient corresponding to each microphone unit, and then the microphone array is set.
According to the method for setting the microphone array, on the premise that the incident microphone array is a non-free field influenced by a peripheral acoustic structure, non-free field guide vectors of all microphone units in multiple directions are calculated in advance, filter coefficients corresponding to all the microphone units are further calculated by combining with a target direction, the filtering effect is improved, and then a target signal is enhanced.
The method for setting the microphone array adopts a finite element simulation method to accurately calculate the array steering vector of a given acoustic structure under a non-free field. And calculating to obtain the filter coefficient according to the calculated guide vector. By utilizing the design of the non-free field, the design result is more in line with the actual situation, the influence of the recording deviation is not easy to be caused, and the robustness is good.
Referring to fig. 6, a schematic structural diagram of a signal processing apparatus according to an embodiment of the present invention is shown, the signal processing apparatus includes a memory 202 and a processor 201 connected to each other.
The memory 202 is used to store program instructions for implementing any of the methods described above.
The processor 201 is used to execute program instructions stored by the memory 202.
The processor 201 may also be referred to as a Central Processing Unit (CPU). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 202 may be a memory bank, a TF card, etc., and may store all information in the electronic device of the device, including the input raw data, the computer program, the intermediate operation results, and the final operation results. It stores and retrieves information based on the location specified by the controller. With the memory, the signal processing device has a memory function, and can work normally. The memory of the signal processing apparatus can be classified into a main memory (internal memory) and an auxiliary memory (external memory) according to the use, and also into an external memory and an internal memory. The external memory is usually a magnetic medium, an optical disk, or the like, and can store information for a long period of time. The memory refers to a storage component on the main board, which is used for storing data and programs currently being executed, but is only used for temporarily storing the programs and the data, and the data is lost when the power is turned off or the power is cut off.
In the several embodiments provided in the present application, it should be understood that the disclosed method and apparatus may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a system server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method of the embodiments of the present application.
Fig. 7 is a schematic structural diagram of a signal processing system according to the present invention, which includes a microphone array 200 and the signal processing apparatus of fig. 6 coupled to each other.
In one embodiment, the microphone array 200 includes a plurality of microphone units arranged in an array. In one embodiment, the plurality of microphone units may be arranged in a circular array, for example, along the contour of a circle of successively decreasing radius. In another embodiment, the plurality of microphone units may also be arranged in a rectangular array, for example, with a rectangular outline that decreases in length along the sides, as shown in fig. 3 a. Each microphone unit is correspondingly connected with one filter, and the arrangement mode of the filter corresponding to each microphone unit is the same as that of the corresponding microphone unit.
Please refer to fig. 8, which is a schematic structural diagram of a storage medium according to the present invention. The storage medium of the present application stores a program file 203 capable of implementing all the methods described above, wherein the program file 203 may be stored in the storage medium in the form of a software product, and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present application. The aforementioned storage device includes: various media capable of storing program codes, such as a usb disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, or terminal devices, such as a computer, a server, a mobile phone, and a tablet.
The above embodiments are merely examples and are not intended to limit the scope of the present disclosure, and all modifications, equivalents, and flow charts using the contents of the specification and drawings of the present disclosure or those directly or indirectly applied to other related technical fields are intended to be included in the scope of the present disclosure.
Claims (15)
1. A microphone array placement method, wherein the microphone array includes a plurality of microphone elements arranged in an array and a filter connected to each of the microphones, the method comprising:
acquiring non-free field steering vectors of the microphone units in multiple directions under the action of an acoustic structure on the periphery of the microphone array;
calculating a filter coefficient corresponding to each microphone unit based on the non-free-field steering vector and a target direction;
and filtering the filter corresponding to each microphone unit based on the filter coefficient.
2. The method of claim 1, wherein the non-free-field steering vectors for the microphone units in the plurality of directions are calculated by:
three-dimensionally modeling the acoustic structure to obtain an acoustic model;
calculating, based on the acoustic model, a transfer function of sound waves arriving at each of the microphone units from the plurality of directions under the influence of the acoustic structure;
calculating non-free-field steering vectors for each of the microphone units in the plurality of directions based on the transfer function.
3. The method of claim 2, wherein the step of calculating a transfer function of sound waves arriving at each of the microphone units from the plurality of directions under the influence of the acoustic structure based on the acoustic model comprises:
determining a relative positional relationship of each of the microphone units to the acoustic structure;
setting a plurality of first coordinate points at different positions on the periphery of the acoustic model, and setting a plurality of second coordinate points corresponding to the microphone units inside the acoustic model based on the relative positional relationship;
setting an acoustic source point on one of the first coordinate point and the second coordinate point, and setting a reception point on the other of the first coordinate point and the second coordinate point;
and calculating frequency response data of sound waves generated by each sound source point reaching each receiving point, thereby obtaining a transmission function reaching each microphone unit from the multiple directions, wherein the frequency response data of each receiving point comprises response characterization values of the receiving points to the sound waves at multiple frequency sampling points.
4. The method of claim 3, wherein the sound source point is located at the second coordinate point and the receiving point is located at the first coordinate point.
5. The method of claim 2, wherein said step of calculating non-free-field steering vectors for each of said microphone units in said plurality of directions based on said transfer function comprises:
selecting a reference microphone unit from the microphone units;
for each of said directions, calculating a ratio between said response characterizing value of each of said microphone units and said response characterizing value of said reference microphone unit at the same frequency sampling point;
and respectively taking the ratios of a plurality of frequency sampling points in the same direction corresponding to each microphone unit as vector elements of the non-free field oriented vector in the corresponding direction.
6. The method of claim 5, wherein the ratio is a ratio between the response characterizing value of each of the microphone units and a result of summing the response characterizing value of the reference microphone unit and a non-zero constant.
7. The method of claim 5, wherein the step of calculating non-free-field steering vectors for each of the microphone units in the plurality of directions based on the transfer function further comprises:
and carrying out smooth correction on the non-free field guide vector.
8. The method of claim 7, wherein the step of smoothly modifying the non-free-field steering vector comprises:
arranging a sliding window sliding along the frequency;
averaging the vector elements within the sliding window to serve as the vector element at the center point of the sliding window.
9. The method of claim 5, wherein the step of calculating filter coefficients corresponding to each of the microphone units based on the non-free-field steering vector and a target direction comprises:
for a specified frequency point in the frequency sampling points, substituting a steering vector matrix corresponding to the specified frequency point and the target direction into an optimization function and a constraint function, wherein matrix elements in the steering vector matrix are the vector elements corresponding to the target direction and the specified frequency point in the non-free-field steering vector of each microphone unit;
and taking the optimization function as a target to be maximized, and taking the constraint function as a constraint condition, and calculating filter coefficients corresponding to the microphone units in an iterative mode, so that the power ratio of a target signal, which is output by the microphone array and corresponds to the target direction, to the extended field noise is maximized after the filter coefficients act.
10. The method of claim 9, wherein the optimization function comprises:
wherein L (ω) is the optimization function, ω is the designated frequency point, W (ω) is a filter coefficient matrix at the designated frequency point, wherein matrix elements in the filter coefficient matrix are filter coefficients of each of the microphone units at the designated frequency point, W (ω)H(ω) is the conjugate transpose of W (ω),
is the directorQuantity matrix, θ0Gamma is a preset diffusion field noise coherent matrix for the target direction;
the constraint function includes:
where 1 is the unit vector, σthrIs a white noise gain lower limit constraint.
11. The method of claim 1, wherein the step of obtaining non-free-field steering vectors for the microphone elements in a plurality of directions by the acoustic structures at the periphery of the microphone array comprises:
and acquiring the non-free field oriented vector stored in advance from a memory.
12. The method of claim 1, wherein the step of calculating filter coefficients corresponding to each of the microphone units based on the non-free-field steering vector and the target direction is preceded by the step of:
and tracking the target to be acquired, and generating the target direction in real time according to a tracking result.
13. A signal processing apparatus comprising a memory and a processor coupled to each other, the memory having stored therein program instructions, the processor being configured to execute the program instructions to implement the method of any one of claims 1 to 12.
14. A signal processing system comprising an array of microphones coupled to each other and a signal processing device according to claim 13.
15. A storage medium, characterized by program instructions executable by a processor for implementing the method of any one of claims 1 to 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011615054.0A CN112714383B (en) | 2020-12-30 | 2020-12-30 | Microphone array setting method, signal processing device, system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011615054.0A CN112714383B (en) | 2020-12-30 | 2020-12-30 | Microphone array setting method, signal processing device, system and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112714383A true CN112714383A (en) | 2021-04-27 |
| CN112714383B CN112714383B (en) | 2022-03-11 |
Family
ID=75547373
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011615054.0A Active CN112714383B (en) | 2020-12-30 | 2020-12-30 | Microphone array setting method, signal processing device, system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112714383B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114305485A (en) * | 2021-12-31 | 2022-04-12 | 科大讯飞股份有限公司 | Heartbeat monitoring method, heartbeat monitoring device and computer readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004274763A (en) * | 2003-03-06 | 2004-09-30 | Samsung Electronics Co Ltd | Microphone array structure, beam forming apparatus and method, and method and apparatus for estimating acoustic source direction |
| JP2011199474A (en) * | 2010-03-18 | 2011-10-06 | Hitachi Ltd | Sound source separation device, sound source separating method and program for the same, video camera apparatus using the same and cellular phone unit with camera |
| US20140294211A1 (en) * | 2011-09-27 | 2014-10-02 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Apparatus and method for listening room equalization using a scalable filtering structure in the wave domain |
| US20160044411A1 (en) * | 2014-08-05 | 2016-02-11 | Canon Kabushiki Kaisha | Signal processing apparatus and signal processing method |
| US20160165339A1 (en) * | 2014-12-05 | 2016-06-09 | Stages Pcs, Llc | Microphone array and audio source tracking system |
| CN109633527A (en) * | 2018-12-14 | 2019-04-16 | 南京理工大学 | Inlaid flat microphone array sound source direction-finding method based on low-rank and geometrical constraint |
| US20190387312A1 (en) * | 2018-06-13 | 2019-12-19 | Beijing Xiaoniao Tingting Technology Co., Ltd | Audio interaction device, data processing method and computer storage medium |
-
2020
- 2020-12-30 CN CN202011615054.0A patent/CN112714383B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004274763A (en) * | 2003-03-06 | 2004-09-30 | Samsung Electronics Co Ltd | Microphone array structure, beam forming apparatus and method, and method and apparatus for estimating acoustic source direction |
| JP2011199474A (en) * | 2010-03-18 | 2011-10-06 | Hitachi Ltd | Sound source separation device, sound source separating method and program for the same, video camera apparatus using the same and cellular phone unit with camera |
| US20140294211A1 (en) * | 2011-09-27 | 2014-10-02 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Apparatus and method for listening room equalization using a scalable filtering structure in the wave domain |
| US20160044411A1 (en) * | 2014-08-05 | 2016-02-11 | Canon Kabushiki Kaisha | Signal processing apparatus and signal processing method |
| US20160165339A1 (en) * | 2014-12-05 | 2016-06-09 | Stages Pcs, Llc | Microphone array and audio source tracking system |
| US20190387312A1 (en) * | 2018-06-13 | 2019-12-19 | Beijing Xiaoniao Tingting Technology Co., Ltd | Audio interaction device, data processing method and computer storage medium |
| CN109633527A (en) * | 2018-12-14 | 2019-04-16 | 南京理工大学 | Inlaid flat microphone array sound source direction-finding method based on low-rank and geometrical constraint |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114305485A (en) * | 2021-12-31 | 2022-04-12 | 科大讯飞股份有限公司 | Heartbeat monitoring method, heartbeat monitoring device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112714383B (en) | 2022-03-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109102822B (en) | Filtering method and device based on fixed beam forming | |
| KR102305066B1 (en) | Sound processing method and device | |
| KR102009274B1 (en) | Fir filter coefficient calculation for beam forming filters | |
| US20160173978A1 (en) | Audio Signal Processing Method and Apparatus and Differential Beamforming Method and Apparatus | |
| EP2431973B1 (en) | Apparatus and method for enhancing audio quality using non-uniform configuration of microphones | |
| US20200118580A1 (en) | Sound processing method and apparatus | |
| US11651772B2 (en) | Narrowband direction of arrival for full band beamformer | |
| CA2956016A1 (en) | Planar sensor array | |
| US9479867B2 (en) | Method and circuitry for direction of arrival estimation using microphone array with a sharp null | |
| US10149055B2 (en) | Loud-speaking, loud-speaker and interactive device | |
| CN108401218B (en) | Fault diagnosis method for active noise reduction earphone | |
| CN111060874B (en) | Sound source positioning method and device, storage medium and terminal equipment | |
| US9866957B2 (en) | Sound collection apparatus and method | |
| CN112714383B (en) | Microphone array setting method, signal processing device, system and storage medium | |
| CN108717495A (en) | The method, apparatus and electronic equipment of multi-beam beam forming | |
| US10356508B2 (en) | Sound collecting apparatus | |
| CN109669158A (en) | A kind of sound localization method, system, computer equipment and storage medium | |
| Taghizadeh et al. | Enhanced diffuse field model for ad hoc microphone array calibration | |
| CN116508328A (en) | Speech existence probability calculation method, speech existence probability calculation system, speech enhancement method, speech enhancement system and earphone | |
| Cho et al. | Adaptive near-field beamforming techniques for sound source imaging | |
| Alon et al. | Spherical microphone array with optimal aliasing cancellation | |
| CN112750455A (en) | Audio processing method and device | |
| CN106448693A (en) | Speech signal processing method and apparatus | |
| CN109309888A (en) | Voice information processing method, playback equipment and computer readable storage medium | |
| CN110661510B (en) | Beam former forming method, beam forming device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |