CN110068797B - Method for calibrating microphone array, sound source positioning method and related equipment - Google Patents
Method for calibrating microphone array, sound source positioning method and related equipment Download PDFInfo
- Publication number
- CN110068797B CN110068797B CN201910328573.XA CN201910328573A CN110068797B CN 110068797 B CN110068797 B CN 110068797B CN 201910328573 A CN201910328573 A CN 201910328573A CN 110068797 B CN110068797 B CN 110068797B
- Authority
- CN
- China
- Prior art keywords
- microphones
- sound data
- microphone
- time delay
- sound source
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
- G01S5/18—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using ultrasonic, sonic, or infrasonic waves
- G01S5/24—Position of single direction-finder fixed by determining direction of a plurality of spaced sources of known location
Abstract
The application provides a method for calibrating a microphone array, a sound source positioning method and related equipment, which are used for improving the accuracy of the calibration value of the obtained microphone. The method comprises the following steps: collecting sound data from a set sound source point by N microphones in a microphone array; the distances from the N microphones to the set sound source point are the same; obtaining the correlation coefficients of every two microphones in the N microphones by adopting a preset correlation coefficient algorithm according to the sound data of the N microphones; determining M correlated microphones according to the correlation coefficient which is smaller than a preset correlation coefficient threshold value in the correlation coefficients; acquiring time delay differences of sound data collected by every two microphones in the rest microphones except the M microphones based on the sound data of every two microphones in the N microphones; and determining a time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones based on the time delay difference.
Description
Technical Field
The present application relates to the field of sound source localization technologies, and in particular, to a method for calibrating a microphone array, a sound source localization method, and a related device.
Background
In many situations where voice interaction is relevant, such as a voice assistant and a consumer electronics product such as a mobile phone that completes voice communication, it is common to use a microphone array to determine a location of a user according to the voice of the user, and perform voice enhancement on the voice at the location. Calibration of the microphone array is often required before the device is shipped.
In the prior art, a method for calibrating a microphone array generally includes: and testing the calibration parameters of the single microphones, and taking the sum of the parameters of the single microphones as the calibration result of the microphone array. The method for obtaining the calibration result by the sound source positioning calibration method ignores the influence of the performance parameter change of each microphone and ignores the correlation among the microphones, so that the accuracy of the calibration result obtained by the calibration method is poor.
Disclosure of Invention
The embodiment of the application provides a microphone array calibration method, a sound source positioning method and related equipment, which are used for improving the accuracy of the calibration value of an obtained microphone.
In a first aspect, a method for calibrating a microphone array is provided, including:
collecting sound data from a set sound source point by N microphones in a microphone array; the distances from the N microphones to the set sound source point are the same, and N is an integer greater than or equal to 3;
obtaining the correlation coefficients of every two microphones in the N microphones by adopting a preset correlation coefficient algorithm according to the sound data of the N microphones;
determining M correlated microphones according to the correlation coefficients which are smaller than a preset correlation coefficient threshold value; wherein M is an integer less than N;
obtaining time delay differences of sound data collected by every two microphones in the rest microphones except the M microphones in the N microphones based on the sound data of every two microphones in the rest microphones;
and determining a time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones based on the time delay difference.
In the scheme, the abnormal microphone is determined according to the correlation coefficient of every two microphones in the microphone array, and the time delay difference calibration value is calculated according to the microphones except the abnormal microphone in the microphone array, so that the abnormal microphone is prevented from influencing the calibration result, and the accuracy of the calibration result is improved. And the correlation coefficient is skillfully utilized to eliminate abnormal microphones in the microphone array, so that the quality of the product can be detected.
In one possible design, obtaining correlation coefficients of every two microphones in the N microphones by using a preset correlation coefficient algorithm according to sound data of the N microphones includes:
intercepting sound data of each microphone in every two microphones in the N microphones with preset lengths to obtain the intercepted sound data of every two microphones in the N microphones;
and normalizing the intercepted sound data of every two microphones in the N microphones to obtain the correlation coefficients of every two microphones in the N microphones.
In the above solution, intercepting the sound data of every two microphones, on one hand, it may be ensured that the lengths of the sound data of the correlation calculation are the same, so that the calculated correlation coefficient has a reference value, and in a possible design, obtaining the delay difference of the sound data collected by every two microphones in the remaining microphones based on the sound data of every two microphones in the remaining microphones except the M microphones in the N microphones includes:
constructing a cost function between sound data of every two microphones in the rest microphones;
obtaining the maximum value of the cost function of every two microphones in the rest microphones based on the preset time delay difference value range;
and determining the time delay difference corresponding to the maximum value as the time delay difference of the sound data collected by every two microphones in the rest microphones.
In the above scheme, the cost function of every two microphones is obtained by traversing the cost function to obtain the time delay difference of the sound data of every two microphones, so that more frames of sound data can participate in calculation, the influence on the result due to the abnormal sound data of a certain frame can be avoided, and the obtained time delay difference of the sound data of every two microphones can be more accurate.
In one possible design, determining, based on the delay difference, a delay calibration value corresponding to each of the remaining microphones that needs to be calibrated includes:
determining at least two delay differences, wherein the difference between the delay difference and the average value of the delay differences is larger than a preset difference;
and determining a calibration value corresponding to at least one microphone associated with the at least two time delay differences according to the average value and the at least two time delay differences, thereby obtaining the time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones.
In the above scheme, whether a delay difference is large or not is determined according to a difference between an average value of a plurality of delay differences and each delay difference, and a calibration value of a corresponding microphone is determined according to the average value and the delay difference, that is, the delay calibration value can be relatively accurately obtained according to the principle of delay consistency of a plurality of microphones.
In a second aspect, a sound source localization method is provided, including:
according to the time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones obtained by the method of the first aspect and any one of possible designs, compensating the target sound data collected by the rest microphones from the sound source point to be measured;
and obtaining the position of the sound source point to be detected based on the target sound data of the microphones except the compensated microphone in the other microphones and the target sound data of the compensated microphone.
In the above scheme, on the basis of positioning the sound source to be detected, the sound data of the microphone is compensated according to the accurate time delay calibration value corresponding to the microphone, so as to obtain the position of the sound source point to be detected.
In a third aspect, an apparatus for calibrating a microphone array is provided, comprising:
the detection module is used for collecting sound data from a set sound source point through N microphones in the microphone array; the distances from the N microphones to the set sound source point are the same, and N is an integer greater than or equal to 3;
the processing module is used for obtaining the correlation coefficients of every two microphones in the N microphones by adopting a preset correlation coefficient algorithm according to the sound data of the N microphones;
the processing module is further configured to determine, according to a part of the correlation coefficients smaller than a preset correlation coefficient threshold, corresponding M associated microphones; wherein M is an integer less than N;
the processing module is further configured to obtain a time delay difference of sound data collected by each two microphones of the rest microphones based on sound data of each two microphones of the rest microphones of the N microphones except the M microphones;
the processing module is further configured to determine, based on the time delay difference, a time delay calibration value corresponding to each microphone that needs to be calibrated among the other microphones.
In one possible design, the processing module is specifically configured to:
intercepting sound data of each microphone in every two microphones in the N microphones with preset lengths to obtain the intercepted sound data of every two microphones in the N microphones;
and normalizing the intercepted sound data of every two microphones in the N microphones to obtain the correlation coefficients of every two microphones in the N microphones.
In a fourth aspect, there is provided a sound source localization apparatus comprising:
a compensation module, configured to compensate, by using the time delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones obtained by the method in the first aspect, target sound data acquired by the remaining microphones from a sound source point to be measured;
and the processing module is used for obtaining the position of the sound source point to be detected based on the target sound data of the microphones except the compensated microphone in the other microphones and the target sound data of the compensated microphone.
In a fifth aspect, a smart device is provided, comprising:
at least one processor, and
a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the method as described in the first or second aspect by executing the instructions stored by the memory.
A sixth aspect provides a computer readable storage medium having stored thereon computer instructions which, when run on a computer, cause the computer to perform the method as described in the first or second aspect.
Drawings
Fig. 1 is a schematic layout diagram of a microphone array according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a method for calibrating a microphone array according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of audio data provided by an embodiment of the present application;
fig. 4 is a flowchart of a sound source positioning method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of an apparatus for calibrating a microphone array according to an embodiment of the present disclosure;
fig. 6 is a schematic structural diagram of a sound source positioning device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of an intelligent device according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions provided by the embodiments of the present application, the following detailed description is made with reference to the drawings and specific embodiments.
In order to improve the accuracy of the calibration values of the obtained quasi microphone array, the present application is implemented to a method for calibrating a microphone array, and the microphone array related to the method in the embodiments of the present application is described below.
Referring to fig. 1, fig. 1 shows a microphone array 110 including N microphones 120 and a set sound source location point 130. The performance parameters of the plurality of microphones 120 are consistent, such as sampling rate, sampling precision, number of channels, and bit rate. The N microphones 120 are all equidistant from the set sound source location point 130. The set sound source location point 130 is used to place the set sound source point. Wherein the microphone array does not generally include a set sound source point. In the actual calibration process, the user may place a set sound source point on the set sound source location point 130, and the distances from the plurality of microphone arrays 120 to the set sound source point are all equal.
Although fig. 1 illustrates the microphone array 110 as a circular shape, the shape of the microphone array 110 may be a regular multi-shape, a regular cube shape, or a spherical shape, and the shape of the microphone array 110 is not particularly limited in the embodiment of the present invention. Fig. 1 illustrates an example in which 6 microphones 120 are included in the microphone array 110, but the number of microphones 120 in the microphone array 110 is not limited in practice.
In addition, the method for calibrating the microphone array in the embodiment of the present application is applicable to a scenario of far field calibration, and the far field calibration is briefly described below.
With reference to fig. 1, the distance L between the sound source point 130 and the microphone array 110 is set to be much greater than the wavelength λ of the signal emitted from the sound source point, and the specific relationship between L and λ is as follows:
where D represents the distance between every two sound source points.
Two usage scenarios of the method for calibrating a microphone array in the embodiment of the present application are described below with reference to the microphone array of fig. 1.
In a first scenario, a manufacturer can calibrate a microphone array by using the method before the microphone array leaves a factory;
in the second scenario, after the user sets the microphone array, the user may perform calibration with the microphone array according to the method.
Referring to fig. 2, a method for calibrating a microphone array according to an embodiment of the present invention is described below with reference to the microphone array described in fig. 1, and the method specifically includes:
Embodiments of the present disclosure relate to a method of calibrating a microphone array 110 that may be performed by a device that calibrates a microphone array. The device for calibrating a microphone array may be implemented by a microphone array and a processing unit. The Processing Unit may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits configured to implement the embodiments of the present Application, for example: one or more microprocessors (digital signal processors, DSPs), or one or more Field Programmable Gate Arrays (FPGAs).
The following describes a procedure for performing the above steps in an apparatus for calibrating a microphone array.
The user prepares to calibrate the microphone array 110, the user turns on the device for calibrating the microphone array, and the device for calibrating the microphone array performs step 201, i.e. collects sound data from the set sound source point through the N microphones 120 in the microphone array 110.
Specifically, each microphone 120 in the device for calibrating the microphone array is turned on, the user controls the set sound source point to emit sound, and the N microphones 120 correspondingly acquire sound data emitted by the set sound source point. The sound data collected by the N microphones 120 is typically time domain data.
The sound data may be in units of frames, and each of the N microphones 120 collects at least one frame of sound data from a set sound source point. The apparatus for calibrating a microphone array acquires sound data corresponding to each microphone 120 of the N microphones 120 from the N microphones 120.
While acquiring the sound data, the apparatus for calibrating the microphone array establishes a correspondence between each microphone 120 of the N microphones 120 and the corresponding sound data.
Specifically, the apparatus for calibrating a microphone array acquires the sound data from the N microphones 120 and simultaneously acquires the identifications of the N microphones 120, so as to establish the relationship between each microphone 120 of the N microphones 120 and the sound data corresponding to the microphone 120.
Alternatively, the device for calibrating a microphone array pre-stores the identification of each microphone 120 in advance, and after obtaining the sound data of each microphone 120, the device for calibrating a microphone array establishes the relationship between the microphone 120 and the corresponding sound data.
After the apparatus for calibrating a microphone array performs step 201, the apparatus for calibrating a microphone array performs step 202, and obtains correlation coefficients of every two microphones in the N microphones by using a preset correlation coefficient algorithm according to the sound data of the N microphones 120.
In a possible embodiment, the device for calibrating a microphone array may acquire multiple frames of sound data corresponding to each of the N microphones, and in order to ensure that the lengths of the sound data capable of participating in the sound data corresponding to each of the N microphones are the same, the device for calibrating a microphone array may intercept sound data of a corresponding preset length from the sound data of each of the N microphones, or may perform a shift process on the sound data of one of every two microphones.
Where two microphones are understood to be any one of N microphones combined with each of the other microphones. The preset length is measured in time length, or in frame number, for example, the preset length is 3s, or the preset length is 20 frames.
Further, when intercepting the sound data of every two microphones, the overlapping area of the sound data of the two microphones can be correspondingly covered so as to increase the accuracy of the calculated correlation coefficient.
Specifically, the overlapping area of the sound data of each two microphones may be understood as the most similar sound data of each two microphones, and since the overlapping areas of the sound data of each two microphones may not be completely the same, the intercepted sound data corresponding to different two microphones may not be completely the same.
For example, referring to fig. 3, the sound data of the first microphone of the N microphones is shown as a1 in fig. 3, the sound data of the second microphone of the N microphones is shown as a2 in fig. 3, and the sound data of the preset length L in a1 is intercepted, and in order to make the sound data of a1 and the sound data of a2 more similar, the sound data of the preset length L delayed by t2 in a2 may be intercepted.
After intercepting the sound data of a preset length of each microphone, the apparatus for calibrating the microphone array converts the sound data of the preset length of each microphone into frequency-domain data through a fourier transform (FFT).
After intercepting the sound data with the preset length of each microphone, normalization processing may be performed on the sound data corresponding to each microphone of the intercepted N microphones according to the sound data corresponding to each microphone of the intercepted N microphones, so as to obtain correlation coefficients of each two microphones of the N microphones. The correlation coefficient can be understood as a degree of similarity of sound data corresponding to the two microphones.
There are many calculation methods for obtaining the correlation coefficient of each two microphones in the N microphones by performing normalization processing on the sound data corresponding to each microphone in the N microphones after being intercepted, and the following description is given by way of example.
The first method is as follows:
normalization is performed according to the following formula:
wherein L represents the length of the intercepted voice data, Xi(k) Represents the k frame of sound data, X, collected by the ith microphone of the N microphonesh(k) Represents the k frame of sound data, X, collected by the j microphone of the N microphonesh(k)*And the conjugate number of the sound data collected by the jth microphone in the N microphones is represented.
The second method comprises the following steps:
normalization is performed according to the following formula:
wherein, XiA vector X representing the multi-frame sound data collected by the ith microphone in the N microphones and formed according to the time sequencehAnd the vector represents a vector formed by multi-frame sound data collected by the h microphone in the N microphones according to the time sequence.
It should be noted that, if the previous sound data is sound time domain data, the sound time domain data should be converted into sound frequency domain data for calculation when calculating the correlation coefficient of every two microphones.
After step 202 is performed, the apparatus for calibrating a microphone array performs step 203, i.e. determines M associated microphones according to the part of the correlation coefficients smaller than the preset correlation coefficient threshold, where M is an integer smaller than N.
Specifically, since there are a total of N microphones, and each microphone corresponds to one correlation coefficient with each of the other microphones, and therefore, there are (N-1) correlation coefficients associated with each microphone, the apparatus for calibrating the microphone array may compare the plurality of correlation coefficients with the preset correlation coefficient threshold after obtaining the correlation coefficients of each two microphones of the N microphones.
The apparatus for calibrating the microphone array determines that a microphone is normal if the correlation coefficients associated with the microphone are both greater than or equal to the correlation coefficient threshold or if only one of the correlation coefficients associated with the microphone is less than the correlation coefficient threshold. The apparatus for calibrating the microphone array determines that one microphone is abnormal if at least two of the correlation coefficients of the microphone and the other microphones are both less than a correlation coefficient threshold.
Wherein the preset correlation coefficient threshold may be set by a user or by default by a device calibrating the microphone array.
The device for calibrating the microphone array may determine one or more abnormal microphones from the N microphones, or may not determine an abnormal microphone from the N microphones, that is, M may be 0, or may be an integer greater than or equal to 1.
For example, the threshold value of the correlation coefficient stored in the device for calibrating the microphone array is 0.7, the device for calibrating the microphone array includes 3 microphones (a, B, C), and the correlation coefficients corresponding to the 3 microphones are specifically shown in table 1 below.
| Microphone (CN) | Correlation coefficient |
| A-B | 0.65 |
| B-C | 0.6 |
| A-C | 0.8 |
For example, the apparatus for calibrating the microphone array determines that the correlation coefficients corresponding to the microphone B and the other microphones are all less than the correlation coefficient threshold, and the apparatus for calibrating the microphone array determines that the microphone B is abnormal.
Since the N microphones are all the same distance from the set sound source point, the correlation coefficient of sound data received by each two of the N microphones should be theoretically large. If the correlation coefficient corresponding to a certain microphone is abnormal, the microphone can be determined to be abnormal. In the embodiment of the present application, microphones with low signal amplitude values or completely non-operational values can be excluded by comparing the correlation coefficient with the correlation coefficient threshold. And eliminating abnormal microphones to avoid the influence of the abnormal microphones on the accuracy of the calibration value obtained subsequently, wherein even if a plurality of abnormal microphones appear in the microphone array, the microphone array can work relatively accurately as usual. And, the abnormal microphone is eliminated, so that the processing amount in the subsequent process can be relatively reduced. And the method can be used for screening unqualified microphone array products.
After performing step 203, the apparatus for calibrating a microphone array performs step 204, i.e. obtains the time delay difference of the sound data collected by each two microphones of the rest microphones except the M microphones based on the sound data of each two microphones of the rest microphones of the N microphones.
Specifically, after determining that the M microphones are abnormal, the apparatus for calibrating the microphone array does not process the sound data of the abnormal M microphones any more, but calculates the time delay difference between every two microphones of the remaining microphones based on the sound data of every two microphones of the remaining microphones of the N microphones except the abnormal M microphones. The delay difference can be understood as the delay difference existing when two microphones receive the same frame of sound data which sets the sound source point.
If the value of M is 0, the rest microphones are also N microphones, and if the value of M is not 0, the rest microphones are microphones other than the abnormal M microphones.
There are many ways to obtain the time delay difference of the sound data collected by every two microphones in the rest microphones, and the following examples are provided.
One way to obtain the delay difference is:
constructing a cost function between sound data of every two microphones in the rest microphones;
obtaining the maximum value of the cost function of every two microphones in the other microphones based on the preset time delay difference value range;
and determining the time delay difference corresponding to the maximum value as the time delay difference of the sound data collected by every two microphones in the rest microphones.
Specifically, the device for calibrating the microphone array may first construct a cost function between sound data of two microphones, traverse the cost function according to a preset delay difference range, and obtain a maximum value of the cost function, where a delay difference corresponding to the maximum value is a delay difference between every two microphones.
A specific expression of the cost function is as follows:
wherein k represents the kth frame in the sound data, w represents the weighting coefficient, j represents the imaginary part, τ represents the time delay difference, s represents the length of the intercepted sound data, s and the value of L in the preceding text can be the same or different, Xi(k) Represents the k frame of sound data, X, collected by the ith microphone of the N microphonesh(k) Represents the k frame of sound data, X, collected by the j microphone of the N microphonesh(k)*And the conjugate number of the sound data collected by the jth microphone in the N microphones is represented.
Specifically, the predetermined delay difference range is [ -x, x [ -x]And if the preset traversal step is z, sequentially taking tau as-x, (-x + z), … (-x +2z) … x by the equipment for calibrating the microphone array, and sequentially calculating the values corresponding to different tau
A value of (a), obtained
Is the time delay difference between the ith microphone and the jth microphone.
Since the distances between the other microphones and the set sound source point are equal, the time delay differences of every two microphones in the other microphones should be equal or similar to each other, excluding the influence caused by differences of the position deviation of the microphone array 110, the installation deviation of the microphones, the amplitude deviation of the microphones, and the like. However, some microphones may have large deviations due to physical performance degradation, etc. of one or some microphones, and therefore, the microphones need to be calibrated.
Therefore, after the device for calibrating the microphone array performs step 204, step 205 is performed, that is, based on the delay difference, a delay calibration value corresponding to each microphone needing calibration in the rest of microphones is determined.
Specifically, the device for calibrating the microphone array obtains the delay difference of every two microphones in the rest microphones, so that a plurality of delay differences corresponding to the rest microphones can be obtained. And the equipment for calibrating the microphone array determines a time delay calibration value corresponding to the microphone to be calibrated according to the plurality of time delay differences. There are many ways to determine the delay calibration value based on the delay difference, and the following description will exemplify the ways to determine the delay calibration value.
One way to determine the delay calibration value is:
and obtaining the time delay differences corresponding to the microphones needing to be calibrated in the other microphones according to the time delay differences and the average value of the time delay differences.
Specifically, the device for calibrating the microphone array determines at least two time delay differences, wherein the difference between the time delay difference and the average value of the time delay differences is greater than a preset difference;
and determining a calibration value corresponding to at least one microphone associated with the at least two time delay differences according to the average value and the at least two time delay differences, thereby obtaining the time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones.
The device for calibrating the microphone array determines an average value of a plurality of delay differences, determines a difference value between each delay difference and the average value, and if the difference value corresponding to some delay differences is larger than a preset difference value, the delay differences represent that the delay differences are likely to be larger in deviation, so that the device for calibrating the microphone array can determine that the microphone is likely to be larger in deviation according to a microphone associated with the delay differences. The device for calibrating the microphone array may obtain the time delay calibration value of the microphone by subtracting the average value of the time delay differences corresponding to the remaining microphones from the average value of the time delay differences of the remaining microphones.
One way to determine the delay calibration value is:
and obtaining a time delay calibration value corresponding to the microphone according to the average value of the time delay differences associated with the microphone minus the average value of the time delay differences corresponding to the microphones except the microphone in the rest microphones.
The method for determining the microphone with larger deviation can refer to the manner discussed in the foregoing, and is not described in detail here.
After obtaining the time delay calibration value corresponding to each microphone that needs to be calibrated in the rest of microphones, when the device that calibrates the microphone array locates the sound source, the time delay calibration may be performed on the sound data acquired by the microphone based on the time delay calibration value corresponding to each microphone.
In scenario one discussed above, after the manufacturer obtains the delay calibration value by using the apparatus for calibrating microphones, the delay calibration value may be stored in the apparatus for calibrating the microphone array, and the apparatus performs subsequent processing, so that the delay calibration value may be used at any time.
In the scenario two discussed above, after the user purchases the corresponding microphone calibration device, the microphone calibration device may be calibrated at a preset time interval, and in the subsequent processing, the latest delay calibration value is delayed.
On the basis of the method for calibrating a microphone array discussed above, an embodiment of the present invention further provides a sound source calibration method, please refer to fig. 4, which includes the following specific processes:
and step 402, obtaining the position of the sound source point to be detected based on the target sound data of the microphones except the compensated microphone in the other microphones and the target sound data of the compensated microphone.
The method is performed by a sound source localization arrangement comprising or being equivalent to a device for calibrating a microphone array.
The general idea of the embodiment of the application is as follows:
before positioning a sound source each time, the sound source positioning device may obtain, through the method for calibrating a microphone array discussed above, a time delay calibration value corresponding to each microphone that needs to be calibrated from among the other microphones, perform time delay compensation on sound data of the microphone according to the obtained time delay calibration value, and obtain a position of the sound source according to the compensated sound data. And because the time delay calibration value is more accurate, the relatively obtained sound source position is more accurate.
The following describes a sound source localization process performed by the sound source localization apparatus.
The sound source localization apparatus executes step 401, that is, the time delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones is obtained according to the method discussed in fig. 2, so as to compensate the target sound data acquired by the remaining microphones from the sound source point to be measured.
For ease of understanding, the sound source point to be measured and the set sound source point are described below, and the set sound source point is understood to be a sound source point placed at the set sound source position 130 when the array of microphones is calibrated. The sound source point to be measured is the sound source point which needs to be positioned, and the sound source point can be positioned at any position.
Specifically, the content of obtaining the delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones may refer to the content discussed in fig. 1, and is not described herein again. The sound source positioning device may acquire the time delay calibration value corresponding to each microphone before positioning the sound source each time. Or, the sound source positioning device may obtain the time delay calibration value only once, and after obtaining the time delay calibration value, the time delay calibration value is used as the standard when the sound source is subsequently positioned. Or, the sound source positioning device may obtain the time delay calibration value once at a preset interval time, and after obtaining the current time delay calibration value, the sound source positioning device will take the time delay calibration value as the standard when positioning the sound source next time.
The sound source positioning device can collect the sound data of the sound source point to be detected through the other microphones, and the sound data is also the target sound data. The target sound data may be time domain sound data or frequency domain sound data. And the sound source positioning device carries out compensation calibration on the target sound data according to the time delay calibration value obtained in the foregoing.
The manner in which step 401 is performed is explained below.
The manner of executing step 401 is:
and the sound source positioning device compensates the target sound data collected by the rest microphones from the sound source point to be detected according to the time delay calibration value corresponding to each microphone.
For example, the sound source positioning device determines that the time delay calibration value corresponding to the ith microphone needing calibration in the rest microphones is τ ', and when the ith microphone is calibrated, the time delay difference of the ith microphone is (τ - τ').
After the sound source localization apparatus performs step 401, it performs step 402 to obtain the position of the sound source point to be measured based on the target sound data of the microphones other than the compensated microphone among the remaining microphones and the target sound data of the compensated microphone. The following description is made.
One way to perform step 402 is to:
the sound source positioning device constructs a cost function between sound data of every two microphones in the rest microphones;
obtaining the maximum value of cost functions of every two microphones in the rest microphones based on the value range of the sound source position;
and determining the position corresponding to the time delay difference corresponding to the maximum value as the position of the sound source to be detected.
Specifically, the device for calibrating the microphone array may first construct a cost function between sound data of two microphones, traverse the cost function according to a preset sound source position range, and obtain a maximum value of the cost function, where a position corresponding to the maximum value is a position of a sound source to be detected.
Wherein the sound source position comprises an azimuth angle of the sound source and a pitch angle of the sound source. The azimuth angle may be understood as an angle to the microphone array 120 in the horizontal direction, and the pitch angle may be understood as an angle to the microphone array 120 in the vertical direction. The sound source position range refers to an azimuth angle of the sound source, and a pitch angle of the sound source.
A specific expression of the cost function is as follows:
where s represents the length of the sound data of the ith microphone and the h-th microphone, and s may be the same as or different from L in the foregoing. Other letter meanings in the formula can refer to the contents discussed in the foregoing, and are not described in detail here.
The sound source positioning device is pre-stored with corresponding relations between different sound source positions and different time delay differences, the sound source positioning device obtains a corresponding time delay difference range according to a preset sound source position range, then the corresponding time delay difference in the time delay difference range is substituted into the cost function, when one microphone in every two microphones corresponds to a time delay calibration value, the sound source positioning device compensates the time delay difference in the cost function according to the calibration time delay difference when calculating the value of the cost function, so that the values of the cost function corresponding to the different time delay differences are obtained, and the sound source position corresponding to the maximum value of the cost function is used as the position of the sound source to be detected.
Every two microphones correspondingly determine the position of the sound source to be measured, and the rest microphones have(N-M) can determine a plurality of
And position, averaging the positions to obtain the position of the sound source to be measured.
On the basis of the method for calibrating a microphone array discussed in fig. 2, the present application further provides an apparatus for calibrating a microphone array, please refer to fig. 5, the apparatus includes an acquisition module 501 and a processing module 502, wherein:
an acquisition module 501, configured to acquire sound data from a set sound source point through N microphones in a microphone array; the distances from the N microphones to a set sound source point are the same, and N is an integer greater than or equal to 3;
the processing module 502 is configured to obtain correlation coefficients of every two microphones in the N microphones by using a preset correlation coefficient algorithm according to sound data of the N microphones;
the processing module 502 is further configured to determine, according to a part of the correlation coefficients smaller than a preset correlation coefficient threshold, corresponding M associated microphones; wherein M is an integer less than N;
the processing module 502 is further configured to obtain a time delay difference of sound data collected by each two microphones of the rest microphones, based on sound data of each two microphones of the rest microphones of the N microphones except the M microphones;
the processing module 502 is further configured to determine, based on the delay difference, a delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones.
In a possible implementation, the processing module 502 is specifically configured to:
intercepting sound data of each microphone in every two microphones in N microphones with preset lengths to obtain the intercepted sound data of every two microphones in the N microphones;
and normalizing the intercepted sound data of every two microphones in the N microphones to obtain the correlation coefficients of every two microphones in the N microphones.
In a possible implementation, the processing module 502 is specifically configured to:
constructing a cost function between sound data of every two microphones in the rest microphones;
obtaining the maximum value of the cost function of every two microphones in the other microphones based on the preset time delay difference value range;
and determining the time delay difference corresponding to the maximum value as the time delay difference of the sound data collected by every two microphones in the rest microphones.
In a possible implementation, the processing module 502 is specifically configured to:
determining at least two delay differences, wherein the difference between the delay difference and the average value of the delay differences is larger than a preset difference;
and determining a calibration value corresponding to at least one microphone associated with the at least two time delay differences according to the average value and the at least two time delay differences, thereby obtaining the time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones.
On the basis of the sound source localization method discussed in fig. 4, an embodiment of the present application further provides a sound source localization apparatus, please refer to fig. 6, the apparatus includes an obtaining module 601 and a processing module 602, where:
an obtaining module 601, configured to obtain, by using the method discussed in fig. 2, a time delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones;
the processing module 602 is configured to compensate, according to a time delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones, target sound data acquired by the remaining microphones from the sound source point to be measured;
the processing module 602 is further configured to obtain a position of a sound source point to be measured based on target sound data of microphones other than the compensated microphone among the other microphones and the target sound data of the compensated microphone.
On the basis of the discussion in the foregoing fig. 2 or fig. 4, the embodiment of the present application further provides an intelligent device, such as an intelligent sound, a ball machine, or an intelligent voice assistant. Referring to fig. 7, the intelligent device includes a processor 701 and a memory 702, wherein:
the memory 702 stores instructions executable by the processor 701, and the processor 701 implements the method as discussed in fig. 2 or fig. 4 by executing the instructions stored by the memory 702.
Fig. 7 illustrates an example of one processor 701, but the number of processors 701 is not limited in practice.
As an embodiment, the apparatus for calibrating a microphone array in fig. 5 may be implemented by the processor 701 in fig. 7.
As an example, the sound source localization apparatus in fig. 6 may be implemented by the processor 701 in fig. 7.
On the basis of the method discussed in the foregoing fig. 2 or fig. 4, the present application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the method as discussed in fig. 2 or fig. 4.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (9)
1. A method of calibrating a microphone array, comprising:
collecting sound data from a set sound source point by N microphones in a microphone array; the distances from the N microphones to the set sound source point are the same, and N is an integer greater than or equal to 3;
obtaining the correlation coefficients of every two microphones in the N microphones by adopting a preset correlation coefficient algorithm according to the sound data of the N microphones;
determining M correlated microphones according to the correlation coefficients which are smaller than a preset correlation coefficient threshold value; wherein M is an integer less than N;
constructing a cost function between sound data of every two microphones in the rest microphones except the M microphones in the N microphones;
obtaining the maximum value of the cost function of every two microphones in the rest microphones based on the preset time delay difference value range;
determining the time delay difference corresponding to the maximum value as the time delay difference of the sound data collected by every two microphones in the rest microphones;
and determining a time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones based on the time delay difference.
2. The method of claim 1, wherein obtaining the correlation coefficient of each two microphones in the N microphones by using a preset correlation coefficient algorithm according to the sound data of the N microphones comprises:
intercepting sound data of each microphone in every two microphones in the N microphones with preset lengths to obtain the intercepted sound data of every two microphones in the N microphones;
and normalizing the intercepted sound data of every two microphones in the N microphones to obtain the correlation coefficients of every two microphones in the N microphones.
3. The method of claim 1, wherein determining a time delay calibration value for each of the remaining microphones requiring calibration based on the time delay difference comprises:
determining at least two delay differences, wherein the difference between the delay difference and the average value of the delay differences is larger than a preset difference;
and determining a calibration value corresponding to at least one microphone associated with the at least two time delay differences according to the average value and the at least two time delay differences, thereby obtaining the time delay calibration value corresponding to each microphone needing to be calibrated in the rest microphones.
4. A sound source localization method, comprising:
compensating the target sound data collected by the rest microphones from the sound source point to be measured according to the time delay calibration value corresponding to each microphone to be calibrated in the rest microphones obtained by the method of any one of claims 1 to 3;
and obtaining the position of the sound source point to be detected based on the target sound data of the microphones except the compensated microphone in the other microphones and the target sound data of the compensated microphone.
5. An apparatus for calibrating a microphone array, comprising:
the detection module is used for collecting sound data from a set sound source point through N microphones in the microphone array; the distances from the N microphones to the set sound source point are the same, and N is an integer greater than or equal to 3;
the processing module is used for obtaining the correlation coefficients of every two microphones in the N microphones by adopting a preset correlation coefficient algorithm according to the sound data of the N microphones;
the processing module is further configured to determine, according to a part of the correlation coefficients smaller than a preset correlation coefficient threshold, corresponding M associated microphones; wherein M is an integer less than N;
the processing module is further configured to construct a cost function between sound data of every two microphones of the rest microphones, except the M microphones, of the N microphones; obtaining the maximum value of the cost function of every two microphones in the rest microphones based on the preset time delay difference value range; determining the time delay difference corresponding to the maximum value as the time delay difference of the sound data collected by every two microphones in the rest microphones;
the processing module is further configured to determine, based on the time delay difference, a time delay calibration value corresponding to each microphone that needs to be calibrated among the other microphones.
6. The device of claim 5, wherein the processing module is specifically configured to:
intercepting sound data of each microphone in every two microphones in the N microphones with preset lengths to obtain the intercepted sound data of every two microphones in the N microphones;
and normalizing the intercepted sound data of every two microphones in the N microphones to obtain the correlation coefficients of every two microphones in the N microphones.
7. A sound source localization apparatus, comprising:
a compensation module, configured to compensate, by using the time delay calibration value corresponding to each microphone that needs to be calibrated in the remaining microphones obtained by the method according to any one of claims 1 to 3, target sound data acquired by the remaining microphones from the sound source point to be measured;
and the processing module is used for obtaining the position of the sound source point to be detected based on the target sound data of the microphones except the compensated microphone in the other microphones and the target sound data of the compensated microphone.
8. A smart device, comprising:
at least one processor, and
a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the method of any one of claims 1-3 or 4 by executing the instructions stored by the memory.
9. A computer-readable storage medium having stored thereon computer instructions which, when executed on a computer, cause the computer to perform the method of any one of claims 1-3 or 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910328573.XA CN110068797B (en) | 2019-04-23 | 2019-04-23 | Method for calibrating microphone array, sound source positioning method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910328573.XA CN110068797B (en) | 2019-04-23 | 2019-04-23 | Method for calibrating microphone array, sound source positioning method and related equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110068797A CN110068797A (en) | 2019-07-30 |
| CN110068797B true CN110068797B (en) | 2021-02-02 |
Family
ID=67368360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910328573.XA Active CN110068797B (en) | 2019-04-23 | 2019-04-23 | Method for calibrating microphone array, sound source positioning method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110068797B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110400571B (en) * | 2019-08-08 | 2022-04-22 | Oppo广东移动通信有限公司 | Audio processing method and device, storage medium and electronic equipment |
| CN110441737B (en) * | 2019-08-26 | 2023-02-24 | 上海声茵科技有限公司 | Sound source positioning method and equipment adopting fisheye lens |
| CN110677780B (en) * | 2019-09-26 | 2021-12-14 | 北京小米移动软件有限公司 | Detection method and device of audio input module and storage medium |
| CN113297911A (en) * | 2021-04-25 | 2021-08-24 | 普联国际有限公司 | Microphone array-based positioning error correction method and device and storage medium |
| CN113791386B (en) * | 2021-08-06 | 2024-03-29 | 浙江大华技术股份有限公司 | Sound source positioning method, device, equipment and computer readable storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6912178B2 (en) * | 2002-04-15 | 2005-06-28 | Polycom, Inc. | System and method for computing a location of an acoustic source |
| WO2009012354A1 (en) * | 2007-07-17 | 2009-01-22 | Clemson University | System and method to assess signal similarity with applications to diagnostics and prognostics |
| JP5235723B2 (en) * | 2009-03-02 | 2013-07-10 | 日本電信電話株式会社 | Utterance direction estimation apparatus, method and program |
| CN103873979A (en) * | 2012-12-11 | 2014-06-18 | 联想(北京)有限公司 | Sound source location acquisition method and electronic device |
| CN103995252A (en) * | 2014-05-13 | 2014-08-20 | 南京信息工程大学 | Three-dimensional space sound source positioning method |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN105467364A (en) * | 2015-11-20 | 2016-04-06 | 百度在线网络技术（北京）有限公司 | Method and apparatus for localizing target sound source |
| CN106886010A (en) * | 2017-01-17 | 2017-06-23 | 南京航空航天大学 | A kind of sound bearing recognition methods based on mini microphone array |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8374854B2 (en) * | 2008-03-28 | 2013-02-12 | Southern Methodist University | Spatio-temporal speech enhancement technique based on generalized eigenvalue decomposition |
| RU2493617C2 (en) * | 2008-09-11 | 2013-09-20 | Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. | Apparatus, method and computer programme for providing set of spatial indicators based on microphone signal and apparatus for providing double-channel audio signal and set of spatial indicators |
| CN101685153B (en) * | 2008-09-28 | 2011-11-16 | 华为终端有限公司 | Microphone space measuring method and device |
| KR101612704B1 (en) * | 2009-10-30 | 2016-04-18 | 삼성전자 주식회사 | Apparatus and Method To Track Position For Multiple Sound Source |
| CN108318862B (en) * | 2017-12-26 | 2021-08-20 | 北京大学 | Sound source positioning method based on neural network |
| CN108802687A (en) * | 2018-06-25 | 2018-11-13 | 大连大学 | The more sound localization methods of distributed microphone array in reverberation room |
-
2019
- 2019-04-23 CN CN201910328573.XA patent/CN110068797B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6912178B2 (en) * | 2002-04-15 | 2005-06-28 | Polycom, Inc. | System and method for computing a location of an acoustic source |
| WO2009012354A1 (en) * | 2007-07-17 | 2009-01-22 | Clemson University | System and method to assess signal similarity with applications to diagnostics and prognostics |
| JP5235723B2 (en) * | 2009-03-02 | 2013-07-10 | 日本電信電話株式会社 | Utterance direction estimation apparatus, method and program |
| CN103873979A (en) * | 2012-12-11 | 2014-06-18 | 联想(北京)有限公司 | Sound source location acquisition method and electronic device |
| CN103995252A (en) * | 2014-05-13 | 2014-08-20 | 南京信息工程大学 | Three-dimensional space sound source positioning method |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN105467364A (en) * | 2015-11-20 | 2016-04-06 | 百度在线网络技术（北京）有限公司 | Method and apparatus for localizing target sound source |
| CN106886010A (en) * | 2017-01-17 | 2017-06-23 | 南京航空航天大学 | A kind of sound bearing recognition methods based on mini microphone array |
Non-Patent Citations (3)
| Title |
|---|
| 《Acoustic source tracking in long baseline microphone arrays》;Kim Arild Steen、James H.McClellan、Ole Green、Henrik Karstoft;《Applied Acoustics》;20150131;第87卷;38-45 * |
| 《基于麦克阵列的点声源定位及声源类型分析》;王晓东;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170215(第2期);I136-506 * |
| 《声源定位系统的设计与实现》;李哲;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160315(第3期);I136-619 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110068797A (en) | 2019-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110068797B (en) | Method for calibrating microphone array, sound source positioning method and related equipment | |
| CN103781095B (en) | A kind of bearing calibration of TDOA measure error, transfer point and system | |
| CN108414974B (en) | Indoor positioning method based on ranging error correction | |
| CN110475339B (en) | Time difference of arrival positioning method and device | |
| US10834505B2 (en) | Apparatus and a method for unwrapping phase differences | |
| EP2732301B1 (en) | Sound source localization using phase spectrum | |
| JP2017215208A (en) | Pulse radar | |
| GB2431051A (en) | System for correcting amplitude and phase errors across an antenna array | |
| US11076250B2 (en) | Microphone array position estimation device, microphone array position estimation method, and program | |
| CN107171981B (en) | Channel correction method and device | |
| WO2017012337A1 (en) | Uplink time adjustment method and apparatus | |
| CN114679227A (en) | Space frequency domain correction method for direction finding error | |
| US10830861B2 (en) | Method for direction finding and direction finder | |
| CN114089276A (en) | Self-adaptive passive positioning method and system for underwater sound source | |
| CN111928854B (en) | Millimeter wave indoor positioning and angle estimation method and system based on multiple access points | |
| JP5438466B2 (en) | Image processing apparatus and image processing method | |
| US10151823B2 (en) | Method for passive approximate localization using frequency modulation and software defined radio | |
| CN112750455A (en) | Audio processing method and device | |
| CN109725188B (en) | Current measuring method and device | |
| CN115856532A (en) | Transformer partial discharge source positioning method based on ultrasonic propagation path identification | |
| CN106550447B (en) | Terminal positioning method, device and system | |
| JP6349214B2 (en) | Position estimation apparatus and position estimation program | |
| KR20140070723A (en) | Apparatus and method for evaluating statistical interference concerning multi-source noise environment of electromagnetic zone | |
| CN117406173B (en) | Beam forming method and device for sound source detection | |
| JP2018032931A (en) | Acoustic signal processing device, program and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |