JP2007129373A - Method and system for adjusting sensitivity of microphone - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a system of adjusting sensitivity in microphones capable of providing uniform sensitivity in a plurality of microphones, and capable of forming desired directional patterns. <P>SOLUTION: When the directional patterns are controlled for achieving various kinds of processing such as sound source separation and speech recognition; a calibration signal for sensitivity adjustment is inputted into a plurality of microphones 41, 42, and the output signals are used for creating information for calibration. For example, the spectrum ratio of each output signal is obtained for each frequency band, and an impulse response is obtained based on the obtained spectrum ratio. When various kinds of equipment, such as a portable telephone, are in an actual operating state for performing various kinds of processing, such as sound source separation and speech recognition; the information for calibration having been created by calibration processing is used for adjusting the sensitivity. For example, the obtained impulse response is used for performing convolutional arithmetic operation to correct the output signal of the microphone 42, and the sensitivity is adjusted to the same level as that of the microphone 41 to form target directional patterns. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a microphone sensitivity adjustment method and system for adjusting sensitivity between a plurality of microphones used in combination to form a desired directional characteristic, for example, a portable device such as a mobile phone, a car navigation system, and the like. It can be used for adjusting the sensitivity between a plurality of microphones mounted on an in-vehicle device.

  In normal speech recognition, speech uttered at the mouth is recorded by a close-talking microphone and recognition processing is performed. On the other hand, there are many applications where it is unnatural to impose the use of a close-talking microphone on the user, such as dialogue with a robot, voice operation on a vehicle-mounted device such as a car navigation system, and creation of meeting minutes. In such an application, it is desired to record and recognize a voice using a microphone installed on the system side. However, when sound collection and speech recognition are performed with a microphone placed away from the speaker, the S / N ratio deteriorates, making it difficult to hear and the accuracy of speech recognition extremely deteriorates.

  In response to such a problem, the applicant of the present application has developed a system that selectively records only desired sound by performing directivity control and band selection using a microphone array ( Patent Document 1). In addition, as a directivity control using a small number of microphones, a super-directional microphone using two unidirectional microphone units (see Patent Document 2) and a multi-directional using four omnidirectional microphones. There is a sound collecting device for channel stereo (see Patent Document 3). There is also a microphone device (see Patent Document 4) in which three pairs of microphones are arranged around a reference microphone. Recently, the applicant of the present application performs directivity control by performing linear combination processing for target sound enhancement and linear combination processing for target sound suppression using a small number of microphone output signals. A system for separating target sound and interference sound by performing band selection and spectral subtraction using the spectrum of a plurality of signals obtained by characteristic control has been developed (Japanese Patent Application No. 2004). -366202, Japanese Patent Application No. 2005-270931). In any of the above cases, directivity control is performed using the output signals of a plurality of microphones.

For example, as shown in FIG. 9, when directional characteristics control is performed using two omnidirectional microphones 901 and 902, (1) the output signal X1 of _one microphone 901 and the other microphone 902 If a signal (X ₁ −X ₂ ) that is different from the output signal X ₂ is generated, a directional characteristic having an approximately 8 shape as shown by a solid line in the figure can be obtained. (2) The output signal of one microphone 901 be generated a difference between the signal of the X ₁ signal subjected to the delay processing in the D and (X ₁₎ and the output signal X ₂ of the other microphone _{902 (D (X 1) -X} 2), as indicated by the dotted line in FIG. (3) A signal D (X ₂ ) obtained by delaying the output signal X ₂ of the other microphone 902 and the output signal X ₁ of the _one microphone 901 are obtained. difference signal (D (X ₂₎ -X ₁ By generating the obtained directivity characteristic cardioid (heart-shaped) as shown by a chain line in the drawing, the output signal X ₂ of the output signal X ₁ and the other microphone 902 of one of the microphone 901 (4) If a sum signal (X ₁ + X ₂ ) is generated, a substantially elliptical directivity characteristic as shown by a two-dot chain line in the figure can be obtained. In performing various processes such as sound source separation and voice recognition, these directivity characteristics may be used alone or in combination.

Japanese Patent Laying-Open No. 2005-77731 (Claim 1, FIG. 1, FIG. 2, summary) Japanese Patent Laid-Open No. 10-126876 (Claim 1, FIG. 1, FIG. 2, Summary) JP 2002-223493 A (Claim 1, FIG. 1, FIG. 3, summary) JP 2002-271885 A (Claim 1, FIG. 1, FIG. 11, summary)

  However, when various directional characteristics as shown in the example of FIG. 9 described above are formed, if the sensitivities of the two microphones 901 and 902 are different, the directional characteristics having the shape shown in the figure can be formed. Therefore, the sound source separation performance is deteriorated.

In particular, in FIG. 9, (1) in the directivity characteristics of the difference signal (X ₁ −X ₂ ), the sound from that direction is almost completely suppressed in the directions of θ = 90 degrees and θ = −90 degrees. (2) In the directivity characteristic of the signal (D (X ₁ ) −X ₂ ) obtained after delay processing is performed on the output signal X ₁ of _one microphone 901, the direction of θ = 0 degrees (3) A signal (D (X ₂ ) −X obtained by performing a delay process on the output signal X ₂ of the other microphone 902 and then taking a difference between them. _In the directivity characteristic of ₁ ), there is a portion where the sound from the direction is almost completely suppressed in the direction of θ = 180 degrees (−180 degrees), but the sensitivity of the two microphones 901 and 902 is different. , Where the graph shape showing the directional characteristics of these parts is zero (center position) ) From away, it becomes impossible to suppress the sound. For this reason, the difference in sensitivity between the microphones 901 and 902 has a great influence on the directivity formed by taking the signal difference (including the case where the difference is taken after delay processing).

  The same applies to directivity control using three or more microphones. If the sensitivity of each microphone is different, the desired directivity cannot be formed, leading to deterioration of sound source separation performance, etc. become.

  An object of the present invention is to provide a microphone sensitivity adjusting method and system capable of aligning the sensitivities of a plurality of microphones and forming a desired directional characteristic.

  The present invention is a microphone sensitivity adjustment method for adjusting the sensitivity between a plurality of microphones used in combination to form a target directivity characteristic, and the same calibration signal for sensitivity adjustment is input to the plurality of microphones, The calibration information for adjusting the sensitivity of the plurality of microphones to the same level of sensitivity is created using the output signals of the plurality of microphones at this time. It is used to correct an output signal of at least one of the plurality of microphones.

  In such a microphone sensitivity adjustment method of the present invention, in performing directivity control for realizing various processes such as sound source separation and voice recognition, calibration is performed by inputting calibration signals for sensitivity adjustment to a plurality of microphones in advance. Process and create calibration information. Then, when the target directivity characteristic is formed in a scene where various processes such as sound source separation and voice recognition are actually performed, at least one of the plurality of microphones is used by using the calibration information obtained in the calibration process. Correct the output signal of the microphone.

  For this reason, since a plurality of microphones are arranged at the same level of sensitivity, it becomes possible to form desired directivity characteristics, and the performance of various processes such as sound source separation and voice recognition can be improved. In particular, when the directivity is formed by taking the difference between the output signals of a plurality of microphones (including the case where the difference is taken after delay processing), a desired suppression effect is obtained in the direction in which the incoming sound is to be suppressed. Therefore, the performance of various processes such as sound source separation and voice recognition can be improved more remarkably, thereby achieving the object.

  More specifically, for example, the following method can be employed. That is, in the microphone sensitivity adjustment method described above, when obtaining calibration information, one of the plurality of microphones is selected as a reference microphone that serves as a reference for sensitivity adjustment, and microphones other than the reference microphone are selected. Make the output signal to be corrected so that the sensitivity is the same level as the reference microphone, and input the same calibration signal for sensitivity adjustment to the reference microphone and the correction microphone. Each output signal is subjected to Fourier transform, and the ratio of the spectrum of the output signal of the reference microphone obtained by the Fourier transform and the spectrum of the output signal of the correction target microphone is obtained for each frequency band. Spect The impulse response as calibration information is obtained by inverse Fourier transform of the ratio, and the convolution calculation is performed using the impulse response as the calibration information and the output signal of the correction target microphone when forming the desired directivity characteristics. By performing the above, it is possible to obtain a corrected signal of the correction target microphone.

  Here, the number of “correction target microphones” may be one or plural.

  The sound source of the “calibration signal” may be an environmental sound (eg, human voice, noise, car horn), or a mechanical calibration sound output from a speaker. Also, in the case where human speech is used as the calibration signal among the environmental sounds, whether or not the voiced sound section is determined when the average value of the effective values (for example, the average value of about 20 ms) exceeds a certain level. It may be determined and adopted as a calibration signal when it is a voiced sound section. Note that the calibration signal does not necessarily need to be a dedicated signal for the calibration process, and in a situation where the same signal at the same level can be simultaneously input to a plurality of microphones (reference microphone and correction target microphone), The environmental sound acquired in the actual working state may be used. For example, the voice of a call acquired during use of the mobile phone may be used as a calibration signal. In this case, there are a plurality of users using the mobile phone. It is only necessary to speak from a symmetrical position with respect to the microphone (reference microphone and correction target microphone). The reference to the “calibration signal” here is not limited to the case where the impulse response is obtained by performing the Fourier inverse transform on the spectrum ratio for each frequency band as described above, but for all the calibration processes performed in the present invention. The same is true.

  Further, the calibration process for obtaining the impulse response may be preset at the factory, or automatic adjustment in an actual working state (equipment use state) after shipment from the factory. In the case of the former, for example, if the microphone is mounted on various devices (products) such as portable devices such as mobile phones and in-vehicle devices such as a car navigation system, before those devices (products) are shipped from the factory. The calibration process can be performed in advance at the factory. In the latter case, after the various devices reach the user's hands, the user is using the devices (working state), immediately before starting use, immediately after using the device, or not using it. The calibration process can be performed automatically when the device is turned on. And when performing calibration processing while using the device, the impulse response for the output correction processing may be fixed by performing the calibration processing only once immediately after starting the device use. The impulse response for output correction processing may be updated periodically during calibration, or when calculating the spectrum ratio for each frequency band as the average value for multiple times as described later, use the equipment During this process, the spectrum ratio for each frequency band is continuously obtained, the moving average value or cumulative average value for the spectrum ratio for each frequency band is obtained, and the spectrum ratio updated by this moving average or cumulative average is used. The impulse response for output correction processing may be updated. Further, when the calibration process is automatically performed after the factory shipment, the timing for performing the process is a predetermined time (for example, every night at 22:00, every Monday at 22:00, every month at 22:00). However, it may be linked to the operation of the device (operation other than calibration processing) or user operation. For example, in the case of a mobile phone, when a call signal is received or sent, or when a folding mobile phone is opened or closed The calibration process may be performed corresponding to the above. Note that it is preferable to make complete automatic adjustment including control of the timing for performing the calibration process from the viewpoint of reducing the user's effort and avoiding the influence of the user's forgetting the instruction. However, the adjustment by the user instruction, that is, the calibration process itself Is automatically performed mechanically, but the user may give an instruction to start the calibration process. The timing of performing the calibration process here is not limited to the case where the impulse response is obtained by performing Fourier inverse transform on the spectrum ratio for each frequency band, but the calibration process for creating the calibration information in the present invention. The same can be said for all of the above.

  As described above, the action and effect of the present invention will be described more specifically by taking the more specific microphone sensitivity adjustment method as an example. In this example, when performing directional characteristics control for realizing various processes such as sound source separation and voice recognition, when performing calibration processing by inputting calibration signals for sensitivity adjustment to a plurality of microphones in advance. The spectral ratio between the output signal of the reference microphone and the output signal of the correction target microphone when the calibration signal is input is obtained for each frequency band, and the impulse response is obtained based on the spectral ratio for each frequency band. This impulse response becomes information for calibration. When the desired directivity characteristics are formed in various scenes such as sound source separation and voice recognition, the convolution calculation is performed by using the impulse response obtained in the calibration process. The output signal is corrected, and the sensitivity of the correction target microphone is adjusted to the same level as that of the reference microphone.

  For this reason, since the reference microphone and the correction target microphone have the same level of sensitivity, it becomes possible to form a desired directional characteristic, and the performance of various processes such as sound source separation and voice recognition can be improved. In particular, when the directivity is formed by taking the difference between the output signals of a plurality of microphones (including the case where the difference is taken after delay processing), a desired suppression effect is obtained in the direction in which the incoming sound is to be suppressed. Therefore, the performance of various processes such as sound source separation and voice recognition can be improved more remarkably.

  In the microphone sensitivity adjustment method described above, when obtaining the impulse response, the average value of the spectrum ratio for each frequency band is obtained as calibration information, and the difference between the obtained average value and 1 is obtained for each frequency band. After adjusting so that the average value of the spectral ratio for each frequency band becomes 1 by subtracting from the spectral ratio, the inverse spectral Fourier transform is performed on the spectral ratio for each frequency band after adjustment to form the desired directivity characteristics. When performing the convolution operation using the impulse response and the output signal of the microphone to be corrected, the average value of the spectral ratios for each frequency band as the calibration information is added to the signal obtained by performing the convolution operation. The corrected signal of the correction target microphone may be obtained by multiplying.

  Furthermore, in the above-described microphone sensitivity adjustment method, it is desirable to set the average value for a plurality of times when obtaining the spectrum ratio for each frequency band.

  Thus, when obtaining the spectrum ratio for each frequency band as an average value for a plurality of times, stable calibration information can be obtained, so that more appropriate correction processing is performed on the output signal of the correction target microphone. It becomes possible.

  In the above-described microphone sensitivity adjustment method, when obtaining the spectral ratio for each frequency band, it is desirable to interpolate and correct the spectral ratio indicating an abnormal value using the spectral ratio of the previous and subsequent frequency bands.

  Here, the “spectral ratio of the frequency bands before and after” indicates the spectral ratio of the frequency band immediately before (low frequency side) or after (high frequency side) the spectral ratio to be corrected for interpolation indicating an abnormal value. In addition, the spectral ratio of the frequency band two or more before or two or more after is included. Therefore, for example, interpolation correction processing of a spectral ratio indicating an abnormal value may be performed using spectral ratios of a total of four frequency bands, two before, one before, one after, and two after.

  In addition, “to interpolate and correct the spectral ratio indicating an abnormal value using the spectral ratio of the previous and subsequent frequency bands” includes (1) the spectral ratio of each frequency band using the spectral ratio of the previous and subsequent frequency bands. And calculating the spectral ratio after correction of interpolation using the spectral ratio of the front and rear frequency bands and replacing it with a spectral ratio indicating an abnormal value, and (2) the spectral ratio of the front and rear frequency bands. Not limited to, for example, whether or not the spectrum ratio of each frequency band is an abnormal value based on various indicators such as the degree of deviation from the average value of the spectrum ratio of each frequency band, and the frequency bands before and after It is included to obtain a spectral ratio after correction of interpolation using the spectral ratio of the above and replace it with a spectral ratio indicating an abnormal value.

  In this way, when the abnormal value of the spectrum ratio is corrected by interpolation, stable calibration information can be obtained, so that more appropriate correction processing can be performed on the output signal of the correction target microphone. It becomes.

  In the microphone sensitivity adjustment method described above, when obtaining the spectrum ratio for each frequency band, it is desirable to target the spectrum in the frequency band within the range of 100 Hz to 1 kHz.

  As described above, when the spectrum ratio for each frequency band is obtained for the spectrum of the frequency band in the range of 100 Hz to 1 kHz, the difference between the output signals of the plurality of microphones (the difference may be obtained after delay processing). It is such a low-frequency region that needs to ensure the sound suppression effect in a specific direction when forming directivity characteristics by taking these into account. Therefore, it is possible to obtain appropriate and effective calibration information with a small amount of calculation.

  Further, in the microphone sensitivity adjustment method described above, the sensitivity between a plurality of microphones mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion is adjusted. When adjusting, a concave portion is formed on the surface of the main body portion on which the operation portion is provided, and a plurality of microphones are provided on the main body portion so as to face the space of the concave portion, and the operation portion of the main body portion is covered with a lid portion. When a speaker is provided on the lid part in a state where it is placed at a position opposite to the concave part, and calibration information is obtained, the operation part of the main body part is covered with the lid part and the speaker is used for sensitivity adjustment. A calibration signal may be output, and the output calibration signal may be input to a plurality of microphones via the recess space.

  Here, the “mobile device” includes, for example, an electronic dictionary and an electronic notebook in addition to a mobile phone (including PHS) and a personal digital assistant (PDA). The same applies to the following inventions.

  In this way, when the calibration signal is output from the speaker to the concave portion with the operation portion of the main body portion of the portable device covered with the lid portion, the mobile phone configured to include the main body portion and the lid portion. Calibration processing can be realized using the structure of the device.

  In the above, when the mobile device is a foldable mobile phone (including PHS) including a main body portion provided with an operation unit and a lid portion provided with a screen display unit. When the mobile phone is folded, a speaker is provided on the lid so that the speaker is placed in a position opposite to the recess. When calibration information is obtained, the sensitivity is adjusted from the speaker while the mobile phone is folded. The calibration signal can be output, and the output calibration signal can be input to a plurality of microphones via the space of the recess.

  Further, in the above, when the portable device is a foldable personal digital assistant (PDA) configured to include a main body portion and a lid portion (the screen display unit is provided on either the main body portion or the lid portion) In other words, when the portable information terminal is folded, the speaker is provided at the lid portion so that the portable information terminal is disposed at a position facing the concave portion, and the portable information terminal is folded when obtaining the calibration information. In this state, a calibration signal for sensitivity adjustment can be output from the speaker, and the output calibration signal can be input to a plurality of microphones via the space of the recess.

  As described above, when the calibration signal is output from the speaker to the recess while the cellular phone or the portable information terminal is folded, the calibration process is performed using the structure of the folding cellular phone or the portable information terminal. Can be realized.

  Furthermore, in the microphone sensitivity adjustment method described above, the sensitivity between a plurality of microphones mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion is adjusted. When adjusting, a concave portion is formed on the surface of the main body portion where the operation portion is provided, and a plurality of microphones are provided on the main body portion so as to face the space of the concave portion, and the operation portion of the main body portion is covered with a lid portion. When a sound path that connects the space of the recess that is sometimes closed by the lid part and the external space is formed in the main body part or the lid part, In a state where the cover is covered with the lid, a calibration signal for sensitivity adjustment may be guided from the external space into the space of the recess through the sound path and input to a plurality of microphones.

  As described above, when the calibration signal is guided from the external space into the concave space through the sound path with the operation portion of the main body portion of the portable device covered with the lid portion, the main body portion, the lid portion, The calibration process can be realized by using the structure of the portable device that is provided with the above.

  In the above, when the mobile device is a foldable mobile phone (including PHS) including a main body portion provided with an operation unit and a lid portion provided with a screen display unit. When the mobile phone is folded, the sound path is formed in the main body portion in a state where the space of the recessed portion that is closed by the lid portion and the external space are communicated, and the calibration information is obtained. In a state where the cellular phone is folded to close the concave space, a calibration signal for sensitivity adjustment is guided from the external space into the concave space via the sound path and input to a plurality of microphones. Can do.

  Further, in the above, when the portable device is a foldable personal digital assistant (PDA) configured to include a main body portion and a lid portion (the screen display unit is provided on either the main body portion or the lid portion) The sound path is formed in the main body portion or the lid portion in a state where the space of the recess that is closed by the lid portion when the portable information terminal is folded and the external space communicate with each other. When obtaining the calibration information, the calibration signal for sensitivity adjustment is guided from the external space into the concave space through the sound path with the portable information terminal folded to close the concave space. Can be input to a plurality of microphones.

  As described above, when the calibration signal is guided from the external space into the recessed space through the sound path with the cellular phone or the portable information terminal folded, the folding cellular phone or the portable information terminal Calibration processing can be realized using the structure.

  Moreover, as a system for realizing the microphone sensitivity adjustment method of the present invention described above, the following microphone sensitivity adjustment system of the present invention can be cited.

  That is, the present invention is a microphone sensitivity adjustment system that adjusts the sensitivity between a plurality of microphones used in combination to form a desired directivity characteristic, and the same calibration signal for sensitivity adjustment is input to the plurality of microphones. Calibration information creation means for creating calibration information for adjusting the sensitivity of the plurality of microphones to the same level using the output signals of the plurality of microphones at the time, and the calibration created by the calibration information creation means Calibration information storage means for storing information for use, and output correction means for correcting an output signal of at least one of the plurality of microphones using the calibration information stored in the calibration information storage means. It is characterized by that.

  In such a microphone sensitivity adjustment system of the present invention, the actions and effects obtained by the above-described microphone sensitivity adjustment method of the present invention can be obtained as they are, thereby achieving the object.

  More specifically, for example, the following configuration can be adopted. That is, in the microphone sensitivity adjustment system described above, the calibration information creation means corrects the output signal so that one reference microphone serving as a reference for sensitivity adjustment among the plurality of microphones and the same level of sensitivity as this reference microphone are obtained. Fourier transform means for Fourier transforming each output signal of the reference microphone and the correction target microphone when the same calibration signal for sensitivity adjustment is input to the correction target microphone, and the output of the reference microphone obtained by this Fourier transform means Spectral ratio calculation means for obtaining the ratio of the spectrum of the signal and the spectrum of the output signal of the microphone to be corrected for each frequency band, and calibration by inverse Fourier transform of the spectral ratio for each frequency band obtained by the spectral ratio calculation means For information and And an inverse Fourier transform means for obtaining the impulse response, and the calibration information storage means comprises an impulse response storage means for storing the impulse response obtained by the Fourier inverse transform means, and is an output correction means. Can be configured to obtain a corrected signal of the correction target microphone by performing a convolution operation using the impulse response as the calibration information stored in the impulse response storage means and the output signal of the correction target microphone. .

  Further, in the microphone sensitivity adjustment system described above, the calibration information creation means calculates the spectrum ratio average value for obtaining an average value of these spectrum ratios as calibration information for the spectrum ratio for each frequency band obtained by the spectrum ratio calculation means. The average value of the spectrum ratio for each frequency band is adjusted to 1 by subtracting the difference between the average value obtained by the means and the spectrum ratio average value calculating means and 1 from the spectrum ratio for each frequency band. And a calibration ratio information storage unit including a spectrum ratio average value storage unit that stores an average value as calibration information obtained by the spectrum ratio average value calculation unit. The inverse transform means is adapted to the spectrum ratio for each frequency band after adjustment by the spectrum ratio adjustment means. The output correction means performs the convolution operation using the impulse response as the calibration information stored in the impulse response storage means and the output signal of the correction target microphone, and then performs the convolution operation. It is good also as a structure which multiplies the average value as the information for calibration memorize | stored in the spectrum ratio average value memory | storage means to the signal obtained by performing.

  Furthermore, in the above-described microphone sensitivity adjustment system, it is desirable that the spectrum ratio calculation unit is configured to take an average value a plurality of times when obtaining the spectrum ratio for each frequency band.

  In the microphone sensitivity adjustment system described above, the spectrum ratio calculation means interpolates and corrects the spectrum ratio indicating an abnormal value using the spectrum ratio of the previous and subsequent frequency bands when obtaining the spectrum ratio for each frequency band; It is desirable that

  Further, in the microphone sensitivity adjustment system described above, the spectrum ratio calculation means is configured to target a spectrum in a frequency band within a range of 100 Hz to 1 kHz when obtaining a spectrum ratio for each frequency band. It is desirable.

  Further, in the microphone sensitivity adjustment system described above, the plurality of microphones are mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion. A concave portion is formed on the surface provided with the operation portion, and a plurality of microphones are provided on the main body portion so as to face the space of the concave portion. When the operation portion of the main body portion is covered with a lid portion, In the state where the speaker is provided on the lid part in the state of being arranged in the state where the operation part of the main body part is covered with the lid part, a calibration signal for sensitivity adjustment is output from the speaker, and the output calibration signal passes through the recess space. It is good also as a structure input into a several microphone via.

  And in the above, when the mobile device is a foldable mobile phone configured to include a main body portion provided with an operation portion and a lid portion provided with a screen display portion, the mobile phone is When folded, a speaker is provided at the lid portion in a state of being placed at a position opposite to the concave portion, and with the cellular phone folded, a calibration signal for sensitivity adjustment is output from the speaker, and the output calibration signal is output from the concave portion. It can be set as the structure input into a some microphone via space.

  Moreover, in the above, when the portable device is a foldable portable information terminal configured to include a main body portion and a lid portion, the portable device is disposed at a position facing the recess when the portable information terminal is folded. In the state where the speaker is provided on the lid, and the portable information terminal is folded, the calibration signal for sensitivity adjustment is output from the speaker, and the output calibration signal is input to a plurality of microphones via the recess space. It can be set as a structure.

  Furthermore, in the microphone sensitivity adjustment system described above, the plurality of microphones are mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion. A concave portion is formed on the surface provided with the operation portion, and a plurality of microphones are provided on the main body portion so as to face the space of the concave portion. When the operation portion of the main body portion is covered with the lid portion, the lid portion closes it. A sound path that connects the space of the recessed portion that is in a detached state and the external space is formed in the main body part or the lid part, and the calibration signal for sensitivity adjustment is generated with the operation part of the main body part covered with the lid part. It is good also as a structure which is guide | induced into the space of a recessed part via an acoustic path from external space, and is input into a some microphone.

  And in the above, when the mobile device is a foldable mobile phone configured to include a main body portion provided with an operation portion and a lid portion provided with a screen display portion, the mobile phone is A sound path is formed in the main body part in a state where the space of the recessed portion that is closed by the lid portion when folded and the external space are communicated, and the space of the recessed portion is closed by folding the mobile phone, A calibration signal for sensitivity adjustment can be introduced from the external space into the concave space via the sound path and input to a plurality of microphones.

  Further, in the above, when the portable device is a foldable portable information terminal configured to include a main body portion and a lid portion, the portable device is closed by the lid portion when the portable information terminal is folded. The sound path is formed in the main body or lid part in a state where the concave space and the external space are in communication, and the calibration signal for sensitivity adjustment is externally closed when the portable information terminal is folded to close the concave space. It is possible to adopt a configuration in which a plurality of microphones are guided through the sound path into the recess space and input to a plurality of microphones.

  As described above, according to the present invention, calibration is performed by inputting calibration signals for sensitivity adjustment to a plurality of microphones in advance in order to form a desired directional characteristic for performing various processes such as sound source separation and voice recognition. The calibration information is created by performing the process, and when the target directivity characteristic is actually formed, the calibration information created by the calibration process is used to obtain at least one of the plurality of microphones. Since the sensitivity is adjusted by correcting the output signal, the sensitivities of a plurality of microphones can be made uniform, and desired directivity characteristics can be formed.

  An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows an overall configuration of a microphone sensitivity adjustment system 10 of the present embodiment. FIG. 2 is a perspective view of the mobile phone 40 equipped with the microphone sensitivity adjustment system 10. FIG. 3 is a flowchart showing the flow of calibration processing (calibration) by the microphone sensitivity adjustment system 10, and FIG. 4 shows acoustic signals (speech voices and voices) in an actual working state (a state in which the mobile phone 40 is in use). The flow of processing such as noise is shown in the flowchart.

  In FIG. 1, a microphone sensitivity adjustment system 10 includes a calibration signal output means 11, a window processing means 12, a Fourier transform means 13, a spectrum ratio calculation means 14, a spectrum ratio average value calculation means 15, and a spectrum ratio adjustment means. 16, an inverse Fourier transform unit 17, an output correction unit 18, a spectrum ratio average value storage unit 21, and an impulse response storage unit 22.

  Among these, the calibration signal output means 11, the window processing means 12, the Fourier transform means 13, the spectrum ratio calculation means 14, the spectrum ratio average value calculation means 15, the spectrum ratio adjustment means 16, and the Fourier inverse transform means 17 Thus, a calibration information creating means 10A for performing calibration processing and creating calibration information is configured. Further, the spectrum ratio average value storage means 21 and the impulse response storage means 22 constitute a calibration information storage means 20 for storing the calibration information created by the calibration information creation means 10A.

  The calibration signal output means 11 uses a calibration signal (in this embodiment, as a mechanical calibration sound as an example) to be supplied to the reference microphone 41 and the correction target microphone 42 mounted on the mobile phone 40. A process of outputting from the speaker 43 is performed. The calibration signal output means 11 also controls the output timing of the calibration signal. For example, it is ascertained whether or not the mobile phone 40 is in a folded state, and immediately after receiving a call signal in the folded state (in this case, the ring tone itself may be used as a calibration signal). The calibration signal is output from the speaker 43 immediately after the operation of folding the phone 40 is performed, or at a predetermined time (date and time) when the cellular phone 40 is folded.

Windowing means 12, the output signal X _C2 of the output signals X _C1 and corrected microphone 42 of the reference microphone 41 when inputting the same calibration signal for sensitivity adjustment, and performs the window processing, respectively.

  The Fourier transform unit 13 performs fast Fourier transform (FFT) on each output signal of the reference microphone 41 and the correction target microphone 42 after the window processing unit 12 performs window processing.

The spectrum ratio calculating means 14 calculates the ratio S ₁ / S ₂ between the spectrum S ₁ of the output signal of the reference microphone 41 obtained by the Fourier transform means 13 and the spectrum S ₂ of the output signal of the correction target microphone 42 in each frequency band. The processing to be obtained is performed every time. In the present embodiment, as an example, the spectrum ratio S ₁ / S ₂ is obtained for the spectrum of each frequency band within the range of 100 Hz to 1 kHz.

In addition, the spectrum ratio calculation means 14 obtains the spectrum ratio S ₁ / S ₂ for each frequency band, and the spectrum obtained by multiple times (N times) of fast Fourier transform processing for each frequency band. The average value of the ratio S ₁ / S ₂ is taken.

Furthermore, the spectrum ratio calculating means 14, when obtaining a spectral ratio S ₁ / S ₂ for each frequency band, the spectral ratio of each frequency band using the spectral ratio S ₁ / S ₂ before and after the frequency band S ₁ / S ₂ it is determined whether or not an abnormal value, when an abnormal value, the spectral ratio S ₁ / S ₂ indicating the abnormal value, the spectral ratio S ₁ / S ₂ before and after frequency bands The correction processing is performed by substituting the normal spectral ratio S ₁ / S ₂ calculated by performing the interpolation processing.

The spectrum ratio average value calculation means 15 obtains the average value R _AVE for the spectrum ratios S ₁ / S ₂ for each frequency band obtained by the spectrum ratio calculation means 14 (N average values in this embodiment). Is to do. That is, the average value R _AVE is obtained by summing the spectrum ratios S ₁ / S ₂ for each frequency band and dividing by the number of bands. This average value R _AVE becomes information for calibration.

The spectrum ratio adjusting means 16 subtracts the difference between the average value R _AVE obtained by the spectrum ratio average value calculating means 15 and 1 from the spectrum ratio S ₁ / S ₂ for each frequency band, thereby obtaining the frequency ratio for each frequency band. A process of adjusting the average value of the spectral ratios S ₁ / S ₂ to be 1 is performed.

The inverse Fourier transform means 17 performs an inverse Fourier transform (IFFT) on the spectrum ratio S ₁ / S ₂ (in this embodiment, N average values) for each frequency band obtained by the spectrum ratio calculation means 14 to generate an impulse. Processing for obtaining a response is performed. This impulse response is information for calibration. In the present embodiment, the inverse Fourier transform unit 17, the spectrum ratio adjusting device 16 spectral ratio for each frequency band after adjustment by S ₁ / S ₂ (average spectral ratio was adjusted to 1 S ₁ / Inverse Fourier transform is performed for S ₂ ).

The output correction unit 18 obtains a corrected signal of the correction target microphone 42 by performing a convolution operation using the impulse response stored in the impulse response storage unit 22 and the output signal X ₂ of the correction target microphone 42. The processing is performed. In the present embodiment, the output correction unit 18 performs a process of multiplying the signal obtained by performing the convolution operation by the average value R _AVE stored in the spectrum ratio average value storage unit 21 after performing the above convolution operation. .

The spectrum ratio average value storage means 21 stores an average value R _AVE that is calibration information obtained by the spectrum ratio average value calculation means 15. At this time, the stored average value R _AVE is one real value (for example, 1.1).

  The impulse response storage means 22 stores an impulse response that is calibration information obtained by the inverse Fourier transform means 17. At this time, the impulse response is stored as an array of values obtained by sampling the response waveform in the time domain.

  In the above, each means 11-18 which comprises the microphone sensitivity adjustment system 10 is implement | achieved by the central processing unit (CPU) mounted in the mobile telephone 40, and the 1 or several program which prescribes | regulates the operation | movement procedure of this CPU. Is done. The same applies when the microphone sensitivity adjustment system of the present invention is realized by a device or a computer other than the mobile phone 40, and the functions corresponding to the means 11 to 18 are the central processing units mounted on the various devices and computers. It is realized by a device (CPU) and one or a plurality of programs that define the operation procedure of the CPU. Some functions may be realized by a hardware circuit.

  The spectral ratio average value storage means 21 and the impulse response storage means 22 are realized by various memories mounted on the mobile phone 40. The same applies when the microphone sensitivity adjustment system of the present invention is realized by a device or computer other than the mobile phone 40. As storage means corresponding to the storage means 21, 22, for example, a hard disk, ROM, EEPROM, flash A memory, RAM, MO, CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, FD, magnetic tape, or a combination thereof can be adopted.

  In FIG. 2, a cellular phone 40 is a foldable type comprising a main body part 45 provided with an operation unit 44 constituted by various keys and a lid part 47 provided with a screen display unit 46 such as a liquid crystal display. It is a mobile phone. A concave portion 49 is formed on the surface 48 of the main body portion 45, and the reference microphone 41 and the correction target microphone 42 are arranged so that sound can be collected in a state facing the space of the concave portion 49, that is, in the space of the concave portion 49. .

  When the cellular phone 40 is folded, it is formed on the surface 43 of the main body portion 45 provided with the speaker 43 disposed on the surface 50 of the lid portion 47 provided with the screen display portion 46 and the operation portion 44. The recess 49 is configured to face each other. In the illustrated example, the opening shape of the concave portion 49 is substantially elliptical, but is not limited to this, and the calibration signal output from the speaker 43 is equal to the reference microphone 41 and the correction target microphone 42. It only has to be able to communicate with. In the state where the cellular phone 40 is folded, the reference microphone 41 and the correction target microphone 42 are arranged at symmetrical positions when viewed from the speaker 43. Therefore, the opening shape of the concave portion 49 is the reference microphone 41 side and the correction target microphone 42 side. It has a symmetrical shape. In the present embodiment, the reference microphone 41 and the correction target microphone 42 are omnidirectional microphones, but the present invention may be applied to a directional microphone.

  In this embodiment, sensitivity adjustment between a plurality of microphones is performed by the microphone sensitivity adjustment system 10 as follows.

  First, as shown in FIG. 3, a microphone calibration process (calibration) is performed in advance before performing voice input using a mobile phone 40 (including a videophone) and other uses (games, etc.). For this purpose, the mobile phone 40 is turned on, a program including a calibration processing execution function is started, and calibration processing is started (step S1). The calibration process described below is also performed at the factory when the mobile phone 40 is manufactured, and the spectrum ratio average value storage means 21 and the impulse response storage means 22 in the mobile phone 40 are already stored at the factory shipment stage. The spectral ratio average value and impulse response obtained by the calibration process at the factory are stored and preset. As described above, only the factory preset may be used. However, in the present embodiment, the spectrum ratio average value and the impulse response are updated by performing the calibration process even after the mobile phone 40 reaches the user's hand. Alternatively, the calibration process may be performed for the first time after the mobile phone 40 reaches the user's hand without performing factory presetting.

  The calibration signal output means 11 is, for example, at a predetermined timing such as immediately after the user performs a folding operation of the mobile phone 40 or when a predetermined time (date and time) comes when the mobile phone 40 is folded. Then, a calibration signal is output from the speaker 43 (step S2).

The calibration signal output from the speaker 43 is simultaneously input at an equal level to the reference microphone 41 and the correction target microphone 42 via the recess 49 formed in the surface 48 of the main body portion 45 of the mobile phone 40. the output signal X _C2 of the output signals X _C1 and corrected microphone 42 of the reference microphone 41 when, taking the microphone sensitivity adjustment system 10 is sampled (step S3). At this time, the frame length is, for example, 2048 samples (64 ms).

Subsequently, the microphone sensitivity adjustment system 10, the window processing unit 12, the output signal X _C2 of the output signals X _C1 and corrected microphone 42 of the reference microphone 41 acquired, performs windowing, respectively (step S4).

  Then, the Fourier transform means 13 performs fast Fourier transform (FFT) on the output signals of the reference microphone 41 and the correction target microphone 42 after the window processing (step S5).

Thereafter, the spectrum S ₁ (S ₁ (1), S ₁ (2) of the output signal of the reference microphone 41 obtained by the Fourier transform means 13 in each frequency band in the range of 100 Hz to 1 kHz by the spectrum ratio calculation means 14. , S ₁ (3) ...; however, the numbers in parentheses are numbers assigned to the respective frequency bands) and the spectrum S ₂ (S ₂ (1), S ₂ (2), S ₂ ( 3) ..., where the ratios in parentheses are the numbers S ₁ / S ₂ (S ₁ (1) / S ₂ (1), S ₁ (2) / S ₂ (2), S ₁ (3) / S ₂ (3)...) Is obtained for each frequency band (step S6).

At this time, the spectrum ratio calculation means 14 determines whether or not the spectrum ratio S ₁ / S _{2 of} each frequency band is an abnormal value by using the spectrum ratio S ₁ / S ₂ of the previous and subsequent frequency bands. value a was the case, the spectral ratio S ₁ / S ₂ indicating an abnormal value, the spectral ratio S ₁ / S ₂ normal spectral ratio S calculated by performing interpolation processing using the before and after the frequency band modifying replaced with ₁ / S ₂ (step S7).

For example, when determining whether or not the spectral ratio S ₁ (j) / S ₂ (j) of the frequency band f _j is an abnormal value, first, the frequency band f _j before that (low band side). ₋₁ spectral ratio S ₁ (j−1) / S ₂ (j−1), and then (high frequency side) frequency band f _{j + 1} spectral ratio S ₁ (j + 1) / S ₂ (j + 1) It is determined whether the absolute value of the difference is equal to or less than a certain value. That is, it is determined whether or not | {S ₁ (j−1) / S ₂ (j−1)} − {S ₁ (j + 1) / S ₂ (j + 1)} | ≦ α (constant value). When was below a predetermined value α is the spectral ratio of the determination target _{S 1 (j) / S 2} (j), the spectral ratio S ₁ before and after the frequency band _{f j-1, f j +} 1 It is determined whether or not the absolute value of the difference from either (j−1) / S ₂ (j−1) or S ₁ (j + 1) / S ₂ (j + 1) is a certain value or more. That is, | {S ₁ (j) / S ₂ (j)} − {S ₁ (j−1) / S ₂ (j−1)} | ≧ β (constant value), or | {S ₁ (j) It is determined whether or not / S ₂ (j)} − {S ₁ (j + 1) / S ₂ (j + 1)} | ≧ β (constant value). Here, in the case were more than a predetermined value β, it is determined that the spectral ratio of the frequency band _{f j S 1 (j) /} S 2 (j) is an abnormal value, before and after the frequency band f _j-1, First, linear (linear) interpolation processing is performed using spectral ratios S ₁ (j−1) / S ₂ (j−1) and S ₁ (j + 1) / S ₂ (j + 1) of f _{j + 1} , and after correction The spectral ratio S ₁ (j) / S ₂ (j) of the frequency band f _j is calculated and replaced with an abnormal value. That is, S ₁ (j) / S ₂ (j) after correction = [{S ₁ (j−1) / S ₂ (j−1)} + {S ₁ (j + 1) / S ₂ (j + 1)}] / 2. In addition, the spectral ratios S ₁ (j−2) / S ₂ (j−2) of the total of four frequency bands f _j−2 , f _j−1 , f _{j + 1} , and f _{j + 2} in two at the front and rear, Using S ₁ (j−1) / S ₂ (j−1), S ₁ (j + 1) / S ₂ (j + 1), S ₁ (j + 2) / S ₂ (j + 2), quadratic interpolation processing (least square) Secondary interpolation processing or the like) or four-point linear interpolation processing, or three-point secondary interpolation processing or three-point linear interpolation processing may be performed using the spectral ratio of the total three frequency bands before and after, Various other interpolation processes can be employed. Further, the number of spectrum ratios of the frequency bands before and after used for determining whether or not the value is an abnormal value may be different from the number of spectrum ratios of the frequency bands before and after used for interpolation correction.

Then, repeated acquisition of the output signal X _C2 of the output signals X _C1 and corrected microphone 42 of the reference microphone 41 interpolation correction processing outliers from (Step S3) each processing a predetermined number of times until (step S7) (N times) Whether or not it has been reached is determined (step S8), and if it has not reached N times, the processing returns to step S3 again, and thereafter, the processing of steps S3 to S8 is repeated until it reaches N times. On the other hand, when N times have been reached, the spectrum ratio calculation means 14 performs fast Fourier transform processing a plurality of times (N times) for each of the frequency bands f ₁ , f ₂ , f ₃ , f ₄ . The average value of the obtained spectrum ratios S ₁ / S ₂ (the average value of N S ₁ (j) / S ₂ (j); where j = 1, 2, 3, 4,...) Is taken (step S9). That is, by summing the _first spectral ratio S ₁ (1) / S ₂ (1) to the Nth spectral ratio S ₁ (1) / S ₂ (1) in the frequency band f ₁ and dividing by N. , averaged spectral ratio S ₁ (1) in the frequency band f ₁ / S seek ₂ (1), in a similar manner, spectral ratio S ₁ (2) of the first in the frequency band f ₂ / S ₂ (2 ) from N th spectral ratio _{S 1 (2) / S 2} (2) divided by the total of N up, averaged spectral ratio S ₁ (2 in the frequency band f ₂₎ / S _{2 (2} ) And such an averaging process is also performed in the other frequency bands f ₃ , f ₄ .

Subsequently, the spectral ratio S ₁ / S ₂ (S ₁ (1) / S ₂ (1), S ₁ (2) for each frequency band obtained by the spectral ratio calculating means 14 by the spectral ratio average value calculating means 15. / S ₂ (2), S ₁ (3) / S ₂ (3), S ₁ (4) / S ₂ (4)...)) _Is calculated (step S10). That is, the spectral ratio of the frequency band _{f 1 S 1 (1) /} S 2 (1) ( in the present embodiment, the average value of N times), and the spectral ratio S ₁ of the frequency band _{f 2 (2) / S 2} ( 2) (in this embodiment, N average values) and other spectral ratios S ₁ (3) / S ₂ (3), S ₁ (4) / S _{2 of} other frequency bands f ₃ , f ₄ . (4)... (In the present embodiment, N average values in each case) are totaled and divided by the number of bands to obtain the average value R _AVE . Further, the spectrum ratio average value calculation means 15 stores the obtained average value R _AVE in the spectrum ratio average value storage means 21 and stores it (step S10).

After that, the spectrum ratio adjusting means 16 calculates the difference between the average value R _AVE obtained by the spectrum ratio average value calculating means 15 and 1 as the spectrum ratio S ₁ / S ₂ (S ₁ (1) / S ₂ ) for each frequency band. (1), S ₁ (2) / S ₂ (2), S ₁ (3) / S ₂ (3)...)), Respectively, so that the spectral ratio S ₁ / S ₂ (S ₁ (1) / S ₂ (1), S ₁ (2) / S ₂ (2), S ₁ (3) / S ₂ (3)...)) Are adjusted so that the average value is 1 (step S11). . For example, if R _AVE = 1.1, then 0.1, which is the difference between 1.1 and 1, is the spectral ratio S ₁ (1) / S of each frequency band f ₁ , f ₂ , f ₃ . ₂ (1), S ₁ (2) / S ₂ (2), S ₁ (3) / S ₂ (3)... (In this embodiment, N average values, respectively).

Then, the spectral ratio S ₁ / S ₂ (adjusted S ₁ (1) / S ₂ ) for each frequency band adjusted by the inverse Fourier transform means 17 so that the average value becomes 1 by the spectral ratio adjustment means 16. (1), S ₁ (2) / S ₂ (2), S ₁ (3) / S ₂ (3)...)) Is subjected to inverse Fourier transform (IFFT) to obtain an impulse response (step S12). Further, the inverse Fourier transform means 17 stores the obtained impulse response in the impulse response storage means 22 and saves it (step S12).

  Thus, the microphone calibration process (calibration) is completed (step S13). The calibration process described above is repeated by causing the calibration signal output means 11 to output a calibration signal from the speaker 43 at a predetermined timing.

  Next, as shown in FIG. 4, when making a call using the mobile phone 20, or when using the mobile phone 20 for other purposes such as a game and performing voice input, etc. The processing of the acoustic signal is started (step S21).

When the audio signal to be processed, such as source separation and speech recognition in production state is input to the reference microphone 41 and corrected microphone 42, the output signal X ₂ to be corrected microphone 42 at this time, sampled by the microphone sensitivity It is taken into the adjustment system 10 (step S22). On the other hand, the output signal X ₁ of the reference microphone 41 is not a correction target, is sent to the sound source separation system 30 of the rear stroke as it is sampled (see Figure 1).

Subsequently, the output correction means 18 performs a convolution operation based on the following equation (1) using the impulse response H stored in the impulse response storage means 22 and the output signal X ₂ of the correction target microphone 42. Thus, the corrected signal Y ₂ of the correction target microphone 42 is obtained (step S23).

Y ₂ (n) = Σ _k {X ₂ (k) × H (n−k)} (1)

Here, X ₂ (k) is the kth sampling data on the time axis for the output signal X ₂ of the correction target microphone 42, and Y ₂ (n) is the output signal X ₂ of the correction target microphone 42. This is the n-th data on the time axis for the signal Y ₂ obtained by correction by the convolution operation, H (n−k) is the impulse response of the system, and Σ _k is k = −∞ to k = The sum of ∞.

Further, the output correction means 18 performs an average gain correction by multiplying the signal Y ₂ obtained by performing the convolution operation by the average value R _AVE stored in the spectrum ratio average value storage means 21, and a signal multiplied by R _AVE Y ₂ is output as a corrected signal for the output signal X ₂ of the correction target microphone 42 (step S24).

Thereafter, the output signal X ₁ of the reference microphone 41 whose gain is not changed and the signal Y ₂ × R _AVE obtained by correcting the output signal X ₂ of the correction target microphone 42 by the output correcting means 18 are separated into sound sources. The target sound obtained by taking in the system 30 and performing sound source separation processing with directivity control (see FIG. 9) using these signals to separate the target sound and the disturbing sound, and further separating them as necessary. Various processes such as a voice recognition process are performed for (step S25), and the processing of the acoustic signal in the working state is terminated (step S26). The sound source separation system 30 is arbitrary as long as it is a system that performs directivity control using output signals of a plurality of microphones, and may be an existing system, or the Japanese Patent Application Nos. 2004-366202 and 2005 by the applicant of the present application. -270931 proposed sound source separation system, that is, directivity control is performed by performing linear combination processing for target sound enhancement and linear combination processing for target sound suppression using output signals of a small number of microphones. A system that separates the target sound and the disturbing sound by performing band selection or spectral subtraction using the spectrum of a plurality of signals obtained by directivity control in the above may be used. Similarly, a system for performing other post-processing such as voice recognition processing is also arbitrary.

According to this embodiment, there are the following effects. That is, since the microphone sensitivity adjustment system 10 includes the Fourier transform unit 13, the spectrum ratio calculation unit 14, and the Fourier inverse transform unit 17, a calibration signal for sensitivity adjustment is input to the reference microphone 41 and the correction target microphone 42. obtains the spectral ratio S ₁ / S ₂ of the output signal X _C2 of the output signal X _C1 and corrected microphone 42 of the reference microphone 41 at this time for each frequency band, further spectral ratio for each frequency band S ₁ / it can be determined in the impulse response on the basis of S _2. For this reason, in performing directivity control for realizing various processes such as sound source separation and voice recognition, a microphone calibration process can be performed in advance.

Since the microphone sensitivity adjustment system 10 includes the output correction means 18, it is obtained by a calibration process when forming a desired directivity characteristic in realizing various processes such as sound source separation and voice recognition in an actual operation state. by performing convolution operation using a previously impulse response and corrects the output signal X ₂ to be corrected microphone 42, it can be adjusted to the same level of sensitivity as the reference microphone 41. For this reason, since the reference microphone 41 and the correction target microphone 42 can be aligned with the same level of sensitivity, desired directivity characteristics can be formed, and performance of various processes such as sound source separation and voice recognition can be improved. Can do. In particular, when the directivity is formed by taking the difference between the output signals of a plurality of microphones (including the case where the difference is taken after delay processing) (for example, in the case of (1) to (3) in FIG. 9). Since the desired suppression effect can be obtained in the direction in which the incoming sound is to be suppressed, the performance improvement of various processes such as sound source separation and speech recognition can be made more remarkable.

Further, when the spectrum ratio calculation means 14 obtains the spectrum ratio S ₁ / S ₂ for each frequency band, it is obtained as an average value of a plurality of times (N times), so that stable calibration information can be obtained. Therefore, more appropriate correction processing can be performed on the output signal X ₂ of the correction target microphone 42.

Furthermore, the spectrum ratio calculating means 14, when obtaining a spectral ratio S ₁ / S ₂ for each frequency band, the spectral ratio S ₁ indicating an abnormal value by using the spectral ratio S ₁ / S ₂ before and after frequency bands Since / S ₂ is interpolated and corrected, stable calibration information can be obtained at this point as well, and more appropriate correction processing can be performed on the output signal X ₂ of the correction target microphone 42.

Then, the spectrum ratio calculating means 14, when obtaining a spectral ratio S ₁ / S ₂ for each frequency band, since the target spectrum of a frequency band in the range of 100Hz～1kHz, proper and a small amount of calculation Effective calibration information can be obtained.

Further, since the microphone sensitivity adjustment system 10 includes the spectrum ratio average value calculation means 15 and the spectrum ratio adjustment means 16, the average value R _AVE of the spectrum ratio S ₁ / S ₂ for each frequency band is obtained, and each frequency is calculated. After adjusting the average value of the spectrum ratios S ₁ / S ₂ for each band to be 1, the impulse response can be obtained. For this reason, in the correction process by the output correction means 18, the variation of the spectrum ratio S ₁ / S ₂ between the frequency bands and the average value R _AVE can be handled separately, and the impulse response is used. In the convolution calculation, correction considering the variation can be performed, and then the average gain correction can be performed using the average value R _AVE .

  Note that the present invention is not limited to the above-described embodiment, and modifications and the like within a scope where the object of the present invention can be achieved are included in the present invention.

  In other words, in the above-described embodiment, the microphone sensitivity adjustment system 10 is configured to perform sensitivity adjustment between two microphones in total, that is, the reference microphone 41 and the correction target microphone 42. However, in the present invention, three or more microphones are used. In this case, one microphone is selected as a reference microphone, and the remaining two or more microphones are used as correction target microphones, respectively. The sensitivity adjustment according to the present invention may be performed between each time.

  In the above embodiment, only the output signal of the correction target microphone 42 is corrected without changing the gain of the reference microphone 41. However, the present invention corrects the output signal for all of the plurality of microphones whose sensitivity should be uniformed. In short, any configuration may be used as long as the output signal of at least one of the plurality of microphones is corrected.

In the embodiment, the spectrum ratio calculation means 14 is configured to obtain the spectrum ratio S ₁ / S ₂ for each frequency band as an average value of a plurality of times (predetermined N times) (FIG. 3). In step S9, the spectral ratio S ₁ / S ₂ obtained by one Fourier transform process may be used as it is without performing such an averaging process. However, it is preferable to perform an averaging process from the viewpoint of obtaining stable calibration information and performing a more appropriate correction process on the output signal X ₂ of the correction target microphone 42.

Further, in the above-described embodiment, after performing a predetermined number of times (N times) of the loop processing (steps S3 to S8 in FIG. 3), the average value of the spectrum ratios S ₁ / S ₂ obtained N times is taken. Therefore, in order to update the impulse response by performing the next calibration process (see step S9 in FIG. 3), the loop process is repeated N times (steps S3 to S8 in FIG. 3) and then N times. The average value of the spectrum ratio S ₁ / S ₂ is taken, but when the configuration is such that the spectrum ratio S ₁ / S ₂ is obtained by continuously performing Fourier transform processing, the loop of the above embodiment is used. In the process (steps S3 to S8 in FIG. 3) (a position immediately after step S7 is preferable), an averaging process corresponding to step S9 may be included. In this case, a moving average value (for example, the latest N moving average values) of spectrum ratios S ₁ / S ₂ continuously calculated one after another may be calculated, or a cumulative average value (initial spectral ratio) (Average value from S ₁ / S ₂ to the latest spectrum ratio S ₁ / S ₂ ).

In the above-described embodiment, the spectrum ratio calculation unit 14 is configured to perform the interpolation correction of the abnormal value using the spectral ratio S ₁ / S ₂ of the preceding and subsequent frequency bands. The correction process may be omitted. However, it is preferable to perform an abnormal value interpolation correction process in that a more appropriate correction process can be performed on the output signal X ₂ of the correction target microphone 42. Further, when an abnormal value is found, the corrected value is not simply calculated by performing an interpolation process using the spectral ratio S ₁ / S ₂ of the preceding and subsequent frequency bands, but the abnormal value is simply calculated for each frequency band. it may be replaced with the average value of the spectral ratio S ₁ / S _2. However, it is preferable to perform the interpolation process using the spectral ratio S ₁ / S ₂ of the front and rear frequency bands from the viewpoint of reflecting the difference in frequency level.

In the above embodiment, the spectral ratio calculating means 14, as had been configured to determine the spectral ratio S ₁ / S ₂ for each frequency band within the 100Hz～1kHz, to be limited to this range is not. However, it is preferable to set the frequency within the range of 100 Hz to 1 kHz in that appropriate and effective calibration information can be obtained with a small amount of calculation.

Furthermore, in the embodiment, the microphone sensitivity adjustment system 10 obtains the average value R _AVE of the spectrum ratios S ₁ / S ₂ for each frequency band by the spectrum ratio average value calculation means 15 in the calibration process, and the spectrum ratio adjustment means. 16 is adjusted so that the average value of the spectral ratios S ₁ / S ₂ for each frequency band becomes 1, and then the impulse response is obtained by the inverse Fourier transform means 17. In the correction process by the output correction means 18, after the convolution calculation The average value R _AVE is multiplied, but the variation of the spectrum ratio S ₁ / S ₂ between the frequency bands and the average value R _AVE are separated and handled in this way. rather, as shown in FIG. 5, in the calibration process, processing is performed in which the average value of the spectral ratio S ₁ / S ₂ for each frequency band is adjusted to be 1 And performing inverse Fourier transform seek Inpasuru response as calibration information without, in the correction process by the output correction unit 18 may be configured not to perform the average gain correction multiplying the average value R _AVE.

  In FIG. 5, a microphone sensitivity adjustment system 60 includes calibration signal output means 11, window processing means 12, Fourier transform means 13, and spectrum ratio calculation means having the same configuration and function as those of the microphone sensitivity adjustment system 10 of the above embodiment. 14, but no means equivalent to the spectrum ratio average value calculation means 15, the spectrum ratio adjustment means 16, and the spectrum ratio average value storage means 21 are provided.

The microphone sensitivity adjustment system 60 includes a Fourier inverse transform unit 61, an output correction unit 62, and an impulse response storage unit 63. The Fourier inverse transform unit 61 includes the Fourier inverse transform unit of the above embodiment. Instead of performing the inverse Fourier transform using the spectrum ratio S ₁ / S ₂ for each frequency band after being adjusted so that the average value becomes 1 as shown in FIG. The inverse Fourier transform is performed by using the spectrum ratio S ₁ / S ₂ for each frequency band in a state where it is not performed, and the impulse response obtained as a result is stored in the impulse response storage means 63 and stored. . Accordingly, the impulse response storage unit 63 stores an impulse response obtained through a process different from that of the impulse response storage unit 22 of the embodiment. The output correction means 62 performs the convolution calculation using the impulse response stored in the impulse response storage means 63, but multiplies the average value R _AVE after the convolution calculation as in the output correction means 18 of the embodiment. The processing is not performed.

Therefore, in the calibration process (calibration) by the microphone sensitivity adjustment system 60 of FIG. 5, the average value R _AVE of step S10 is obtained in the calibration process flow by the microphone sensitivity adjustment system 10 of the embodiment shown in FIG. The process and each process corresponding to the adjustment process for setting the average value in step S11 to 1 are not performed. Further, in the processing of the acoustic signal (speech voice, noise, etc.) in the actual working state (the state where the mobile phone is being used) by the microphone sensitivity adjustment system 60 of FIG. 5, the microphone sensitivity adjustment system of the embodiment shown in FIG. In the flow of the processing of the acoustic signal in the actual working state by 10, the average gain correction by multiplying by the average value R _AVE in step S24 is not performed.

  In the embodiment, in the state where the cellular phone 40 is folded, the surface of the main body portion 45 provided with the operation portion 44 is changed from the speaker 43 disposed on the surface 50 of the lid portion 47 provided with the screen display portion 46. A calibration signal is output toward the space of the concave portion 49 formed in 48, and this calibration signal is input by the reference microphone 41 and the correction target microphone 42 arranged so as to face the space of the concave portion 49. However, as shown in FIG. 6, in the foldable mobile phone 70, a sound path 75 is formed which communicates the concave portion 74 formed on the surface 73 of the main body portion 72 provided with the operation portion 71 and the external space. In addition, when the cellular phone 70 is folded, the surface of the cellular phone 70 on the screen display unit side (the surface of the lid portion) is in a state where the space of the recess 74 is closed. A calibration signal for sensitivity adjustment (environmental sound such as human voice) is guided from the external space into the space of the recess 74 via the sound path 75 and input to the reference microphone 76 and the correction target microphone 77. Also good. In this case, the sound path 75 is arranged so that the reference microphone 76 and the correction target microphone 77 are in a symmetrical position when viewed from the sound path 75. That is, the distance from the outlet on the space side of the concave portion 74 of the sound path 75 to the reference microphone 76 is made equal to the distance to the correction target microphone 77. In this way, the calibration signal guided from the external space into the space of the recess 74 via the sound path 75 can be simultaneously input to the reference microphone 76 and the correction target microphone 77 as signals of the same level. it can.

  Further, not only the sound path 75 formed as a through-hole penetrating the main body portion 72 as shown in FIG. 6, but a portable device 80 such as a mobile phone or a personal digital assistant (PDA) as shown in FIG. 8 may be a sound path 86 formed in a groove shape so that the reference microphone 84 and the correction target microphone 85 are positioned symmetrically on the surface 83 provided with the operation portion 82 of the main body portion 81 of FIG. As shown, the lid portion 94 (whether or not the lid portion is provided with the screen display portion so as to correspond to the concave portion 93 formed in the surface 92 provided with the operation portion of the main body portion 91 of the portable device 90. Or the sound path 95 formed through the cover, in other words, when the operation part of the main body part of the portable device is covered with the lid part, the concave part is closed by the lid part. Space and the outside sky Steal may be a sound path to be able to communicate with each other.

  As described above, the microphone sensitivity adjustment method and system of the present invention are used, for example, when adjusting the sensitivity between a plurality of microphones mounted on a mobile device such as a mobile phone or an in-vehicle device such as a car navigation system. Suitable for use.

1 is an overall configuration diagram of a microphone sensitivity adjustment system according to an embodiment of the present invention. The perspective view of the mobile telephone carrying the microphone sensitivity adjustment system of the said embodiment. The figure of the flowchart which shows the flow of the calibration process (calibration) by the microphone sensitivity adjustment system of the said embodiment. The figure of the flowchart which shows the flow of a process of the acoustic signal (speech voice, noise, etc.) in the working state (state in use of a mobile telephone) by the microphone sensitivity adjustment system of the said embodiment. The whole microphone sensitivity adjustment system block diagram which shows the 1st modification of this invention. The perspective view of a part of a mobile phone showing a second modification of the present invention. The perspective view of a part of portable apparatus which shows the 3rd modification of this invention. Sectional drawing of a part of portable apparatus which shows the 4th modification of this invention. The figure which shows the example of the directional characteristic control using two omnidirectional microphones.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,60 Microphone sensitivity adjustment system 10A Calibration information preparation means 13 Fourier transform means 14 Spectral ratio calculation means 15 Spectral ratio average value calculation means 16 Spectral ratio adjustment means 17, 61 Inverse Fourier transform means 18, 62 Output correction means 20 For calibration Information storage means 21 Spectral ratio average value storage means 22, 63 Impulse response storage means 40, 70 Mobile phone 41, 76, 84 Reference microphone 42, 77, 85 Correction target microphone 43 Speaker 44, 71, 82 Operation unit 45, 72, 81, 91 Main body portion 46 Screen display portion 47, 84, 94 Lid portion 49, 74, 83, 93 Recess 75, 86, 95 Sound path 80, 90 Portable device

Claims (24)

A microphone sensitivity adjustment method for adjusting sensitivity between a plurality of microphones used in combination to form a desired directional characteristic,
The same calibration signal for sensitivity adjustment is input to the plurality of microphones, and calibration information for adjusting the plurality of microphones to the same level of sensitivity is generated using the output signals of the plurality of microphones at this time. And
A method for adjusting a microphone sensitivity, comprising: correcting the output signal of at least one of the plurality of microphones using the calibration information when forming the target directivity. When obtaining the calibration information,
One of the plurality of microphones is selected as a reference microphone that serves as a reference for sensitivity adjustment, and the output signal is corrected so that microphones other than the reference microphone have the same level of sensitivity as the reference microphone. As a microphone to be corrected,
The same calibration signal for sensitivity adjustment is input to the reference microphone and the correction target microphone, and each output signal of the reference microphone and the correction target microphone at this time is subjected to Fourier transform,
Obtain the ratio of the spectrum of the output signal of the reference microphone obtained by Fourier transform and the spectrum of the output signal of the correction target microphone for each frequency band,
The impulse response as the calibration information is obtained by inverse Fourier transform of the obtained spectrum ratio for each frequency band,
When forming the target directivity, a signal after correction of the correction target microphone is obtained by performing a convolution operation using the impulse response as the calibration information and the output signal of the correction target microphone. The microphone sensitivity adjustment method according to claim 1, wherein: When obtaining the impulse response,
The average value of the spectrum ratio for each frequency band is obtained by obtaining the average value of the spectrum ratio for each frequency band as the calibration information and subtracting the difference between the obtained average value and 1 from the spectrum ratio for each frequency band. Is adjusted to be 1, then Fourier inverse transform is performed on the spectral ratio for each frequency band after adjustment,
When forming the desired directivity,
After performing a convolution operation using the impulse response and the output signal of the correction target microphone, the signal obtained by performing the convolution operation is multiplied by an average value of spectral ratios for each frequency band as the calibration information. The microphone sensitivity adjustment method according to claim 2, further comprising: obtaining a corrected signal of the microphone to be corrected.   The microphone sensitivity adjustment method according to claim 2 or 3, wherein when obtaining the spectrum ratio for each frequency band, an average value of a plurality of times is used.   The microphone according to any one of claims 2 to 4, wherein, when obtaining the spectrum ratio for each frequency band, the spectrum ratio indicating an abnormal value is interpolated and corrected using the spectrum ratio of the preceding and following frequency bands. Sensitivity adjustment method.   The microphone sensitivity adjustment method according to any one of claims 2 to 5, wherein when obtaining a spectrum ratio for each frequency band, a spectrum in a frequency band within a range of 100 Hz to 1 kHz is targeted. When adjusting the sensitivity between the plurality of microphones mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion,
A concave portion is formed on a surface of the main body portion on which the operation portion is provided, and the plurality of microphones are provided on the main body portion in a state of facing the space of the concave portion, and the operation portion of the main body portion is provided on the lid portion. A speaker is provided on the lid portion in a state of being disposed at a position facing the concave portion when covered,
When obtaining the calibration information,
A calibration signal for sensitivity adjustment is output from the speaker in a state where the operation unit of the main body part is covered with the lid part, and the output calibration signal is input to the plurality of microphones via the space of the recess. The microphone sensitivity adjustment method according to any one of claims 1 to 6. The mobile device is a foldable mobile phone configured to include the main body portion provided with the operation portion and the lid portion provided with a screen display portion. When the mobile phone is folded, The speaker is provided on the lid portion in a state of being disposed at a position facing the recess,
When obtaining the calibration information,
8. The calibration signal for sensitivity adjustment is output from the speaker in a state in which the cellular phone is folded, and the output calibration signal is input to the plurality of microphones via the space of the recess. A method for adjusting the microphone sensitivity described in 1. The portable device is a foldable portable information terminal configured to include the main body portion and the lid portion, and the lid is disposed in a position facing the recess when the portable information terminal is folded. The speaker is provided in the part,
When obtaining the calibration information,
5. The calibration signal for sensitivity adjustment is output from the speaker in a state where the portable information terminal is folded, and the output calibration signal is input to the plurality of microphones via the space of the recess. 8. The microphone sensitivity adjustment method according to 7. When adjusting the sensitivity between the plurality of microphones mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion,
A concave portion is formed on a surface of the main body portion on which the operation portion is provided, and the plurality of microphones are provided on the main body portion in a state of facing the space of the concave portion, and the operation portion of the main body portion is provided on the lid portion. Forming a sound path in the main body portion or the lid portion that communicates the space of the concave portion and the external space that is closed by the lid portion when covered;
When obtaining the calibration information,
In a state where the operation part of the main body part is covered with the lid part, a calibration signal for sensitivity adjustment is guided from the external space into the space of the recess through the sound path and input to the plurality of microphones. The microphone sensitivity adjustment method according to any one of claims 1 to 6. The mobile device is a foldable mobile phone configured to include the main body portion provided with the operation portion and the lid portion provided with a screen display portion. When the mobile phone is folded, The sound path is formed in the main body portion in a state in which the space of the recess that is closed by the lid portion communicates with the external space,
When obtaining the calibration information,
In a state where the cellular phone is folded to close the space of the recess, a calibration signal for sensitivity adjustment is guided from the external space into the space of the recess through the sound path and input to the plurality of microphones. The microphone sensitivity adjustment method according to claim 10. The portable device is a foldable portable information terminal configured to include the main body portion and the lid portion, and the concave portion that is closed by the lid portion when the portable information terminal is folded. The sound path is formed in the main body part or the lid part in a state where the space and the external space communicate with each other,
When obtaining the calibration information,
In a state where the portable information terminal is folded to close the space of the recess, a calibration signal for sensitivity adjustment is guided from the external space into the space of the recess through the sound path and input to the plurality of microphones. The microphone sensitivity adjustment method according to claim 10. A microphone sensitivity adjustment system that adjusts sensitivity between a plurality of microphones used in combination to form a desired directional characteristic,
Calibration information for creating calibration information for adjusting the plurality of microphones to the same level of sensitivity using the output signals of the plurality of microphones when the same calibration signal for sensitivity adjustment is input to the plurality of microphones. Information creation means;
Calibration information storage means for storing the calibration information created by the calibration information creation means;
A microphone sensitivity adjustment system comprising: output correction means for correcting an output signal of at least one of the plurality of microphones using the calibration information stored in the calibration information storage means. . The calibration information creating means includes
The same calibration signal for sensitivity adjustment is input to one reference microphone serving as a reference for sensitivity adjustment among the plurality of microphones and a correction target microphone whose output signal is corrected to have the same level of sensitivity as the reference microphone. Fourier transform means for Fourier transforming each output signal of the reference microphone and the correction target microphone when
Spectral ratio calculation means for obtaining a ratio of the spectrum of the output signal of the reference microphone obtained by the Fourier transform means and the spectrum of the output signal of the correction target microphone for each frequency band;
Fourier inverse transform means for obtaining an impulse response as the calibration information by performing Fourier inverse transform on the spectrum ratio for each frequency band obtained by the spectrum ratio calculating means,
The calibration information storage means includes impulse response storage means for storing the impulse response obtained by the Fourier inverse transform means,
The output correction means performs a convolution operation using the impulse response as the calibration information stored in the impulse response storage means and the output signal of the correction target microphone, thereby correcting the correction target microphone. A microphone sensitivity adjustment system characterized by being configured to obtain a signal. The calibration information creating means includes
Spectral ratio average value calculating means for obtaining an average value of these spectral ratios as the calibration information for the spectral ratio for each frequency band obtained by the spectral ratio calculating means;
A spectrum that is adjusted so that the average value of the spectrum ratio for each frequency band becomes 1 by subtracting the difference between the average value obtained by the spectrum ratio average value calculating means and 1 from the spectrum ratio for each frequency band. And a ratio adjusting means,
The calibration information storage means includes a spectrum ratio average value storage means for storing the average value as the calibration information obtained by the spectrum ratio average value calculation means,
The Fourier inverse transform means is configured to perform Fourier inverse transform on the spectrum ratio for each frequency band after adjustment by the spectrum ratio adjustment means,
The output correction unit is obtained by performing a convolution operation after performing a convolution operation using the impulse response as the calibration information stored in the impulse response storage unit and the output signal of the correction target microphone. The microphone sensitivity adjustment system according to claim 14, wherein the signal is multiplied by the average value as the calibration information stored in the spectrum ratio average value storage means.   The microphone sensitivity adjustment system according to claim 14 or 15, wherein the spectrum ratio calculation means is configured to take an average value a plurality of times when obtaining a spectrum ratio for each frequency band.   The spectrum ratio calculating means is configured to interpolate and correct a spectrum ratio indicating an abnormal value using the spectrum ratio of the previous and subsequent frequency bands when obtaining the spectrum ratio for each frequency band. Item 17. The microphone sensitivity adjustment system according to any one of Items 14 to 16.   The spectrum ratio calculating means is configured to target a spectrum in a frequency band within a range of 100 Hz to 1 kHz when obtaining a spectrum ratio for each frequency band. The microphone sensitivity adjustment system according to any one of the above. The plurality of microphones are mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion,
A concave portion is formed on the surface of the main body portion on which the operation portion is provided, and the plurality of microphones are provided on the main body portion in a state facing the space of the concave portion,
When the operation portion of the main body portion is covered with the lid portion, a speaker is provided on the lid portion in a state of being disposed at a position facing the concave portion,
A calibration signal for sensitivity adjustment is output from the speaker in a state where the operation portion of the main body portion is covered with the lid portion, and the output calibration signal is input to the plurality of microphones via the spaces of the recesses. The microphone sensitivity adjustment system according to claim 13, wherein the microphone sensitivity adjustment system is configured as described above. The portable device is a foldable mobile phone configured to include the main body portion provided with the operation unit and the lid portion provided with a screen display unit,
When the mobile phone is folded, the speaker is provided on the lid portion in a state of being disposed at a position facing the concave portion,
A calibration signal for sensitivity adjustment is output from the speaker in a state in which the cellular phone is folded, and the output calibration signal is input to the plurality of microphones via the space of the recess. The microphone sensitivity adjustment system according to claim 19. The portable device is a foldable portable information terminal configured to include the main body portion and the lid portion,
When the portable information terminal is folded, the speaker is provided on the lid portion in a state where the portable information terminal is disposed at a position facing the concave portion,
With the portable information terminal folded, a calibration signal for sensitivity adjustment is output from the speaker, and the output calibration signal is input to the plurality of microphones via the space of the recess. The microphone sensitivity adjustment system according to claim 19. The plurality of microphones are mounted on a portable device configured to include a main body portion provided with an operation portion and at least one lid portion covering the operation portion,
A concave portion is formed on the surface of the main body portion on which the operation portion is provided, and the plurality of microphones are provided on the main body portion in a state facing the space of the concave portion,
When the operating portion of the main body portion is covered with the lid portion, a sound path that communicates the space of the recess and the external space that is closed by the lid portion is formed in the main body portion or the lid portion. Formed,
A calibration signal for sensitivity adjustment is guided from the external space into the concave space through the sound path and input to the plurality of microphones with the operation portion of the main body portion covered with the lid portion. The microphone sensitivity adjustment system according to claim 13, wherein the microphone sensitivity adjustment system is configured as described above. The portable device is a foldable mobile phone configured to include the main body portion provided with the operation unit and the lid portion provided with a screen display unit,
The sound path is formed in the main body portion in a state in which the space of the recess that is closed by the lid portion when the mobile phone is folded and the external space are communicated with each other,
A calibration signal for sensitivity adjustment is guided from the external space into the concave space through the sound path and input to the plurality of microphones in a state where the cellular phone is folded to close the concave space. The microphone sensitivity adjustment system according to claim 22, wherein the microphone sensitivity adjustment system is configured as follows. The portable device is a foldable portable information terminal configured to include the main body portion and the lid portion,
The sound path is formed in the main body portion or the lid portion in a state where the space of the recess that is closed by the lid portion when the portable information terminal is folded and the external space are communicated,
In a state where the portable information terminal is folded and the space of the recess is closed, a calibration signal for sensitivity adjustment is guided from the external space into the space of the recess through the sound path and input to the plurality of microphones. The microphone sensitivity adjustment system according to claim 22, wherein the microphone sensitivity adjustment system is configured as described above.

Images