CN113132881B - Method for adaptively controlling sound amplification degree of wearer based on multiple microphones - Google Patents

Method for adaptively controlling sound amplification degree of wearer based on multiple microphones Download PDF

Info

Publication number
CN113132881B
CN113132881B CN202110414004.4A CN202110414004A CN113132881B CN 113132881 B CN113132881 B CN 113132881B CN 202110414004 A CN202110414004 A CN 202110414004A CN 113132881 B CN113132881 B CN 113132881B
Authority
CN
China
Prior art keywords
signal
subband
wearer
sub
energy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110414004.4A
Other languages
Chinese (zh)
Other versions
CN113132881A (en
Inventor
谭波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Wood Core Technology Co ltd
Original Assignee
Shenzhen Wood Core Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Wood Core Technology Co ltd filed Critical Shenzhen Wood Core Technology Co ltd
Priority to CN202110414004.4A priority Critical patent/CN113132881B/en
Publication of CN113132881A publication Critical patent/CN113132881A/en
Application granted granted Critical
Publication of CN113132881B publication Critical patent/CN113132881B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
    • H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/30—Monitoring or testing of hearing aids, e.g. functioning, settings, battery power
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
    • H04R2225/43—Signal processing in hearing aids to enhance the speech intelligibility

Landscapes

  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Neurosurgery (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

The invention provides a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones, for use in a hearing aid comprising a feedforward microphone and a feedback microphone, the feedback microphone being located in the ear canal of the wearer when the hearing aid is worn, the method comprising: capturing, by the feedforward microphone, ambient signals for a target time period; capturing, by the feedback microphone, an ear canal signal for the target time period; determining whether the ambient environment signal comprises a sound signal of the wearer according to the ear canal signal; and adjusting a gain of the ambient environment signal if the ambient environment signal comprises the wearer's voice signal. The invention can detect the sound production of the wearer based on the ear canal signal of the feedback microphone, and adjust the gain of the surrounding environment signal when the wearer produces the sound, thereby preventing the sound of the wearer from being excessively amplified and ensuring the hearing comfort.

Description

Method for adaptively controlling sound amplification degree of wearer based on multiple microphones
Technical Field
The present invention relates to the field of speech processing, and in particular, to a method, system, computer device, and computer-readable storage medium for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones.
Background
With the rapid development of electronic devices, hearing aids have been developed for compensating for the hearing loss of hearing-impaired persons for hearing-impaired persons. Hearing aids are typically fitted in or behind the ear of a user to amplify and provide the amplified sound to the wearer. Common types of hearing aids include Behind The Ear (BTE) hearing aids, In The Ear (ITE) hearing aids, In The Canal (ITC) hearing aids, completely in the canal (CIC) hearing aids, and the like. Hearing aids typically comprise a housing in which: a microphone for collecting sound signals; a processing circuit for amplifying the sound signal; and a speaker (which may be referred to as a receiver in the hearing aid art) for outputting sound.
The present inventors have found that when a wearer wears a conventional hearing aid, the wearer's own voice is likely to be excessively amplified. Especially, when the wearer needs to listen to a sound signal at a long distance, the gain is often adjusted to be relatively large, so that the wearer's own sound is amplified, and the hearing comfort of the wearer is reduced.
Disclosure of Invention
The invention aims to provide a method, a system, computer equipment and a computer readable storage medium for adaptively controlling the sound amplification degree of a wearer based on multiple microphones, which are used for solving the following problems: existing hearing aids are prone to over-amplifying the wearer's own voice.
An aspect of an embodiment of the invention provides a method of adaptively controlling the degree of sound amplification of a wearer based on multiple microphones for use in a hearing aid comprising a feed-forward microphone and a feedback microphone, the feedback microphone being located in the ear canal of the wearer when the hearing aid is worn, the non-target speech signal comprising the wearer's sound signal; the method comprises the following steps: capturing, by the feedforward microphone, ambient signals for a target time period; capturing, by the feedback microphone, an ear canal signal for the target time period; determining whether the ambient environment signal comprises a sound signal of the wearer according to the ear canal signal; and adjusting a gain of the ambient environment signal if the ambient environment signal comprises the wearer's voice signal.
Optionally, the step of determining whether the ambient signal includes the sound signal of the wearer according to the ear canal signal includes: calculating a first signal energy of the ear canal signal; and judging whether the ambient environment signal comprises the sound signal of the wearer or not according to the first signal energy and a preset threshold value.
Optionally, the method further includes a threshold setting step: calculating a second signal energy of the ambient signal; adjusting the preset threshold value according to the energy of the second signal; the energy of the second signal and the preset threshold value are in a positive relationship.
Optionally, if the ambient signal includes the sound signal of the wearer, the step of adjusting the gain of the ambient signal includes: decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Decomposing the ambient signalInto N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iCorresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; and respectively adjusting the gain of the second sub-band signal corresponding to each first sub-band signal according to each first sub-band signal.
Optionally, the step of respectively adjusting the gain of the second subband signal corresponding to each first subband signal according to each first subband signal includes: calculating the ith first subband signal S1iThe signal energy of (a); and according to said ith first subband signal S1iOf the ith second subband signal S, adjusting the signal energy of the ith second subband signal S2iThe gain of (c).
Optionally, if the ambient signal includes the sound signal of the wearer, the step of adjusting the gain of the ambient signal further includes: according to the Mth first subband signal S1MAdjusting the gain of the M +1 th to nth second subband signals; calculating the energy average value of the M first sub-band signals, and adjusting the gains of the M +1 th to Nth second sub-band signals according to the energy average value; or calculating energy weighted values of the M first sub-band signals based on the auditory characteristics of the human ears, and adjusting gains of the M +1 th to Nth second sub-band signals according to the energy weighted values.
Optionally, the step of determining whether the ambient signal includes the sound signal of the wearer according to the ear canal signal includes: decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Decomposing the ambience signal into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iCorresponding to the same signal frequency band, i is more than or equal to 1 and less than or equal to M, and i is a positive integer; and calculating the ith first subband signal S1iSignal energy or signal to noise ratio of; calculating the ith second subband signal S2iSignal energy or signal to noise ratio of (d); and if the ith first subband signal S1iIs larger than the ith second subband signal S2iAnd the signal energy or signal-to-noise ratio of, and the ith first subband signal S1iWith said ith second subband signal S2iIf the signal energy or signal-to-noise ratio difference value between the first sub-band signal S and the second sub-band signal S is greater than a set value, the ith second sub-band signal S is judged2iIncluding the wearer's voice signal.
An aspect of an embodiment of the present invention further provides a system for adaptively controlling a degree of sound amplification of a wearer based on multiple microphones for use in a hearing aid, the hearing aid comprising a feedforward microphone and a feedback microphone, the feedback microphone being located in an ear canal of the wearer when the hearing aid is worn, the non-target speech signal comprising a sound signal of the wearer; the system comprises: a first capturing module, configured to capture, through the feedforward microphone, an ambient signal of a target time period; a second capturing module, configured to capture the ear canal signal of the target time period through the feedback microphone; the judging module is used for judging whether the ambient environment signal comprises the sound signal of the wearer or not according to the ear canal signal; and an adjustment module for adjusting a gain of the ambient environment signal if the ambient environment signal includes the sound signal of the wearer.
An aspect of embodiments of the present invention further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor when executing the computer program implementing the steps of the method for adaptively controlling a degree of sound amplification of a wearer based on multiple microphones as described above.
An aspect of embodiments of the present invention further provides a computer-readable storage medium comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for adaptively controlling a degree of sound amplification of a wearer based on multiple microphones as described above when executing the computer program.
According to the method, the system, the equipment and the computer readable storage medium for adaptively controlling the sound amplification degree of the wearer based on the multi-microphone, provided by the embodiment of the invention, the ear canal signal is captured by the feedback microphone, and whether the wearer makes a sound or not is judged according to the ear canal signal. When the wearer makes a sound, the gain of the ambient environment signal is adjusted, so that the gain of the sound signal of the wearer is reduced, the sound of the wearer is effectively prevented from being excessively amplified, and the hearing comfort level is increased.
Drawings
Fig. 1 schematically illustrates an application environment diagram of a method for adaptively controlling a wearer's voice amplification degree based on multiple microphones according to an embodiment of the present invention;
FIG. 2 is a flow chart that schematically illustrates a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones, in accordance with an embodiment of the present invention;
FIG. 3 is a graph of a comparison of the frequency spectra of a feedforward microphone and a feedback microphone;
FIG. 4 is a sub-flowchart of step S204 in FIG. 2;
FIG. 5 is a flow chart that schematically illustrates a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones, in accordance with an embodiment of the present invention;
FIG. 6 is a sub-flowchart of step S204 in FIG. 2;
FIG. 7 is a sub-flowchart of step S206 in FIG. 2;
FIG. 8 is a sub-flowchart of step S704 in FIG. 7;
FIG. 9 is a sub-flowchart of step S206 in FIG. 2;
FIG. 10 is a sub-flowchart of step S206 in FIG. 2;
FIG. 11 is a sub-flowchart of step S206 in FIG. 2;
fig. 12 is an exemplary diagram of a system architecture of a hearing aid;
FIG. 13 is a schematic illustration of detecting whether a wearer is speaking;
FIG. 14 is another schematic illustration of detecting whether a wearer is speaking;
FIG. 15 is a band division diagram of the ambient signal of the feedforward microphone;
FIG. 16 is a gain adjustment diagram;
FIG. 17 is a diagram of a speech output process;
FIG. 18 schematically illustrates a block diagram of a system for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones in accordance with a second embodiment of the present invention; and
fig. 19 schematically shows a hardware architecture diagram of a computer device suitable for implementing a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones, according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the descriptions relating to "first", "second", etc. in the embodiments of the present invention are only for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In addition, technical solutions between various embodiments may be combined with each other, but must be realized by a person skilled in the art, and when the technical solutions are contradictory or cannot be realized, such a combination should not be considered to exist, and is not within the protection scope of the present invention.
In the description of the present invention, it should be understood that the numerical references before the steps do not identify the order of performing the steps, but merely serve to facilitate the description of the present invention and to distinguish each step, and thus should not be construed as limiting the present invention.
The terms of the present invention are explained:
the non-target speech signal is or includes the wearer's own voice signal.
Fig. 1 schematically shows an environmental application diagram of a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones according to an embodiment of the present invention. In an exemplary embodiment, the method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones may be implemented in a hearing aid 2.
The hearing aid 2 comprises a housing containing a feedforward microphone 21, a feedback microphone 22, a processor 23, a speaker 24.
A feedforward microphone 21 is located at one side of the housing for capturing sound signals around the wearer, i.e. ambient signals.
A feedback microphone 22 is located on the other side of the housing, which is located in the ear canal 4 of the wearer when the hearing aid 2 is worn, for capturing ear canal signals. The ear canal signals are primarily those transmitted through the skull of the wearer when speaking. In some exemplary embodiments, the feedback microphone 22 may be a bone conduction microphone.
A processor 23, electrically connected to the feedforward microphone 21, the feedback microphone 22 and the loudspeaker 24, processes the ambient signal and the ear canal signal. The processor 23 may be a DSP (Digital Signal Processing) chip or the like.
And a speaker 24 for receiving the sound signal processed by the processor 23 and outputting the processed sound signal to the ear canal 4.
A silicone sleeve 25 for at least partial insertion into the ear canal 4 when the hearing aid 2 is worn. The silicone sleeve 25 may to some extent block the entry of sound around the wearer into the ear canal 4. Of course, the material of the silicone sleeve 25 may be replaced.
The invention provides a method for adaptively controlling the sound amplification degree of a wearer based on a multi-microphone, which can acquire 22 an ear canal signal through a feedback microphone, judge whether the wearer speaks or not according to the ear canal signal, or distinguish whether the sound is mainly the sound of the wearer or the sound of other external sound producing bodies. When it is determined that the wearer is speaking or mainly speaking the wearer's voice, the gain of the ambient signal captured by the feedforward microphone 21 may be adjusted, for example, the gain of the ambient signal is reduced, that is, the gain of the wearer's own voice signal is reduced, so as to effectively prevent the wearer's own voice from being excessively amplified, and increase the hearing comfort. A number of embodiments will be provided below, each of which may be used to implement the above-described method of adaptively controlling the degree of sound amplification of a wearer based on multiple microphones. For ease of understanding, the following description will exemplarily describe the hearing aid 2 as the execution body.
Example one
In the present embodiment, a method of adaptively controlling the degree of sound amplification of a wearer based on multiple microphones is implemented in the hearing aid 2.
Fig. 2 is a flow chart schematically illustrating a method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones, according to an embodiment of the present invention. As shown in fig. 2, the method for adaptively controlling the degree of sound amplification of a wearer based on multi-microphone may include steps S200 to S206, in which:
step S200, capturing the surrounding environment signal of the target time interval through the feedforward microphone.
The feedforward microphone 21 may capture the ambient environment signal of the wearer when the hearing aid 2 is worn on the ear of the wearer. In an exemplary embodiment, the ambient signal may include various sound signals around the wearer, such as sound signals of other people, sound signals of animals, and various noise signals of automobiles and the like.
When the wearer himself makes a sound, the wearer's own sound signal is also propagated through the air to the feed-forward microphone 21, in which case the ambient signal also comprises the wearer's own sound signal.
Step S202, capturing the auditory canal signal of the target time interval through the feedback microphone.
The ear canal signals are various types of sound signals captured by the feedback microphone 22. However, the ear canal does not include the signal output by the speaker 24.
The feedback microphone 22 is less capable of capturing the ambient signal due to its location in the ear canal 4 and due to the acoustic isolation of the ear plug. In other words, the ambient signal is severely attenuated as it propagates to the feedback microphone 22.
However, as shown in fig. 3, since the feedback microphone 22 is located in the ear canal relative to the feedforward microphone 21, the captured ear canal signal is mainly concentrated in the low frequency part, mainly the signal below 1KHZ, and various external noise signals can be effectively suppressed. And based on the sound insulation effect, the feedback microphone 22 can accurately capture the sound signal of the wearer.
When the wearer makes his own voice, the wearer's own voice signal can be effectively propagated to the feedback microphone 22 by skull transmission. Thus, it may be determined whether the wearer is speaking by capturing the ear canal signals (primarily the wearer's own voice signals transmitted through the skull) through the feedback microphone 22.
Step S204, judging whether the surrounding environment signal comprises the sound signal of the wearer according to the ear canal signal.
The hearing aid 2 may determine whether the wearer is speaking himself by signal energy of the ear canal signal or other acoustic detection information, i.e.: determining whether the ambient environment signal comprises a sound signal of the wearer.
In an exemplary embodiment, as shown in fig. 4, step S204 may be implemented by: step S400, calculating first signal energy of the ear canal signal; step S402, judging whether the ambient environment signal comprises the sound signal of the wearer according to the first signal energy and a preset threshold value. When the wearer is speaking, his/her voice signal can be picked up by the feedback microphone 22 through the skull, which belongs to a solid physical medium and has less propagation attenuation, so that when the feedback microphone 22 picks up the ear canal signal including the wearer's voice signal, the signal energy will be relatively large. In view of this, the present embodiment may dynamically generate or customize a preset threshold, and when the first signal energy of the ear canal signal is greater than the preset threshold, it is determined that the wearer is speaking. It should be noted that when the preset threshold is self-defined, the preset threshold may be set between 60 and 80 decibels (dB).
In an exemplary embodiment, the preset threshold may be dynamically adjusted according to the real-time environment of the surroundings to increase the accuracy of determining whether the wearer is speaking or whether the ambient signals are primarily from the wearer's voice signals. As shown in fig. 5, the threshold setting step may be implemented by: step S500, calculating a second signal energy of the ambient environment signal; step S502, adjusting the preset threshold value according to the second signal energy; the energy of the second signal and the preset threshold value are in a positive relationship. There may be a linear or non-linear relationship between the second signal energy and the preset threshold. The following exemplary provides a mapping relationship between the two, where the preset threshold may be 60dB when the second signal energy is from 60 (excluding the endpoint) to 70 (including the endpoint), the preset threshold may be 70dB when the second signal energy is from 70 (excluding the endpoint) to 80 (including the endpoint), and the preset threshold may be 75dB when the second signal energy is greater than 80 dB.
In an exemplary embodiment, as shown in fig. 6, step S204 may be implemented by: step S600, decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Step S602, decomposing the ambient environment signal into N second subband signals S21、S22、…S2M、…S2N(ii) a A first subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; step S604, calculating the ith first subband signal S1iSignal energy or signal to noise ratio of; step S606, calculating the ith second subband signal S2iSignal energy or signal to noise ratio of; step S608, if the ith first subband signal S1iIs larger than the ith second subband signal S2iAnd the signal energy or signal-to-noise ratio of, and the ith first subband signal S1iWith said ith second subband signal S2iThe signal energy or the signal-to-noise ratio difference value between the first sub-band signal S and the second sub-band signal S is larger than a set value, and the ith second sub-band signal S is judged2iIncluding the wearer's voice signal. In this embodiment, the hearing aid 2 may calculate the probability of whether there is a wearer speaking in each subband by detecting the difference between the energy of the feedback microphone in each subband or the energy of the signal-to-noise ratio and the energy of the feedforward microphone in each subband, and the probability of detecting speech in each subband is between 0 and 1. For example, if the subband energy or signal-to-noise ratio of the feedback microphone is much greater than the subband energy or signal-to-noise ratio of the feedforward microphone, it is more likely that the wearer is speaking, and if the opposite is the case, it is more likely that other sounds in the external environment are present.
Step S206, if the ambient signal includes the sound signal of the wearer, adjusting a gain of the ambient signal.
If the ambient signal comprises the sound signal of the wearer, the gain of the ambient signal may be reduced.
If the ambient signal does not include the wearer's voice signal, then the gain of the ambient signal is maintained or increased.
In an exemplary embodiment, as shown in fig. 7, step S206 may be implemented by: step S700, decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Step S702, decomposing the ambient environment signal into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; and step S704, respectively adjusting the gain of the second sub-band signal corresponding to each first sub-band signal according to each first sub-band signal. This embodiment may perform a difference processing on the respective second subband signals,suppressing the wearer's voice signal in the ambient environment signal without suppressing other voice signals in the ambient environment signal.
The ear canal signal captured by the feedback microphone 22 is mainly concentrated in the low frequency part, mainly signals below 1 KHZ. Thus, N is greater than M.
(1) In the first to mth signal bands:
first subband signal S1iWith corresponding second subband signal S2iHas a one-to-one correspondence relationship. May pass through the first subband signal S1iDirecting the corresponding second subband signal S2iAnd (4) adjusting the gain. For example, by a first subband signal S11Directing the corresponding first second subband signal S21By a second first subband signal S12Directing a corresponding second subband signal S22… passing the mth first subband signal S1MDirect the corresponding Mth second subband signal S2MGain adjustment of (1). It is understood that by such a fine guidance, it is possible to output the sound signal of the non-wearer in the ambient signal to the maximum extent and to improve sound output smoothness.
The above guidance may be based on the respective first subband signal S1iThe signal energy of (a). For example:
in an exemplary embodiment, as shown in fig. 8, step S704 may be implemented by: step S800, calculating the ith first sub-band signal S1iThe signal energy of (a); step S802, according to the ith first sub-band signal S1iOf the ith second subband signal S, adjusting the signal energy of the ith second subband signal S2iThe gain of (c). The ith first subband signal S1iSignal energy E ofiThe larger the i-th second subband signal S is2iThe more likely it is to include the wearer 'S sound signal and the more energy of the wearer' S sound signal, and therefore, a greater amplitude is required to scale down the i-th second subband signal S2iGAIN GAIN ofi. For example, a standard signal energy E (e.g., 60dB) may be set when the ith signal energy isA first subband signal S1iSignal energy E ofiIf the energy is greater than the standard signal energy E, the ith second subband signal S can be obtained by the following formula2iGAIN GAIN ofiThe descending amplitude of (c): GAINi＝Ei-E. As an example, assuming that the standard signal energy E is 60dB, if the first subband signal S11Signal energy E of1At 68dB, the first and second subband signals S are required21The gain of (2) is adjusted down by 8dB based on the standard gain. The standard gain may be a preset value.
(2) In the M +1 th to nth signal bands:
(2.1) passing the Mth first subband signal S1MDirecting the M +1 th to Nth second subband signals S2(M+1)～S2(N)Gain adjustment of (1).
As shown in fig. 9, step S206 may further include the steps of: step S900, according to the Mth first sub-band signal S1MThe gain of the M +1 th to nth second subband signals is adjusted. The gain adjustment amplitude can be referred to above. The advantages of this embodiment are: mth first subband signal S1MIs closer to the M +1 th to Nth second sub-band signals S2(M+1)～S2(N)And thus the adjustment is more accurate.
(2.2) passing the first to Mth first subband signals S11～S1MDirecting the M +1 th to Nth second subband signals S2(M+1)～S2(N)Gain adjustment of (1).
As shown in fig. 10, step S206 may further include the steps of: step S1000, calculating the energy average value of the M first sub-band signals, and adjusting the gains of the M +1 th to Nth second sub-band signals according to the energy average value. The advantages of this embodiment are: preventing the Mth first subband signal S1MUnder the condition of parameter error, for the M +1 th to the N-th second sub-band signals S2(M+1)～S2(N)The wrong guidance of the gain adjustment ensures the effectiveness of the guidance.
(2.3) passing the first to Mth first sub-bandsSignal S11～S1MDirecting the M +1 th to Nth second subband signals S2(M+1)～S2(N)Gain adjustment of (1).
As shown in fig. 11, step S206 may further include the steps of: step S1100, calculating energy weighted values of the M first subband signals based on auditory characteristics of human ears, and adjusting gains of the M +1 th to nth second subband signals according to the energy weighted values. The advantages of this embodiment are: the M +1 th to Nth second subband signals S can be accurately adjusted2(M+1)～S2(N)The gain of (c).
For ease of understanding, one example is provided below. As shown in fig. 12, the system architecture of the hearing aid 2 may comprise 4 modules:
module 1: a feedback microphone, an analysis filter;
and (3) module 2: feedback microphone, multi-subband VAD (acoustic event) discrimination;
and a module 3: a feedforward microphone, an analysis filter;
and (4) module: guiding multi-subband gain amplification by using a feedback microphone VAD;
as in fig. 13, module 1 and module 2 are used to: and calculating to obtain the probability of whether a wearer speaks in each sub-band by detecting the energy of the feedback microphone in each sub-band or the energy of the signal-to-noise ratio, wherein the probability of detecting the speaking in each sub-band is between 0 and 1.
As in fig. 14, module 1 and module 2 are used to: the probability of whether a wearer speaks in each sub-band is calculated by detecting and comparing the energy of the feedback microphone in each sub-band or the energy of the signal to noise ratio and the difference of the energy of each sub-band of the feedforward microphone and the energy of the signal to noise ratio, and the probability of detecting speaking in each sub-band is between 0 and 1. For example, if the subband energy or signal-to-noise ratio of the feedback microphone is much greater than the subband energy or signal-to-noise ratio of the feedforward microphone, it is more likely that the wearer is speaking, and if the opposite is the case, it is more likely that the sound is in the external environment.
As in fig. 15, the module 3 is used to: the input signal of the feedforward microphone is passed through a series of filters, so that the output signal is a signal of each sub-band, that is, the original signal in the time domain is converted into signals in N independent frequency domains.
As in fig. 16 and 17, the module 4 is used to: and dynamically determining the gain for amplifying the external sound according to the probability of the wearer speaking sound judged in each sub-band obtained by calculation. For example, if the signal energy or signal-to-noise ratio collected by the feedforward microphone is much larger than that collected by the feedback microphone, linear amplification may be used, otherwise, amplification with a certain compression ratio needs to be performed in the sub-band to prevent over-amplification of the speaking voice of the wearer.
Example two
As shown in fig. 18, fig. 18 schematically illustrates a block diagram of a system 1800 for adaptive control of a wearer's voice amplification level based on multiple microphones in accordance with a second embodiment of the invention. The system 1800 for adaptively controlling the degree of sound amplification by a wearer based on multiple microphones is used in a hearing aid comprising a feed-forward microphone and a feedback microphone, the feedback microphone being located in the ear canal of the wearer when the hearing aid is worn. The system may be partitioned into one or more program modules, which are stored in a storage medium and executed by one or more processors to implement embodiments of the invention. The program modules referred to in the embodiments of the present invention refer to a series of computer program instruction segments that can perform specific functions, and the following description will specifically describe the functions of the program modules in the embodiments.
As shown in fig. 18, the system 1800 for adaptive control of a wearer's voice amplification based on multiple microphones may include a first capture module 1802, a second capture module 1804, a determination module 1806, and an adjustment module 1808. Wherein:
a first capture module 1802 configured to capture an ambient signal for a target period of time via the feed-forward microphone.
A second capturing module 1804, configured to capture, via the feedback microphone, the ear canal signal of the target time period.
A determining module 1806, configured to determine whether the ambient environment signal includes the sound signal of the wearer according to the ear canal signal.
An adjusting module 1808, configured to adjust a gain of the ambient environment signal if the ambient environment signal includes the sound signal of the wearer.
In an exemplary embodiment, the determining module 1806 is further configured to: calculating a first signal energy of the ear canal signal; and judging whether the ambient environment signal comprises the sound signal of the wearer or not according to the first signal energy and a preset threshold value.
In an exemplary embodiment, the system further comprises a setup module (not identified) for: calculating a second signal energy of the ambient signal; adjusting the preset threshold value according to the energy of the second signal; the energy of the second signal and the preset threshold value are in a positive relationship.
In an exemplary embodiment, the adjusting module 1808 is further configured to: decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a The ambience signal is decomposed into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; and respectively adjusting the gain of the second sub-band signal corresponding to each first sub-band signal according to each first sub-band signal.
In an exemplary embodiment, the adjusting module 1808 is further configured to: calculating the ith first subband signal S1iThe signal energy of (a); and according to said ith first subband signal S1iOf the ith second subband signal S, adjusting the signal energy of the ith second subband signal S2iThe gain of (c).
In an exemplary embodiment, the adjusting module 1808 is further configured to: according to the Mth first subband signal S1MThe gain of the M +1 th to nth second subband signals is adjusted.
In an exemplary embodiment, the adjusting module 1808 is further configured to: calculating the energy average value of the M first sub-band signals, and adjusting the gains of the M +1 th to Nth second sub-band signals according to the energy average value; or calculating energy weighted values of the M first sub-band signals based on the auditory characteristics of the human ears, and adjusting gains of the M +1 th to Nth second sub-band signals according to the energy weighted values.
In an exemplary embodiment, the determining module 1806 is further configured to: decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Decomposing the ambience signal into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; and calculating the ith first subband signal S1iSignal energy or signal to noise ratio of; calculating the ith second subband signal S2iSignal energy or signal to noise ratio of (d); and if the ith first subband signal S1iIs larger than the ith second subband signal S2iAnd the signal energy or signal-to-noise ratio of, and the ith first subband signal S1iWith said ith second subband signal S2iIf the signal energy or signal-to-noise ratio difference value between the first sub-band signal S and the second sub-band signal S is greater than a set value, the ith second sub-band signal S is judged2iIncluding the wearer's voice signal.
EXAMPLE III
As shown in fig. 19, fig. 19 schematically illustrates a hardware architecture diagram of a computer device 1900 adapted to implement the method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones according to the third embodiment of the present invention. In this embodiment, the computer 1900 is a device capable of automatically performing numerical calculation and/or information processing in accordance with a command set or stored in advance. For example, the hearing aid may be a hearing aid, an earphone having a hearing aid function, or the like. As shown in fig. 19, computer device 1900 includes at least, but is not limited to: memory 1910, processor 1920, network interface 1930 may be communicatively linked to each other through a system bus. Wherein:
the memory 1910 includes at least one type of computer-readable storage medium including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or DX memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, storage 1910 may be an internal storage module of computer device 1900, such as a hard disk or memory of computer device 1900. In other embodiments, the memory 1910 may also be an external storage device of the computer device 1900, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), or the like provided on the computer device 1900. Of course, memory 1910 may also include both internal and external memory modules of computer device 1900. In this embodiment, memory 1910 is generally configured to store an operating system installed on computer device 1900 and various types of application software, such as program code for a method for adaptively controlling the degree of wearer's voice amplification based on multiple microphones. In addition, the memory 1910 may also be used to temporarily store various types of data that have been output or are to be output.
Processor 1920 may be, in some embodiments, a Central Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data Processing chip. The processor 1920 is generally configured to control overall operation of the computer device 1900, such as performing control and processing related to data interaction or communication with the computer device 1900. In this embodiment, the processor 1920 is configured to execute program codes stored in the memory 1910 or process data.
Network interface 1930, which can comprise a wireless network interface or a wired network interface, is typically used to establish communication links between computer device 1900 and other computer devices. For example, the network interface 1930 is used to connect the computer apparatus 1900 to an external terminal via a network, establish a data transmission channel and a communication link between the computer apparatus 1900 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth (Bluetooth), or Wi-Fi. A
It should be noted that fig. 19 only shows a computer device having components 1910 and 1930, but it should be understood that not all of the shown components are required to be implemented, and more or fewer components may be implemented instead.
In this embodiment, the method for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones stored in the memory 1910 may be further divided into one or more program modules and executed by one or more processors (in this embodiment, the processor 1920) to implement the embodiment of the present invention.
Example four
The invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of a method of adaptively controlling a degree of sound amplification of a wearer based on multiple microphones in an embodiment.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of the computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the computer device. Of course, the computer-readable storage medium may also include both internal and external storage devices of the computer device. In this embodiment, the computer-readable storage medium is generally used for storing an operating system and various types of application software installed on the computer device, such as program codes of the method for adaptively controlling the sound amplification degree of the wearer based on the multi-microphone in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and alternatively, they may be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described may be performed in an order different than that described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple ones of them may be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (8)

1. A method for adaptive control of the degree of sound amplification of a wearer based on multiple microphones for use in a hearing aid comprising a feed forward microphone and a feedback microphone, said feedback microphone being located in the ear canal of the wearer when the hearing aid is worn; the method comprises the following steps:
capturing, by the feedforward microphone, ambient signals for a target time period;
capturing, by the feedback microphone, an ear canal signal for the target time period;
determining whether the ambient environment signal comprises a sound signal of the wearer according to the ear canal signal; and
adjusting a gain of the ambient environment signal if the ambient environment signal comprises a sound signal of the wearer;
wherein the step of adjusting the gain of the ambient signal if the ambient signal comprises the wearer's voice signal comprises:
decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Decomposing the ambience signal into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; respectively adjusting the gain of a second sub-band signal corresponding to each first sub-band signal according to each first sub-band signal;
wherein, the first sub-band signal S is passed11Directing the corresponding first second subband signal S21By a second first subband signal S12Directing a corresponding second subband signal S22… passing the mth first subband signal S1MDirect the corresponding Mth second subband signal S2MGain adjustment of (3);
according to the Mth first subband signal S1MAdjusting the gain of the M +1 th to nth second subband signals; or calculating the energy average value of the M first sub-band signals, and adjusting the gains of the M +1 th to the Nth second sub-band signals according to the energy average value; or calculating energy weighted values of the M first sub-band signals based on the auditory characteristics of human ears, and adjusting gains of the M +1 th to Nth second sub-band signals according to the energy weighted values.
2. The method of claim 1, wherein the step of determining whether the ambient environment signal includes the wearer's voice signal from the ear canal signal comprises:
calculating a first signal energy of the ear canal signal; and
and judging whether the ambient environment signal comprises the sound signal of the wearer or not according to the first signal energy and a preset threshold value.
3. The multi-microphone based adaptive control method for the degree of sound amplification of a wearer according to claim 2, further comprising the step of setting a threshold value:
calculating a second signal energy of the ambient signal; and
adjusting the preset threshold value according to the second signal energy; the energy of the second signal and the preset threshold value are in a positive relationship.
4. The method of claim 1, wherein the step of adjusting the gain of the second sub-band signal corresponding to each first sub-band signal according to the first sub-band signal comprises:
calculating the ith first subband signal S1iThe signal energy of (a); and
according to the ith first subband signal S1iOf the ith second subband signal S, adjusting the signal energy of the ith second subband signal S2iThe gain of (c).
5. The multi-microphone based adaptive control method for sound amplification of a wearer according to claim 1, wherein the step of determining whether the ambient environment signal includes the wearer's sound signal according to the ear canal signal comprises:
decomposing the ear canal signal into M first subband signals S11、S12、…S1M；
Decomposing the ambience signal into N second subband signals S21、S22、…S2M、…S2N(ii) a First subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; and
calculating the ith first subband signal S1iSignal energy or signal to noise ratio of;
calculating the ith second subband signal S2iSignal energy or signal to noise ratio of (d); and
if the ith first subband signal S1iIs larger than the ith second subband signal S2iAnd the signal energy or signal-to-noise ratio of, and the ith first subband signal S1iWith said ith second subband signal S2iIf the signal energy or signal-to-noise ratio difference value between the first sub-band signal S and the second sub-band signal S is greater than a set value, the ith second sub-band signal S is judged2iIncluding the wearer's voice signal.
6. A system for adaptively controlling the degree of sound amplification of a wearer based on multiple microphones for use in a hearing aid, said hearing aid comprising a feed-forward microphone and a feedback microphone, said feedback microphone being located in the ear canal of said wearer when said hearing aid is worn; the system comprises:
a first capturing module, configured to capture, through the feedforward microphone, an ambient signal of a target time period;
a second capturing module for capturing the ear canal signal of the target time period through the feedback microphone;
the judging module is used for judging whether the ambient environment signal comprises a sound signal of the wearer or not according to the ear canal signal; and
an adjustment module for adjusting a gain of the ambient environment signal if the ambient environment signal comprises a sound signal of the wearer;
wherein the adjustment module is further configured to:
decomposing the ear canal signal into M first subband signals S11、S12、…S1M(ii) a Decomposing the ambience signal into N second subband signals S21、S22、…S2M、…S2N(ii) a A first subband signal S1iWith corresponding second subband signal S2iI is more than or equal to 1 and less than or equal to M, i is a positive integer, and the frequency bands of the signals are the same; respectively adjusting the gain of a second sub-band signal corresponding to each first sub-band signal according to each first sub-band signal;
by a first subband signal S11Directing the corresponding first second subband signal S21By a second first subband signal S12Directing a corresponding second subband signal S22… passing the mth first subband signal S1MDirect the corresponding Mth second subband signal S2MAdjusting the gain of (2);
according to the Mth first subband signal S1MAdjusting the gain of the M +1 th to nth second subband signals; or calculating the energy average value of the M first sub-band signals, and adjusting the gains of the M +1 th to the Nth second sub-band signals according to the energy average value; or calculating energy weighted values of the M first sub-band signals based on the auditory characteristics of human ears, and adjusting gains of the M +1 th to Nth second sub-band signals according to the energy weighted values.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor, when executing the computer program, is adapted to carry out the steps of the method of multi-microphone based adaptive control of the degree of sound amplification of a wearer according to any of claims 1 to 5.
8. A computer-readable storage medium, having stored thereon a computer program for execution by at least one processor to cause the at least one processor to perform the steps of the method for multi-microphone based adaptive control of a wearer's degree of sound amplification of any one of claims 1-5.
CN202110414004.4A 2021-04-16 2021-04-16 Method for adaptively controlling sound amplification degree of wearer based on multiple microphones Active CN113132881B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110414004.4A CN113132881B (en) 2021-04-16 2021-04-16 Method for adaptively controlling sound amplification degree of wearer based on multiple microphones

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110414004.4A CN113132881B (en) 2021-04-16 2021-04-16 Method for adaptively controlling sound amplification degree of wearer based on multiple microphones

Publications (2)

Publication Number Publication Date
CN113132881A CN113132881A (en) 2021-07-16
CN113132881B true CN113132881B (en) 2022-07-19

Family

ID=76776992

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110414004.4A Active CN113132881B (en) 2021-04-16 2021-04-16 Method for adaptively controlling sound amplification degree of wearer based on multiple microphones

Country Status (1)

Country Link
CN (1) CN113132881B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114267373A (en) * 2021-08-31 2022-04-01 北京聆通科技有限公司 Sound processing method and device for hearing-aid equipment
CN117528370A (en) * 2022-07-30 2024-02-06 华为技术有限公司 Signal processing method and device, equipment control method and device
CN118338219A (en) * 2023-01-11 2024-07-12 上海又为智能科技有限公司 Audio signal processing apparatus and method

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1897765A (en) * 2005-07-11 2007-01-17 西门子测听技术有限责任公司 Hearing device and corresponding method for ownvoices detection
CN105979460A (en) * 2015-03-13 2016-09-28 西万拓私人有限公司 Method for operating a hearing aid and hearing aid
CN106792305A (en) * 2015-11-19 2017-05-31 鹦鹉无人机股份有限公司 Audio headset
CN107431867A (en) * 2014-11-19 2017-12-01 西万拓私人有限公司 Method and apparatus for quickly identifying own voices
CN108200523A (en) * 2016-11-24 2018-06-22 奥迪康有限公司 Include the hearing devices of self voice detector

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2877813B2 (en) * 1988-03-23 1999-04-05 富士通テン株式会社 Volume and sound quality adjustment device
US9978397B2 (en) * 2015-12-22 2018-05-22 Intel Corporation Wearer voice activity detection
CN108694955B (en) * 2017-04-12 2020-11-17 华为技术有限公司 Coding and decoding method and coder and decoder of multi-channel signal

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1897765A (en) * 2005-07-11 2007-01-17 西门子测听技术有限责任公司 Hearing device and corresponding method for ownvoices detection
CN107431867A (en) * 2014-11-19 2017-12-01 西万拓私人有限公司 Method and apparatus for quickly identifying own voices
CN105979460A (en) * 2015-03-13 2016-09-28 西万拓私人有限公司 Method for operating a hearing aid and hearing aid
CN106792305A (en) * 2015-11-19 2017-05-31 鹦鹉无人机股份有限公司 Audio headset
CN108200523A (en) * 2016-11-24 2018-06-22 奥迪康有限公司 Include the hearing devices of self voice detector

Also Published As

Publication number Publication date
CN113132881A (en) 2021-07-16

Similar Documents

Publication Publication Date Title
CN113132881B (en) Method for adaptively controlling sound amplification degree of wearer based on multiple microphones
US10631087B2 (en) Method and device for voice operated control
EP3701525B1 (en) Electronic device using a compound metric for sound enhancement
US10631105B2 (en) Hearing aid system and a method of operating a hearing aid system
EP3337190B1 (en) A method of reducing noise in an audio processing device
US10154353B2 (en) Monaural speech intelligibility predictor unit, a hearing aid and a binaural hearing system
EP3289782B1 (en) Process and hearing aid adjustment system architecture for remotely adjusting a hearing aid
WO2009128853A1 (en) Method and device for voice operated control
US9374646B2 (en) Binaural enhancement of tone language for hearing assistance devices
EP2942777B1 (en) Method and apparatus for pre-processing speech to maintain speech intelligibility
CN112822617B (en) Hearing aid system comprising a hearing aid instrument and method for operating a hearing aid instrument
CN113225657A (en) Multi-channel squeal suppression method based on double-microphone architecture
CN115314823A (en) Hearing aid method, system and equipment based on digital sounding chip
US20170325033A1 (en) Method for operating a hearing device, hearing device and computer program product
US11445307B2 (en) Personal communication device as a hearing aid with real-time interactive user interface
CN113132885B (en) Method for judging wearing state of earphone based on energy difference of double microphones
KR102403996B1 (en) Channel area type of hearing aid, fitting method using channel area type, and digital hearing aid fitting thereof
CN113132882B (en) Multi-dynamic-range companding method and system
US11929071B2 (en) Hearing device system and method for operating same
WO2023002515A1 (en) Movement-controlled earphone device for the transceiving of acoustic signals
CN118843059A (en) Fitting method of hearing aid

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant