DE112018000717T5 - METHOD, DEVICES, ARRANGEMENTS AND COMPONENTS FOR DETERMINING THE ACTIVITY OF USER VOICE ACTIVITY - Google Patents

Abstract

Many headsets use Automatic Noise Cancellation (ANC), which dramatically reduces the reception of background noise and improves the listening experience. Unfortunately, the voice microphones in these devices often pick up ambient noise that these headsets emit during phone calls or other communication sessions with other users. In response, many headsets and communication devices provide manual mute circuitry, but users often forget to toggle mute on and / or off, creating additional communication problems. To counteract this, the local inventors have, among other things, devised an exemplary headset that detects the presence or absence of user utterances and automatically mutes and reactivates the voice microphone without the user having to do anything. Some embodiments use relationships between feedback and feedforward signals in the ANC circuit to capture user utterances so that additional hardware in the headset can be eliminated. Other embodiments also use the utterance detection function to enable and disable keyword detectors.

Description

COPYRIGHT NOTE AND PERMISSION

Part of this patent application contains material that is subject to copyright protection. The copyright holder has no objection to anyone regarding a facsimile reproduction of the patent application or the disclosure contained therein, as stored in the patent files or documents, but otherwise reserves all copyrights. The following marking applies to this document: Copyright © 2017, AVNERA CORPORATION.

RELATED APPLICATIONS

The present application claims priority from U.S. Patent Application No. 15 / 711,793 , filed on September 21, 2017, and preliminary US patent applications 62 / 456,100 , filed February 7, 2017, 62 / 459,055, filed February 14, 2017, and 62 / 532,964, filed July 14, 2017. The disclosure content of all four applications is incorporated herein by reference.

TECHNICAL AREA

Various embodiments of the invention generally relate to the automatic detection of user voice activity in headsets of different types, such as those with automatic noise reduction.

BACKGROUND

Headsets, or "headsets", usually have a microphone along with one or two ear pieces or ear plugs that are worn over, on or in the ears of users to enable electronic communication. Many modern headsets also have circuits for automatic noise reduction (ANR) or automatic noise cancellation (ANC) to automatically detect and suppress significant amounts of ambient noise, so that the listening experience is improved for the user. ANC circuits are simple in principle, but complicated in implementation, since many headsets have up to six microphones (“mics”): two for feedback ANC (FB-ANC), two for feedforward ANC (FF-ANC ) and one or two for recording user voice signals.

One problem recognized by the present inventors is that ANC headsets improve the listening experience of the users wearing them, but do little or nothing to improve the quality of the signals transmitted from the headset to other devices. For example, in many noisy environments, users who wear headsets for their cell phones typically need to manually mute their phones to avoid background noise being transmitted from the phone to other phones and devices. While this in principle helps effectively against the transmission of unwanted noise, it often leads to communication gaps, since users who have muted their phones forget to unmute them when they start speaking again. In addition, the problem with possible communication gaps in conference calls with many users who manually switch to mute and ready to record is even greater.

In attempting to solve the problem, some developers have provided circuitry that monitors the voice output of the voice microphone and automatically turns the mute on or off depending on whether or not the level of the voice microphone output exceeds one or more thresholds. However, the monitoring circuits suffer from at least two problems that have prevented their use in mobile phones. First of all, they consume a considerable amount of energy and thus shorten the battery life. Second, the circuits themselves create communication gaps, not only because they react slowly when speech begins or stops, but also because they confuse external noises, such as voices from nearby people, with the user's voice.

Accordingly, the present inventors have recognized that there is a need for better methods for reducing ambient noise transmission by ANC and other types of headsets.

SUMMARY OF THE INVENTION

To address one or more of these and / or other needs or problems, the present inventors, among others, have devised one or more exemplary systems, kits, methods, devices, arrangements, and / or other components that automatically detect the presence or absence of a user's utterances record and implement an assigned muting or other voice or language-dependent functionalities. Some embodiments include a voice microphone, at least two control microphones, and processing circuitry configured to mute the voice microphone based on a relationship between the output signals of the control microphones.

In a first embodiment, the two control microphones, for example a microphone arranged on a left earpiece and a microphone arranged on a right earpiece, are designed to acoustically couple approximately symmetrically into a user's voice range. The processing circuit determines whether a user speaks depending on a measured temporal symmetry - for example a phase relationship - of output signals of the two microphones, a high symmetry (approximately simultaneous arrival at both microphones) characterizing a speech utterance of the user, and a low symmetry ambient noise or indicates other expressions of language. In some variants of the first embodiment, the two control microphones are the left and right feedback microphones in an AND headset. Some other variants measure or estimate the phase relationship using complex coherence of the sampled microphone output signals and activate the mute device whenever the real part of the complex coherence falls below a threshold.

In a second embodiment, the two control microphones are designed to acoustically couple asymmetrically into a user's voice range, that is, one microphone couples more acoustically into a user's voice range than the other. The processing circuit is designed to activate or deactivate the muting depending on the relative strength of the energy level of the output signals of the two microphones. For example, in some ANC embodiments, the microphone with the stronger acoustic coupling can be the feedback error microphone in an ANC ear cup and the other the pilot microphone. The processing circuit compares a ratio of the output signal of the feedback error microphone and the output signal of the pilot microphone to a threshold value in order to determine whether the muting function should be activated or deactivated. Some variants can use two microphones, which are arranged on a bracket or in a cable, instead of the two ANC control microphones.

The processing circuit compares a ratio of the output signal of the feedback error microphone and the output signal of the pilot microphone to a threshold value in order to determine whether the muting function should be activated or deactivated. Some variants can use two microphones, which are arranged on a stick or a cable, instead of the two ANC control microphones. Generalized embodiments use two or more microphones to generate two signals, a first signal being more closely coupled to a user's speech than a second signal, and the second signal being more strongly coupled to ambient noise than the first signal.

A third embodiment estimates a transfer function that takes into account both the phase and amplitude differences between the signals from the two microphones, such as the feedback error microphone and the pilot environment microphone. Some variants of this embodiment can dispense with the ANC microphone and use other pairs of microphones, such as a speech microphone and another microphone. The transfer function can be estimated using a variety of methods, such as Kalman filtering or another type of optimum estimator.

One or more embodiments also include other features. For example, some embodiments include a fader to fade out music or other audio signals that are output through a speaker within an earpiece equipped with one or more of the microphones. Some embodiments have mute delay and defer functions to minimize the risk of being muted between spoken words or during short pauses. Some other embodiments include a buffer memory and a prediction functionality to ensure timely deactivation of the muting of the voice microphone and thereby to avoid partial loss of the user's utterance.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments are described herein with reference to the following accompanying drawings (Fig.). These drawings are provided with reference numerals for various features and components, and these characters are used in the following description as an aid to understanding the teachings, where like characters refer to the same or similar features and components.

1 shows a block diagram of an exemplary headset system or headset arrangement 100 associated with one or more embodiments of the present invention.

2 FIG. 4 shows a block diagram of an exemplary method for operating a user voice activity detector as part of the arrangement 100 associated with one or more embodiments of the present invention.

3 shows a conceptual system diagram of half of a headset system 300 associated with one or more embodiments of the present invention.

4 FIG. 4 shows a block diagram of a generalized electroacoustic system with multiple audio transmitters and audio input signals related to one or more embodiments of the present invention.

5 FIG. 4 shows a block diagram of a user voice activity detector associated with one or more embodiments of the present invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENT (S)

This document, which includes drawings and claims, describes one or more specific embodiments of one or more inventions. These embodiments, which are not intended to be limiting, but are intended to exemplify and explain the invention, are shown and described in sufficient detail so that those skilled in the art can implement and implement the invention (s). In addition, where appropriate, the description may omit certain information known to those skilled in the art to avoid obscuring the invention (s).

1 shows an exemplary ANR headset arrangement or an ANR headset system 100 which in ears 102A and 102B Of the head 101 a user is used (top view). The head 101 of the user includes corresponding left and right ears 102A and 102B , a mouth 103 and a vocal tract 104 of the user. The vocal tract 104 includes the user's mouth and larynx (not shown). The order 100 has left and right earcups or earpieces 110A and 110B , an optional headband 100C , a built-in voice microphone or a bow voice microphone 120 , a processing circuit 130 and a host or external device 140 on.

The earpieces or earpieces 110A and 110B, which can take the form of ear canals, earmolds, over-the-ear or on-ear headphones, and a vertical axis 110D , include corresponding ANR drivers DA and DB, feedforward microphones FFA and FFB, and corresponding feedback microphones FBA and FBB. ANR drivers DA and DB share the internal volumes of the corresponding earpieces or earpieces 110A and 110B into front cavity areas and rear cavity areas (not separately marked). The pilot environment microphones FFA and FFB are located within or adjacent to the rear cavities, further away from the corresponding ears 102A and 102B than the FBA and FBB feedback mics, ensuring that they have a stronger acoustic coupling to ambient noise and a weaker acoustic coupling to the head 101 of the user, the auditory canals of the user and the vocal tract 104 have as the feedback error microphones FBA and FBB. The FBA and FBB feedback error microphones are similarly located within or on the corresponding earpieces, ensuring that they have substantially identical or symmetrical acoustic coupling via the head or (body) bone conduction paths 104A and 104B to the on-axis 110D lying vocal tract 104 of the user. In addition, the feedback microphones have an essentially symmetrical acoustic coupling via air conduction paths P1 and P2 to off-axis noise sources, such as the ambient noise source N. (The feedback and pilot microphones in the same earpiece have one very asymmetrical response function to speech utterances of the user and surrounding sounds.) The microphones and drivers are over appropriate wired or wireless communication connections 131 and 132 with the processing circuit 130 coupled.

The processing circuit 130 which, in some embodiments, takes the form of a digital signal processor with associated memory and other integrated component equipment and entirely from the device 140 separately or partially or completely in the device 140 is integrated, has a set of input / output nodes 133 , an ANR processing module 134 , a user voice recognition module 135 , a mute module 136 , language-dependent modules 137 and a module for the detection of on-ear headphones.

The input / output nodes 133 have a voice microphone output signal node Vm, a left feedback microphone signal node FBA (FBL), a right feedback microphone signal node FBB (FBR), a left feedforward microphone signal node FFA (FFL), a right feedforward microphone signal node FFB (FFRxA) and R input signal nodes for audio signals RB / FFRxA, and RxBnode for input signals , (The use of the abbreviation "Rx" here refers to RxA or RxB in detail or in common and can include telephone audio signals.)

The ANR processing module 134 provides circuitry and machine-readable instructions to suppress ambient noise within the front volumes of the earpieces, including the user's ear cavities. In particular, the module receives 134 Output signals from the pre-control microphones FFA and FFB, which represent a sum of the total acoustic energy within their corresponding ear canals or ear cavities. module 134 also receives an earphone audio signal from the device 140 , such as a smartphone, media player, two-way radio, or other electronic audio source. In response to an ambient noise signal from the pre-control microphones FFA and FFB, the ANR processing circuit generates noise canceling signals and sends a mixture or a sum of the noise canceling signals and the earphone audio signal (Rx) to the ANR drivers DA and DB, which generate acoustic energy, which are essential parts of the extinguish the perceived ambient noise and provide an acoustic reproduction of the earphone audio signal. In some embodiments, the ANR processing module includes 134 adaptive ANR processing, such as in the U.S. Patent Application 15 / 069,271 , filed March 4, 2016, the disclosure of which is incorporated herein by reference.

The user voice activity detection module (UVAD) 135 has logic circuits and / or stored machine-readable instructions to process input signals derived from two or more of the FFA and FFB pilot microphones and the FBA and FBB feedback microphones. the presence or absence of a user's utterances in the output signal of the voice microphone 120 to capture or predict. Specifically, the exemplary embodiment provides two methods of capturing user voice activity. (Some embodiments provide a UVAD that uses any ambient microphone in combination with the feedback microphone. Therefore, for example, the voice microphone and the feedback microphone can be used.)

The first method takes advantage of the insight that acoustic paths 104A and 104B through the head of the user generally have the same length and are essentially similar in material composition, since the head of the user is sufficiently symmetrical. This means that the speech component of the output signals of the feedback microphones are essentially identical in amplitude and phase when the user speaks and are essentially different when the user is not speaking. In other words, the output signals of the feedback microphones have approximately the same voice data portion and therefore have a high coherence (in the phase relationship) when the user speaks. In addition, the distance from ambient noise sources to the two microphones is usually not the same or asymmetrical as through the paths P1 and P2 in 1 indicated, which means that the coherence will be rotated in the phase. There may still be coherence, which means that the amplitude is close to 1, but the angle is not 0, which indicates a situation with phase synchronization.

Therefore, the exemplary embodiment uses the complex coherence of the left and right feedback error microphone signals FBL and FBR in a threshold test to determine whether user voice activity is likely to be present or not. If the threshold criterion is met, a mute command signal is generated to the module 136 mute, and if not, command signals to unmute are provided. More specifically, the swell test takes the following form:

If mean (real (C _xy (freq_range)))> CT then Unmute; else mute,
where mean () denotes an average function (over frequency); real () denotes the real part of a complex argument; C _xy (freq_range) denotes the complex coherence of the signals x and y over a frequency range named freq_range; subscript x denotes the FBL error microphone signal (left feedback microphone signal, FBA) and subscript y denotes the FBR error microphone signal (right feedback microphone signal, FBB); CT denotes the coherence threshold. The real part of the complex coherence C _xy is used because it is a measure of how "in phase" the two signals are. Note: abs (C _xy ) is 1 if only an average is found, but this can still be useful as the important information is still in the phase. The freq_range over which the mean is calculated can vary. However, some embodiments can use a range of 70 to 700 Hz or 100 to 500 Hz because the structure-borne acoustic of the human larynx mainly comprises low frequencies. In response to an indication from the C _xy detector that the user is not speaking (that the user's speech is not available), the exemplary embodiment mutes or attenuates the output signal from one or more microphones, such as from the speech microphone and / or the pre-control microphones, using the mute module 136 ,

In some embodiments, the coherence threshold CT may range between 0.7 and 0.9 inclusive, with slight variations in performance. If you set it too close to 1, the detector becomes more susceptible to recognizing false negatives (utterances are not recognized if they are present) if there is significant background noise; and if you set it too low, there are more false positives (that is, muting even when there is no utterance). In some embodiments, the coherency threshold may be adjusted dynamically based on system or environmental parameters, such as signal-to-noise ratio (of speech signals). That is, in the case of stronger speech versus noise, some embodiments move the threshold closer to 1, for example, 0.9, and in the case of strong noise, those embodiments lower the threshold level to avoid user speech can be muted. An exemplary threshold between 0.7 and 0.8, such as 0.75, is a good compromise.

This approach, based on left-right symmetry, is effective, but can be susceptible to on-axis ambient noise, which can occur, for example, if a second speaker is on an axis with the user (or generally equidistant from both) left and right ear piece). The approach also suffers from a poor signal-to-noise ratio (SNR). Noise on the axis will cause high coherence at the feedback error microphones and will therefore be mistakenly recognized as speech. Interestingly, one way to demonstrate this vulnerability is to have a headset that follows this UVAD tactic (for example, with left and right-hand pilot microphones or left and right-hand feedback microphones, or generally speaking with two microphones that are set up to approximate are directly spaced from the vocal tract), on a conventional head and torso simulator ("Head and Torso Simulator", HATS) such as the B&K Type 4128 manufactured by Brüel & Kjaer Sound & Vibration Measurement A / S in Naerum, Denmark -C simulator. The headsets can then be observed as they automatically mute in response to noise sources approximately equally spaced from the microphones, such as directly in front of, behind, or above the HATS.

If FB_left/FF_left < DT then Mute; else Unmute,

wobei FB_left/FF_left das Verhältnis der Energie im Ausgabesignal des linken Rückkopplungsmikrofons (FBA) zu der Energie im Ausgabesignal des linken Vorsteuerungsmikrofons (FFA) bezeichnet, und DT einen ausgewählten Erfassungsschwellwert für die Sprachäußerungen des Nutzers bezeichnet. In dieser Ausführungsform ist DT plattformunabhängig; allerdings wird im Allgemeinen, wenn der Nutzer spricht, das Verhältnis gegenüber der Abwesenheit von Sprache merklich ansteigen. (Das Verhältnis ist ein Übertragungsverhältnis, und in diesem Fall eine reelle Zahl.)In order to suppress on-axis ambient noise sources, such as utterances by other than the user, and to enable better performance in noisy environments, some embodiments use a second method for user voice activity detection, which has the additional advantage that it is also suitable for systems with a single ear piece is useful, or generally speaking for any situation in which two microphones have an asymmetrical coupling to the user and the ambient noise. This method takes advantage of the insight that the amount of energy that arrives at the feedback microphones (generally speaking, the microphone with the better acoustic coupling to the vocal tract of the user) is considerably greater than that which arrives at the pilot microphones when the User speaks due to the asymmetry in the acoustic paths of the voice signals to the feedback and pilot microphones. The feedback microphones are placed in the front cavity of the earpieces in or near the user's ear canal. With this placement, the feedback microphones receive energy from the user's voice via bone conduction paths 104A and 104B with very low damping and ambient noise via air line with strong damping due to the seal that is usually present in ANR systems. Therefore, the ratio of the output signals of the pilot microphones to the feedback microphones (FB / FF energy ratio) in the presence of a user's voice is much higher than that of Ambient noises without speech, regardless of the direction of the surrounding noises. To do this, some embodiments determine user voice activity according to the following logic:

 If FB_left / FF_left <DT then Mute; else displeasure, 

where FB_left / FF_left denotes the ratio of the energy in the output signal of the left feedback microphone (FBA) to the energy in the output signal of the left feedforward microphone (FFA), and DT denotes a selected detection threshold for the speech utterances of the user. In this embodiment, DT is platform independent; however, generally, when the user speaks, the ratio to the absence of language will increase markedly. (The ratio is a transfer ratio, and in this case a real number.)

In particular, some embodiments can use a complex valued transmission ratio that includes both amplitude and phase information, and thereby offer better performance. When a threshold value is taken into account in this complex case, the threshold value is no longer a scalar, but rather a line (perhaps curved or composed in sections) that intersects the complex level. For example, a line for values of the imaginary part greater than 0 could represent a threshold. Or the threshold could be that positive real parts and imaginary parts identify utterances, dividing a quadrant of the complex level. Note that some embodiments can use the right-hand side feedback and feedforward microphones and not the left-hand side. Other embodiments can automatically determine whether the right or left earpiece should be used. If, for example, the left ear piece is removed, an on-ear detection (OED) circuit detects that only the right ear piece is on and operates the voice recording module depending on the right-hand feedback and feedforward microphones.

ausgewertet werden, wobei Sxy die Interkorrelationsspektraldichte zwischen x und y, und S_xx die Leistungsspektraldichte von x ist. Damit kann Sxy über FFTs gemäß dem folgenden Schema bestimmt werden: $${\text{S}}_{\text{xy\_hat}}=\text{sum}\left(\text{fft}\left(\text{y}\right)*\text{conjugate}\left(\text{fft}\left(\text{x}\right)\right)/\text{sum}(\text{fft}\left(\text{x}\right)*\text{conj}(\text{fft}\left(\text{x}\right)*\text{conj}\left(\text{fft}(\text{x}\right)\right).$$

Oder, falls nur eine FFT genutzt wird (ohne Mittelwertbildung), wird T_xy definiert als $${\text{T}}_{\text{xy}}=\text{fft}\left(\text{y}\right)/\text{fft}\left(\text{x}\right)$$

Für das Verhältnis von FB-Energie (Rückkopplungsmikrofon) zu FF-Energie (Vorsteuerungsmikrofon) ist x das linke Vorsteuerungsmikrofonsignal ffL (FFA) und y das linke Rückkopplungsmikrofonsignal fbL (FBA). Das Verhältnis von FB-Energie zu FF-energie beträgt tatsächlich |Tff2fb|^∧2.A complex transfer function (TF) can be used to determine the energy levels in the output signals of the feedback and feedforward microphones. T _xy according to $${\text{T}}_{\text{xy}}={\text{S}}_{\text{xy}}/{\text{S}}_{\text{xx}}.$$

are evaluated, where Sxy is the intercorrelation spectral density between x and y, and S _{xx is} the power spectral density of x. Sxy can thus be determined using FFTs according to the following scheme: $${\text{S}}_{\text{xy\_hat}}=\text{sum}\left(\text{fft}\left(\text{y}\right)*\text{conjugate}\left(\text{fft}\left(\text{x}\right)\right)/\text{sum}(\text{fft}\left(\text{x}\right)*\text{conj}(\text{fft}\left(\text{x}\right)*\text{conj}\left(\text{fft}(\text{x}\right)\right),$$

Or, if only one FFT is used (without averaging), T _xy defined as $${\text{T}}_{\text{xy}}=\text{fft}\left(\text{y}\right)/\text{fft}\left(\text{x}\right)$$

For the ratio of FB energy (feedback microphone) to FF energy (pilot microphone), x is the left pilot microphone signal ffL (FFA) and y is the left feedback microphone signal fbL (FBA). The ratio of FB energy to FF energy is actually | Tff2fb | ^∧ 2.

This means that in embodiments that use FFTs as the core, the ratio of the squared absolute value of the FFT result can be related to a threshold value. Or, if one uses an implementation without FFTs, a moving average of the sampled time signals could simply be calculated after being passed through a bandpass filter to achieve similar results. Using the complex transfer function estimation (Txy ..) one can take advantage of the meaningful phase information.

The detection threshold DT is generally determined depending on the physical design of the ear pieces and the quality of the encapsulation that they provide for the front cavity volume. With good ANC headphones, attenuation of 20dB is expected if the user does not speak. Because of the bone conduction mechanism, this increases to around 0dB when the user speaks.

However, the degree of change is more pronounced for in-ear headphones, less pronounced for on-ear headphones, and least pronounced for circumaural headphones, which means that different threshold values are used for each type of headset. For example, it is to be expected with ITE headphones at certain frequencies, such as 100 Hz, that about 20dB to 30dB more sound pressure occurs in the covered ear (FB microphone) than outside (FF microphone) due to the user's utterance. This effect can be exploited in circumaural headphones, but the difference in in-ear headphones in terms of the sound pressure of utterances at 100 Hz compared to the external sound pressure at the pilot microphone may be only a few decibels.

Some embodiments can also vary the threshold value dynamically, depending on measured parameters such as the ambient noise or average signal-to-noise ratios, or alternatively to the user an adaptation mechanism via, for example, the device 140 Provide hosted application. Some embodiments use a dynamic detection threshold because the FB to FF ratio is a function of the total attenuation within the ANC headset and can therefore vary over a fairly wide range. In one embodiment, the detection threshold is estimated during the two-year intelligent mute period, particularly as a function of the moving average of energy for muting and the moving average of energy for unmuting. Some embodiments measure the system's active and passive damping so that the threshold is ultimately a function of active damping.

For TF estimation purposes, some embodiments update the mean under certain conditions. Some embodiments also recognize that the feedback microphone signal has several components, namely audio input signal Rx (from the device 140 ), Ambient noise, user utterances and measured noise. This ensures good signal levels that do not correlate with the ambient noise. Alternatively, for purposes of estimating the noise transfer function, some embodiments may update the mean when the FB to FF energy ratio is low or highly correlated to ensure that the TF estimates converge faster than they would otherwise.

This second approach for user voice activity detection (UVAD), which is based on the complex transmission characteristics within a certain frequency range, is assumed to be particularly stable if it is used with any two microphones that have a transmission characteristic with the following constraints exhibit. The first constraint is that the transmission characteristics of speech utterances change significantly compared to interference. The second constraint is that the transfer characteristic remains relatively meaningful (i.e., relatively unambiguous) when there are changes in the relative direction of the interference. These boundary conditions can be met with the feedforward and feedback microphones in an in-ear headset (and in other situations in which one microphone is more connected to the user's vocal tract than the other).

The output signal for ambient sound pressure on the ambient microphone responds first to ambient noise and has the leading phase in relation to the pilot microphone coupled to the ears for two reasons. On the one hand, it is directly coupled to the environment and usually closer to the noise source in terms of the acoustic path length. On the other hand, a common earpiece of a headset has a certain passive attenuation, so that it acts more or less like a kind of low-pass filter, i.e. it makes no difference at very low frequencies, but the higher the frequency attenuates the microphone (FB microphone), which is coupled to the ears. All causal low pass filters induce a phase shift, and all physical filters are causal.

In addition to the acoustic path from the mouth of the user to the ear and from there to the microphone coupling to the ears, there is a further path from the vocal cords through the body for a user's utterances. The speed of sound through the body is considerably higher, about four to five times higher, or even under one millisecond compared to about 3 to 4.5 milliseconds. (The speed of sound in air is about 340 meters per second; roughly 1500 meters per second in muscle meat; and over 4000 meters per second in skull and bones). As a result, the sound from a user's speech uttered through the user's body arrives much earlier than the acoustic signal transmitted through the mouth and through the air.

When looking at transmission characteristics between a microphone coupling to the environment and a microphone coupling to the ears at close range, the ambient microphone is opposite to the Always guide the ear-coupling microphones, regardless of the direction of arrival. When a user uttered speech, the microphone coupling the ears will always lead opposite the microphone coupling to the surroundings. It is therefore clear to see that the asymmetry in the complex transfer function (or any other basis) and the largely insensitivity to a direction of the interference is much more suitable for microphones coupling to the surroundings and to the ears in the same ear.

In order to make the best possible use of the bone conduction path in the microphone coupling to the ears, here called the feedback microphone FB, some embodiments take into account incoming audio signals “Rx” from an external device, such as the device 140 , (In some embodiments, Rx may be mixed with one or more internally generated notification signals, such as beeps, system event tones, message tones, etc.) These incoming audio signals will typically have characteristics that closely resemble bone-guided utterances, meaning that they have a much stronger FB -Microphone amplitude than the amplitude of the surrounding microphone (pilot microphone, FF) and thus can favor false detections of user utterances.

wobei:

• fb_Rx das durch das Rx-Signal verursachte FB-Mikrofonsignal ist, welches von dem an die Ohren ankoppelnden Lautsprecher abgespielt wird;
• fb_ambient das durch das Umgebungsrauschen verursachte FB-Mikrofonsignal ist; und
• fb_{speech_BC} das durch Knochenleitung verursachte FB-Mikrofonsignal ist.

One approach to addressing this problem is to use an Rx suppressor that mathematically clears or eliminates the effect of the Rx signal on the UVAD calculations. The exemplary embodiment uses an Rx suppressor by decomposing the FB signal as follows: $$\text{fb}={\text{fb}}_{\text{Rx}}+{\text{fb}}_{\text{ambient}}+{\text{fb}}_{\text{speech\_BC}}.$$

in which:

• fb _{Rx is} the FB microphone signal caused by the Rx signal, which is played by the loudspeaker coupled to the ears;
• fb _{ambient is} the FB microphone _signal caused by the ambient noise; and
• fb _{speech_BC is} the FB microphone _signal caused by bone conduction.

$${\text{fb}}_{\text{ambient}}=\text{ff}*{\text{T}}_{\text{ff2fb}};$$

wobei T_rx2tb die Übertragungsfunktion von Rx an das FB-Mikrofon ist, bei dem alle anderen Eingaben Null sind, und T_ff2fb die Übertragungsfunktion von dem FF-Mikrofon an das FB-Mikrofon ist, ohne Einfluss von Sprachäußerungen oder Rx und nur durch Rauschen induziert. T_rx2fb und T_ff2fb können mit verschiedenen Verfahren geschätzt werden. Beispielsweise können einige Ausführungsformen einen Kalmanfilter oder eine herkömmliche Schätzung in Abhängigkeit der Autokorrelations- und der Interkorrelationsspektren nutzen, unter Berücksichtigung, dass beispielsweise die Mittelwerte für Tff2fb, wenn Rx vorhanden ist, nicht aktualisiert werden. Es muss auch dafür Sorge getragen werden, nicht die Schätzungen zu aktualisieren, wenn Sprachäußerungen des Nutzers getätigt werden, aber dies ist ein viel entspannteres Problem, da der UVAD für diesen Schritt nicht alle Sprachäußerungen aufnehmen muss, sondern lediglich mit hoher Konfidenz annehmen muss, dass Sprachäußerungen nicht getätigt worden sind.You can still define fb _Rx and fb _ambient as follows: $${\text{fb}}_{\text{RX}}=\text{Rx}*{\text{T}}_{\text{rx2fb}};$$

$${\text{fb}}_{\text{ambient}}=\text{ff}*{\text{T}}_{\text{ff2fb}};$$

where T _{rx2tb is} the transfer function from Rx to the FB microphone, for which all other inputs are zero, and T _{ff2fb is} the transfer function from the FF microphone to the FB microphone, without the influence of speech or Rx and only induced by noise , T _rx2fb and T _ff2fb can be estimated using different methods. For example, some embodiments may use a Kalman filter or a conventional estimate depending on the autocorrelation and intercorrelation spectra, considering that, for example, the mean values for Tff2fb, if Rx is present, are not updated. Care must also be taken not to update the estimates when the user's speech is uttered, but this is a much more relaxed problem since the UVAD does not have to record all the utterances for this step, but only has to assume with high confidence that Speeches have not been made.

Once the estimates for these primarily static transfer functions are available, they can be used to estimate the fb _{speech_BC} signal in near real time. The Trx2fb and Tff2fb estimates would be averaged over time. The exemplary embodiment uses Fast Fourier Transforms (FFTs) to compute the estimates, but in some embodiments, any base that covers the subspace containing the bone conduction information well may be used.

oder als $${\text{fb}}_{\text{speech\_BC}}=\text{fb}-\text{Rx}*{\text{T}}_{\text{rx2fb}}-\text{ff}*{\text{T}}_{\text{ff2fb}}$$

wobei fb^∧ _{speech_BC} die Schätzung von fb_{speech_BC} ist.If you know Trx2fb and Tff2fb, you can express fb _{speech_BC} as: $${\text{fb}}_{\text{speech\_BC}}=\text{fb}-{\text{fb}}_{\text{Rx}}-{\text{fb}}_{\text{ambient}}$$

or as $${\text{fb}}_{\text{speech\_BC}}=\text{fb}-\text{Rx}*{\text{T}}_{\text{rx2fb}}-\text{ff}*{\text{T}}_{\text{ff2fb}}$$

where fb ^∧ _{speech_BC is} the estimate of fb _{speech_BC} .

Therefore, the user speech detection is carried out depending on an estimated signal, which is mainly interference-free from ambient noise and incoming audio signals.

Remarkably, this version of the asymmetrical approach (using the feedback and feedforward microphones on the same side) relies on the bone conduction path between the user's vocal tract and the feedback microphone. Therefore, placing a headset using this asymmetrical approach on a conventional HATS (such as the B&K 4128-C simulator mentioned above) would generally prevent the UVAD from operating properly since conventional HATS have no bone conduction pathway. In other words, depending on a user voice signal input to the HATS in the corresponding frequency range, the headset put on the HATS would not be able to appropriately activate and deactivate the muting (or otherwise correctly detect the user voice activity). , (A path from the vocal tract to the feedback microphone would exist due to vibrations, but this path would be very weakly coupled compared to real bone conduction.)

if (coherence_of_left_and_right < CT), then Mute;

 else {if ((fb_left/ff_left) < ET, Mute; else Unmute}

Bemerkenswerterweise zieht eine Implementierung dieser Erfassungslogik die Nutzung dreier Steuermikrofone, nämlich der links- und rechtsseitigen Rückkopplungsfehlermikrofone sowie einem der Vorsteuerungsmikrofone nach sich. Außerdem erlaubt diese Logik nur den asymmetrischen Schwellwertvergleich (das Verhältnis von Rückkopplungsmikrofonenergie zu Vorsteuerungsmikrofonenergie), um eine Aufhebung der Stummschaltung zu steuern. Andere Ausführungsformen könnten beide Vergleiche zum Auslösen der Aufhebung der Stummschaltung erlauben.Some embodiments combine the threshold comparisons based on symmetry and asymmetry as follows:

 if (coherence_of_left_and_right <CT), then mute;

 else {if ((fb_left / ff_left) <ET, Mute; else Unmute} 

Remarkably, implementation of this detection logic entails the use of three control microphones, namely the left and right side feedback error microphones and one of the pilot microphones. In addition, this logic only allows the asymmetric threshold comparison (the ratio of feedback microphone energy to pilot microphone energy) to control unmute cancellation. Other embodiments could allow both comparisons to trigger unmute.

In addition, some unmute buffers create buffers within the processing circuit 130 to perform voice recording in a delayed version of the relevant signal data. More specifically, some embodiments use a look ahead buffer for X milliseconds to give the detector enough time to detect the presence of a user's speech in the headset and to unmute the system without loss of speech. For example, one embodiment stores 20 milliseconds of sampled data from the system microphones, allowing the detector to capture speech at sample n and then mute all previous samples taken in the previous 20 millisecond period, thereby avoiding that the first part of a user statement is cut off or hidden. In some embodiments, the length of the look-ahead period can be set or calibrated by the user, and in other embodiments it can be dynamically determined depending on a sensed speech cadence of the user, such as a running average of the intervals between peaks in the speech signal.

The mute module 136 sets the mute function depending on the command signals of the user voice recording module 135 ready. In some embodiments, this results in switching off the signal paths connected to the speech microphone and / or to one or more of the other microphones in the arrangement. However, in order to improve the user's listening experience, some embodiments delay the activation or use of mute for 3, 5, 10, 15 or 20 milliseconds, thereby giving a margin of safety to avoid that the ends of utterances are cut off or between Words is muted. In some embodiments, the time period for this delay can be determined dynamically depending on a measured speech rate of the user. In addition, in some embodiments, a visual, auditory, or haptic display may be provided in response to the activation or deactivation of the mute function to inform the user of the change in the mute state. In some embodiments, one or more of these indicators can be on the headset itself and / or on the device 140 to be provided. In some variants, the visual display can take the form of an illuminated and / or flashing LED (light-emitting diode) on the headset and / or an illuminated or flashing or color or shape-changing symbol of a microphone on the device display. In some embodiments, the user may override the mute function via a manual control on the headset and / or device for a predetermined period of time, for example for 1, 2, or 3 minutes, or until the ongoing communication session, such as a phone call, is ended. At the end of the override period, the automatic activation and deactivation of the mute would continue.

If start mute {
            static float decayCoef = 1;
            if (decayCoef > 0.00316) decayCoef *= 0.995 // -25dB
            Scale the output with decayCoef }

Generally speaking, some embodiments step, deliberately slow or fade the state changes between mute activation and mute activation (and vice versa) to avoid very rapid gain changes through the microphone that would otherwise produce audible clicks or clicks, which are annoying and one suggest poor audio component quality. This is generally remedied by making the gain changes gradually rather than instantaneously. For example, in one embodiment, the "attack" from on to off can occur for about 100 milliseconds, which is slow enough to avoid cracking and long enough to minimize the memory requirements on the look-ahead buffer. One embodiment uses a decay function of the following expression:

 If start mute {
            static float decayCoef = 1;
            if (decayCoef> 0.00316) decayCoef * = 0.995 // -25dB
            Scale the output with decayCoef} 

Some embodiments facilitate more efficient automatic mute by sensing when a system with two earpieces properly uses only one earpiece. Some of these embodiments rely on an on-ear detector (OED) to optimize performance. Details of the OED will be continued in the U.S. Patent Application 14 / 850,859 co-owned, filed on September 10, 2015, the disclosure of which is hereby incorporated by reference in its entirety.

The language-dependent module 137 represents one or more functions (the processing circuit 130 and / or the device 140 ) which is a binary signal for the presence or absence of speech from the voice activity module 135 receive. Some of these modules use the signal to activate or deactivate the function of the module in order to save computing power, memory and / or battery life. For example, in some embodiments, the language dependent module 137 have a speech or keyword recognition module which is designed to listen to certain keyword commands or to carry out generalized speech recognition functions.

In some other embodiments, the module comprises 137 also a noise reduction module, which carries out further processing stages in order to reduce noise in the voice microphone signal. In some embodiments, this noise reduction module can be tailored to the specific environment of the user. And in other embodiments, the language dependent module 137 have a sub-tone module or sub-tone circuit that receives the voice microphone output signals and generates a three to ten percent sub-tone signal for the user on one or both earpieces. The generation of secondary tones consumes energy, so switching off this function if the user does not speak extends the battery life. See preliminary U.S. Patent Application 62 / 530,049 , filed on July 7, 2017, which is incorporated herein by reference.

2 shows a flow chart 200 an advanced automatic mute system for an ANR headset with two earpieces. The flow chart 200 includes process blocks 210 to 280 ,

block 210 involves performing an OED (on-ear detection) to determine the status of the earpieces. (See also pending U.S. Patent Application 14 / 850,859 , filed on September 10, 2015, which is incorporated herein by reference.) Execution is at block 220 where it is determined whether or not the ambient noise level is low. If the ambient noise is low, the module's intelligent mute function 134 at block 230 disabled and execution returns to block 220 back; otherwise execution at block 240 continued.

block 240 includes determining whether the user has put on both ear pieces. In this exemplary embodiment, this includes a further call to the OED module. If both earphones the execution branches to block 250 by determining whether or not to mute using the symmetry-based or combined symmetry-asymmetry mute functions described above, both of which require signals from both earpieces. Execution returns to block from here 240 to determine if both earpieces are still in place. If both earpieces are found not to be in place, the version moves with a block 260 in which it is determined whether an earpiece is attached. (It is also determined which of the earpieces is attached.) If one is attached, the execution branches to block 270 to perform an intelligent mute function depending on the asymmetrical threshold comparisons as described above. Execution then returns to block 260 back to determine if an earpiece is still on. (In some embodiments, the earpiece may still be on, but there may not be enough battery power left.) If no more earpiece is on, intelligent mute is no longer performed and execution returns to block 220 ,

3 shows a one-ear system model 300 of the system 100 , including the following blocks: T _p is the passive damping transfer function; T _dm is the driver-to-feedback microphone transfer function; H _ff is the feedforward filter; H _col is the feedback filter; V is the acoustic user utterance-to-pilot microphone path (transfer function); W is the user speech utterance to pilot microphone bone conduction path (transfer function). The model also includes the following signals: s is the user speech utterance signal; a is the ambient noise signal; n is the measurement of the pilot microphone (or more generally, the microphone that is furthest from or least acoustically coupled to the vocal tract); m is the measurement of the feedback microphone (or generally speaking, the microphone that is furthest away from or most acoustically coupled to the vocal tract); u is the RX signal and / or any system audio notification signal; and d is the DAC (driver) output.

$$D=H_{fb}M+H_{ff}N+U$$

$$N=A+\begin{array}{cc}& VS\end{array}$$

Substitution von D in M ergibt $$M=T_{dm}\left(H_{fb}M+H_{ff}N+U\right)+T_{p}N+WS$$

Aufsammeln der Terme ergibt $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)N+T_{dm}U+WS$$

Substitution von N ergibt $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)\left(A+VS\right)+T_{dm}U+WS$$

Aufsammeln der Terme ergibt $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)A+T_{dm}U+\left(\left(T_{dm}{H}_{ff}+T_p\right)V+w\right)S$$

$$M=F_{1}A+F_{2}U+\left(F_{1}V+Fs\right)S$$

wobei $$F_1=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}$$

$$F_2=\frac{T_m}{1-T_{dm}{H}_{fb}}$$

$$F_s=\frac{W}{1-T_{dm}{H}_{fb}}$$

In particular, the system of 3 both feedforward and feedback filters Hff and Hfb. However, some embodiments dispense with these filters, which means that (H _ff = H _fb = O) and the headset is passive. The exemplary embodiment follows the linear statistical model below: $$M=T_{dm}D+T_{p}N+W$$

$$D=H_{fb}M+H_{ff}N+U$$

$$N=A+\begin{array}{cc}& VS\end{array}$$

Substitution of D in M results $$M=T_{dm}\left(H_{fb}M+H_{ff}N+U\right)+T_{p}N+WS$$

Collecting the terms gives $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)N+T_{dm}U+WS$$

Substitution of N results $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)\left(A+VS\right)+T_{dm}U+WS$$

Collecting the terms gives $$\left(1-T_{dm}{H}_{fb}\right)M=\left(T_{dm}{H}_{ff}+T_p\right)A+T_{dm}U+\left(\left(T_{dm}{H}_{ff}+T_p\right)V+w\right)S$$

$$M=F_{1}A+F_{2}U+\left(F_{1}V+Fs\right)S$$

in which $${\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">F</figure-callout>}}_1=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}$$

$$F_2=\frac{T_m}{1-T_{dm}{H}_{fb}}$$

$$F_s=\frac{W}{1-T_{dm}{H}_{fb}}$$

The goal of the linear model is to break down the measurement M of the feedback microphone into a linear combination of independent components (ambient noise, Rx and user utterances). This model can be applied to narrowband signals, i.e. for a specific frequency slot. To model a broadband system, one would add frequency indices for each term.

$$E\left[SS\ast \right]={\sigma }_s^2$$

$$E\left[NN*\right]=E\left[\left(A+VS\right)\left(A+VS\right)*\right]$$

$$E\left[NN*\right]={\sigma }_A^2+VV*{\sigma }_s^2$$

The variances can be expressed as follows: $$e\left[AA*\right]={\sigma }_A^2$$

$$e\left[SS*\right]={\sigma }_s^2$$

$$e\left[NN*\right]=e\left[\left(A+VS\right)\left(A+VS\right)*\right]$$

$$e\left[NN*\right]={\sigma }_A^2+VV*{\sigma }_s^2$$

da Umgebungsgeräusche und Sprachäußerungen nicht statisch sind. Die Zeitindizes werden der Einfachheit der Notierung halber weggelassen. Die Kovarianz E[MN*] definiert sich zu $$E\left[MN*\right]=E\left[\left(F_{1}A+F_{2}U+\left({\text{F}}_{1}V+Fs\right)S\right)\left(A+VS\right)*\right]$$

$$E\left[MN*\right]=F_1{\sigma }_A^2+\left(F_{1}V{V}^{*}+F_s{V}^{*}\right){\sigma }_S^2$$

was umgeschrieben werden kann als $$E\left[MN*\right]=F_1\left({\sigma }_A^2+VV*{\sigma }_s^2\right)+F_s{V}^{-1}VV*{\sigma }_s^2$$

Man beachte, dass die Berechnung der Kovarianz bei jeder Frequenz das Korrelationsleistungsspektrum ergeben würde.Again, this applies to the narrow-band case. Calculating the variances at each frequency would result in the power spectrum of A, S and N. These variances are instantaneous values: ${\sigma }_A^2\left(t\right).{\sigma }_s^2\left(t\right).$

because ambient noise and utterances are not static. The time indices are omitted for the sake of simplicity of notation. The covariance E [MN *] defines itself $$e\left[MN*\right]=e\left[\left(F_{1}A+F_{2}U+\left({\text{F}}_{1}V+Fs\right)S\right)\left(A+VS\right)*\right]$$

$$e\left[MN*\right]={\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">F</figure-callout>}}_1{\sigma }_A^2+\left(F_{1}V{V}^{*}+F_s{V}^{*}\right){\sigma }_S^2$$

what can be rewritten as $$e\left[MN*\right]=F_1\left({\sigma }_A^2+VV*{\sigma }_s^2\right)+F_s{V}^{-1}VV*{\sigma }_s^2$$

Note that calculating the covariance at any frequency would result in the correlation power spectrum.

$$G=F_1+F_2{V}^{-1}\left(\frac{b}{1+b}\right)$$

wobei $b=\frac{VV*{\sigma }_s^2}{{\sigma }_A^2}$

die Nutzersprachäußerungs-zu-Umgebungsrauschen-SNR ist. Substitution von F₁ und F_s in G ergibt $$G=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}+\frac{W{V}^{-1}}{1-T_{dm}{H}_{fb}}\left(\frac{b}{1+b}\right)$$

Falls Nutzersprachäußerungen vorhanden sind, konvergiert das Verhältnis $\left(\frac{b}{1+b}\right)$

gegen Eins (da das Nutzersprachäußerungs-zu-Umgebungsrauschen-SNR groß wird). Falls keine Nutzersprachäußerungen vorhanden sind, ist $\left(\frac{b}{1+b}\right)$

Null. Das heißt, dass der instantane Regressionskoeffizient, G(t), auf einem Liniensegment mit Endpunkten F₁ und F₁ + F_sV^-1 lebt. Man beachte, dass das Berechnen des Regressionskoeffizienten bei jeder Frequenz die Übertragungsfunktion ergeben würde.The regression coefficient G is defined as $$G=\frac{e\left[MN*\right]}{e\left[NN*\right]}=\frac{F_1\left({\sigma }_A^2+VV*{\sigma }_s^2\right)+F_s{V}^{-1}VV*{\sigma }_s^2}{{\sigma }_A^2+VV*{\sigma }_s^2}$$

$$G={\mathrm{<figure-callout\; id="1"\; label="F"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">F</figure-callout>}}_1+F_2{V}^{-1}\left(\frac{\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">b</figure-callout>}}{1+b}\right)$$

in which $b=\frac{VV*{\sigma }_s^2}{{\sigma }_A^2}$

is the user speech utterance to ambient noise SNR. Substitution of F ₁ and F _p in G. $$G=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}+\frac{W{V}^{-1}}{1-T_{dm}{H}_{fb}}\left(\frac{\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">b</figure-callout>}}{1+b}\right)$$

If there are user voices, the relationship converges $\left(\frac{\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">b</figure-callout>}}{1+b}\right)$

against one (since the user speech to ambient noise SNR becomes large). If there are no user utterances, is $\left(\frac{\mathrm{<figure-callout\; id="1"\; label="b"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">b</figure-callout>}}{1+b}\right)$

Zero. This means that the instantaneous regression coefficient, G (t), lives on a line segment with end points F ₁ and F ₁ + F _s V ^-1 . Note that calculating the regression coefficient at any frequency would give the transfer function.

wobei r(t) eine Gauß'sche Zufallsvariable mit Mittelwert Null und Einheitsvarianz ist, σ_r ein abstimmbarer Parameter ist, der nicht modelliertes Systemverhalten erklärt (d.h. geringfügige Nichtlinearitäten in den Messungen). H(t) ist ein Regressionskoeffizient, der den Beitrag des Rx-/Musiksignals zur Messung des Rückkopplungsmikrofons erklärt.Estimation: Tracking the regression coefficient, G (t), in its change behavior over time can be a challenge. Even more stupid is the fact that when Rx is present, the coherence between M and N is reduced, which increases the variance of the estimate. Exploiting a relationship to U simplifies the estimation problem, but is not necessary. Some embodiments formulate the estimation problem in a state space reference model using the measurement model M (t), which is defined as follows $$M\left(t\right)=G\left(t\right)N\left(t\right)+H\left(t\right)U\left(t\right)+\sqrt{{\sigma }_r}r\left(t\right)$$

where r (t) is a Gaussian random variable with mean zero and unit variance, σ _{r is} a tunable parameter that explains non-modeled system behavior (ie minor non-linearities in the measurements). H (t) is a regression coefficient that explains the contribution of the Rx / music signal to the measurement of the feedback microphone.

wobei q₁(t) und q₂(t) unabhängige Gauß'sche Zufallsvariablen mit Mittelwert Null und Einheitsvarianz sind. α₁ und α₂ sind abstimmbare Parameter, die bestimmen, wie schnell G(t) und H(t) sich mit der Zeit ändern können.Some embodiments use the following process model: $$\left[\begin{array}{c}G\left(t+1\right)\\ H\left(t+1\right)\end{array}\right]=\left[\begin{array}{cc}{\mathrm{<figure-callout\; id="1"\; label="\alpha "\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_1& 0\\ 0& {\alpha }_2\end{array}\right]\left[\begin{array}{c}G\left(t\right)\\ H\left(t\right)\end{array}\right]+\left[\begin{array}{cc}{\left(1-{\mathrm{<figure-callout\; id="1"\; label="\alpha "\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_1^2\right)}^{1/2}& 0\\ 0& {\left(1-{\alpha }_2^2\right)}^{1/2}\end{array}\right]\left[\begin{array}{c}{q}_1\left(t\right)\\ q_2\left(t\right)\end{array}\right]$$

where q ₁ (t) and q ₂ (t) are independent Gaussian random variables with mean zero and unit variance. α ₁ and α ₂ are tunable parameters that determine how quickly G (t) and H (t) can change over time.

The state space reference model is useful because there are efficient state estimation algorithms (e.g. Recursive Least Squares (RLS), Least Mean Squares (LMS) and Kalman Filters). Some embodiments estimate the states G (t) and H (t) in multiple frequency slots by using a Kalman filter for each slot.

wobei T die Matrix der Übertragungsfunktionen bezeichnet und definiert ist als $$T=\left[\begin{array}{ccc}{T}_{s\theta }& T_{\alpha \theta }& T_{u\theta }\\ T_{s\varphi }& T_{\alpha \varphi }& T_{u\varphi }\end{array}\right]$$

wobei T_sθ eine Übertragungsfunktion von Nutzersprachäußerungen s auf das Signal θ darstellt; T_aθ eine Übertragungsfunktion von Umgebungsgeräuschen auf das Signal θ darstellt; T_sΦ eine Übertragungsfunktion von Nutzersprachäußerungen s auf das Signal Φ darstellt; und T_aϕeine Übertragungsfunktion von Umgebungsgeräuschen auf das Signal Φ darstellt. Mit diesem Modell erfordert eine verlässliche Detektion von Nutzerstimmaktivität eine Befolgung der folgenden asymmetrischen Randbedingung: $${\left|\frac{T_{s\theta }}{T_{a\theta }}-\frac{T_{s\phi }}{T_{a\phi }}\right|}^2>z$$

wobei z den Asymmetrieschwellwert z bezeichnet, was angibt, dass das Verhältnis der Empfindlichkeit von Sprachäußerungen-zu-Umgebung in Amplitude und/oder Phase unterschiedlich sein müssen. In einigen Ausführungsformen beträgt z Null und in anderen Ausführungsformen beträgt z 0,01, 0,1, 0,5, 0,75, 1,0, 2,0. In weiteren Ausführungsformen ist z größer als 2.Some embodiments involve a somewhat more general approach to user voice activity detection that avoids the need to explicitly characterize the system's electroacoustic parameters. The generalized approach uses two signals θ (theta) and ϕ (phi), each of which is a function of at least two of the variables user speech expression s, ambient noise a and / or incoming audio signal u, each according to the matrix of the transfer functions T. (incoming audio signals u may be a mixture of externally generated audio signals such as Rx / music from a communication device and internally generated audio signals such as system event notifications or alarm signals) The matrix of the transfer functions T is determined by how speech, environment and incoming audio signals u appear on the two or more transmitters and how the transmissions are combined with a reference to the Rx / music to produce phi and theta , Mathematically speaking, this can be expressed as $$\left[\begin{array}{c}\theta \\ \phi \end{array}\right]=T\left[\begin{array}{c}s\\ a\\ u\end{array}\right]$$

where T is the matrix of the transfer functions and is defined as $$T=\left[\begin{array}{ccc}{T}_{s\theta }& T_{\alpha \theta }& T_{u\theta }\\ T_{s\phi }& T_{\alpha \phi }& T_{u\phi }\end{array}\right]$$

where T _{sθ represents} a transfer function of user utterances s to the signal θ; T _{aθ represents} a transfer function of ambient noise on the signal θ; T _{sΦ represents} a transfer function of user utterances s to the signal Φ; and T _{aϕ represents} a transfer function of ambient noise onto the signal Φ. With this model, reliable detection of user voice activity requires compliance with the following asymmetric constraint: $${\left|\frac{T_{s\theta }}{T_{a\theta }}-\frac{T_{s\phi }}{T_{a\phi }}\right|}^2>z$$

where z denotes the asymmetry threshold value z, which indicates that the ratio of the sensitivity of utterances to environment must be different in amplitude and / or phase. In some embodiments, z is zero and in other embodiments, z is 0.01, 0.1, 0.5, 0.75, 1.0, 2.0. In further embodiments, z is greater than 2.

wobei $$T_{s\theta }=\frac{W}{1-T_{dm}{H}_{fb}},T_{a\theta }=V,T_{a9}=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}},T_{a\varphi }=\frac{T_{dm}}{1-T_{dm}{H}_{fb}},T_{u\varphi }=0.$$

Mapping the terms of this generalized model to the specific electroacoustic implementation of 3 gives T as the corresponding matrix $$T=\left[\begin{array}{ccc}\frac{W}{1-T_{dm}{H}_{fb}}& \frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}& \frac{T_{dm}}{1-T_{dm}{H}_{\mathrm{<figure-callout\; id="1"\; label="f\; b\; V"\; filenames="DE112018000717T5\_0047.png,DE112018000717T5\_0048.png"\; state="\{\{state\}\}">f\; </figure-callout>}b}}& \\ V& 1& 0\end{array}\right]$$

in which $$T_{s\theta }=\frac{W}{1-T_{dm}{H}_{fb}}.T_{a\theta }=V.T_{a9}=\frac{T_{dm}{H}_{ff}+T_p}{1-T_{dm}{H}_{fb}}.T_{a\phi }=\frac{T_{dm}}{1-T_{dm}{H}_{fb}}.T_{u\phi }=\mathrm{0th}$$

die erfordert, dass das Verhältnis von Sprachäußerungen zur Umgebungssignalstärke im Signal θ größer als das Verhältnis von Sprachäußerungen zur Umgebungssignalstärke im Signal Φ ist.Some embodiments can use an alternative asymmetric constraint of the following form $${\left|\frac{Ts\theta }{Ta\theta }\right|}^2>{\left|\frac{Ts\phi }{Ta\phi }\right|}^2$$

which requires that the ratio of utterances to the ambient signal strength in the signal θ is greater than the ratio of utterances to the ambient signal strength in the signal Φ.

4 shows a block diagram of a general electroacoustic system 400 , which represents the generation of the signal θ from a linear combination of microphone input signals and Rx / music input signals (incoming audio signals). The system 400 comprises a set of measurements or input signals M ₁ ... M _N from microphones or transmitters and Rx / music signal reference values U1 and U2 , the above Corresponding amplification modules or blocks K ₁ -K _N are output to filter H with corresponding transfer functions H ₁ -H _N. The filter outputs are fed into a summer which generates theta. Therefore the signal θ is a filtered combination of the transmitter and reference input signals.

5 shows a block diagram of a generalized UVAD module 500 , which is part of the UVAD module 135 within the process of 2 can be used. In addition to the input signals user speech s, ambient noise a and incoming audio signals a, the module includes 500 a block 510 for an electro-acoustic system model, an estimator block 520 , a totalizer 530 , a variance ratio estimator block 540 and a decision block 550 , block 510 for the electroacoustic system model, T generally denotes the matrix of the transfer functions T and any electroacoustic system such as system 100 or 500 ,

The estimator block 520 predicts or iteratively (theta) from phi and u, where the prediction error signal e from the summing block 530 is fed back to update each new prediction. In the exemplary embodiment, the estimator block takes 520 the shape of a Kalman filter (as described above). However, other embodiments use forms of linear estimators, such as RLS and LMS estimators.

The variance ratio estimator block 540 estimates the variance for the prediction error, S _e and the variance of the signals ϕ, S _ϕ , and calculates the ratio S _e / S _ϕ . The relationship becomes the decision block 550 provided that compares the ratio with the detection threshold DT. If the ratio exceeds the threshold, the user voice activity detection signal is set to one to indicate the presence of the user's speech. Otherwise the detection signal is set to zero.

CONCLUSION

Certain exemplary embodiments have been described in the above description. However, one of ordinary skill in the art will recognize that various modifications and changes can be made without departing from the scope of the invention as defined in the following claims. Accordingly, the description and figures are to be regarded in an exemplary rather than a restrictive sense, and all such modifications are intended to be included in the disclosure of the present teachings.

The merits, advantages, solutions to problems and all elements which enable or better emphasize any merits, advantages or problem solving should not be regarded as critical, necessary or essential features or elements of each or all of the claims. The invention is defined solely by the appended claims, including all changes made during the pending application of this application and all of its equivalents to these claims.

In addition, relative references, such as second, top, bottom, and the like, may only be used in this document to differentiate an entity or action from another entity or action, without necessarily requiring or implying that an actual such relationship exists between them Entities or actions must exist. The terms "comprise", "comprehensive", "has", "with", "points to", "showing", "contains", "containing" or any variant thereof are intended to represent a non-exclusive inclusion statement such that a Process, a method, an article or an apparatus, which has, has, includes, includes, not only those elements, but other not explicitly or in the list or such a process, method, Items or apparatus may also have inherent elements. An element that includes "includes a", "has a", has a ... "," includes a "does not exclude the existence of additional identical elements in the process, procedure, article, or apparatus without further conditions, which the Element includes, has, has, or includes The articles "a" and "one" are defined as "one or more", unless explicitly stated otherwise. The terms "essentially", "substantial""," Approximately "," approximately "or any other variant thereof are defined as" close to ", as will be apparent to the average person skilled in the art, and in one non-limiting embodiment, the terms are considered to be within 10% deviation, viewed in another embodiment as within a 5% deviation, in another embodiment as within 1% in another embodiment as within 0.5%. The term “coupled” as used herein is defined as connected, but not necessarily in a direct or mechanical manner. A device or structure that is "designed" in a particular way is designed at least in this way, but can also be in not be designed. The term “exemplary” is also used adjectively herein to modify one or more nouns, such as embodiment, system, method, or device, and is intended to specifically indicate that the noun is meant to be a non-limiting example.

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 15711793 [0002]
• US 62456100 [0002]
• US 15069271 [0025]
• US 14/850859 [0055]
• US 62530049 [0057]
• US 14850859 [0059]

Claims (19)

A communication arrangement comprising: an incoming audio signal node configured to connect to a communication device providing the incoming audio signal Rx; a signal processing circuit configured to output a voice activity signal indicating a detected presence or absence of a user's utterance, and depending on a first signal ϕ, a second signal θ and a third signal to determine whether a user's utterance is present is or not, the signals ϕ and θ being derived from two or more transmitters which respond to at least ambient noise a, speech utterances by a user s and the audio input signal u, with the boundary condition for the signals ϕ and θ: $${\left|\frac{T_{s\theta }}{T_{a\theta }}-\frac{T_{s\phi }}{T_{a\phi }}\right|}^2>0$$

wherein T _{sθ represents} a transfer function of user utterances s to the signal θ; Tae represents a transfer function of ambient noise on the signal θ; T _{sΦ represents} a transfer function of user utterances s to the signal ϕ; and T _{aϕ represents} a transfer function of ambient noise on the signal ϕ; and the third signal is at least partially dependent on the incoming audio signal Rx or at least one audio notification signal. The communication arrangement according to Claim 1 , wherein the two or more transmitters comprise first and second microphones, of which the first microphone is placed with a significantly higher acoustic coupling to a vocal tract of a user than the second microphone. The communication arrangement according to Claim 2 , wherein the first microphone is arranged so that it absorbs more acoustic energy via a bone conduction path from the user's vocal tract than the second microphone. The communication arrangement according to Claim 1 , further comprising: an automatic muting module, which is designed to attenuate a voice microphone output signal in response to a voice activity detection signal indicating an absence of user utterances and to take it out of the attenuation in response to a voice activity detection signal indicating a presence of user utterances. The communication arrangement according to Claim 4 , wherein the automatic mute module is configured to mute or attenuate the voice microphone output signal after a predetermined period of time has elapsed after receipt of the voice activity detection signal. The communication arrangement according to Claim 1 , further comprising: a keyword recognition module, which is designed to recognize one or more keywords in user speech utterance signals derived from a speech microphone output signal and to assume a shutdown state or sleep state in response to a voice activity detection signal indicating a detected absence of speech utterances of the user, in order to save energy or memory , The communication arrangement according to Claim 1 , further comprising: an auxiliary sound module which reacts to the user's utterances in order to generate an auxiliary sound signal for the user on a loudspeaker within an earpiece of a headset and, depending on a voice activity detection signal indicating a detected absence of the user's utterances, assume a switch-off state or sleep state, to save energy or storage. The communication arrangement according to Claim 1 , further comprising: an auxiliary sound module, which reacts to the user's utterances in order to generate an auxiliary sound signal for the user on a loudspeaker within an earpiece of a headset and, depending on a voice activity detection signal indicating a detected absence of user utterances, assume a switch-off state or sleep state in order to assume To save energy or storage. The communication arrangement according to Claim 1 , still comprehensive: a noise reduction module which reduces noise to a voice activity detection signal indicating a presence of user utterances, and reduces noise in a second manner depending on a voice activity detection signal indicating an absence of user utterances. The communication arrangement according to Claim 1 , further comprising at least one language-dependent module which is designed to be operated in a manner dependent on a speech content of a speech microphone output signal and to adopt an energy and / or memory-saving mode in response to a voice activity detection signal indicating an absence of the user's utterance. The communication arrangement according to Claim 1 , wherein the arrangement comprises a first node for connection to a loudspeaker assigned to a headset earpiece, a second node for connection to a pilot environment microphone assigned to the headset earpiece, and a third node for connection to a feedback error microphone assigned to the headset earpiece, the signal processing circuit having the first, second and the third node and is configured to output an anti-noise signal to the speaker in response to signals from the pilot environment microphone and the feedback error microphone, and wherein the signals ϕ and θ are derived from signals from the pilot environment microphone and the feedback error microphone are. The communication arrangement according to Claim 11 wherein the signal processing circuitry is further configured to indicate absence of a user utterance when the first and second transmitters are placed on a standardized head and torso simulator (HATS) that outputs a speech signal in an area. The communication arrangement according to Claim 11 wherein the headset earpiece has an ear canal earphone with an ear canal portion containing the feedback error microphone. A method of operating a communication arrangement having a voice microphone output signal and a set of two or more transmitters, the method comprising: Determining whether a user of the communication arrangement speaks depending on the output signal of the two or more transmitters; and in response to determining that the user is not speaking, changing the operating state of one or more language-dependent modules associated with the communication arrangement to a resource-saving state. The procedure according to Claim 14 wherein the one or more language-dependent modules include at least one of the following: a mute module configured to mute the voice microphone output signal in response to determining that the user is not speaking; a keyword recognition module configured to recognize one or more specific keywords or phrases; and a sub-tone module, which is designed to generate a sub-tone signal in response to the voice microphone output signal. The procedure according to Claim 14 , wherein determining whether a user of the communication arrangement is dependent on the output signal of the two or more transmitters comprises: deriving a first signal Signals, a second signal θ, and a third signal, the signals Φ and θ from the two or more, an ambient noise signal a, a user speech expression signal s and a third signal-transmitting transmitters are derived and satisfy a constraint $${\left|\frac{T_{s\theta }}{T_{a\theta }}-\frac{T_{s\phi }}{T_{a\phi }}\right|}^2>0$$

wherein T _{sθ represents} a transfer function of user utterances s to the signal θ; T _{aθ represents} a transfer function of ambient noise on the signal θ; T _{sΦ represents} a transfer function of user utterances s to the signal Φ; and T _{aϕ represents} a transfer function of ambient noise onto the signal Φ and wherein the third signal is a function dependent on at least one incoming audio signal from an external device or at least one audio notification signal. A communication arrangement comprising: first and second transmitters configured to be carried on a head of a user, of which the first transmitter is configured to have a significantly higher acoustic coupling to a vocal tract of a user than the second transmitter; and a signal processing circuit configured to output a voice activity signal indicating a detected presence or absence of a user's utterance in a predetermined frequency range when the first and second transmitters are worn on a user's head; and wherein the signal processing circuit is further configured to indicate absence of a user utterance when the first and second transmitters are placed on a standardized head-and-torso simulator (HATS) which outputs a speech signal in the predetermined frequency range. The communication arrangement according to Claim 17 , the arrangement comprising an earpiece with a loudspeaker, a pilot environment microphone, a feedback error microphone, a speech microphone and a node for incoming audio signals, which is designed to connect to a communication device which provides the incoming audio signal Rx; wherein the signal processing circuit is configured to determine, based on a first signal ϕ, a second signal θ, and a third signal, whether or not there is a user utterance, the signals ϕ and θ from at least two of the pilot environment microphone, the feedback error microphone , and the speech microphone are derived, which react to at least ambient noise a, speech utterances of a user s, and the audio input signal u, with the boundary condition for the signals ϕ and θ: $${\left|\frac{T_{s\theta }}{T_{a\theta }}-\frac{T_{s\phi }}{T_{a\phi }}\right|}^2>0$$

wherein T _{sθ represents} a transfer function of user utterances s to the signal θ; T _{aθ represents} a transfer function of ambient noise on the signal θ; T _{sΦ represents} a transfer function of user utterances s to the signal ϕ; and T _{aϕ represents} a transfer function of ambient noise on the signal ϕ; and the third signal is at least partially dependent on the incoming audio signal Rx or at least one audio notification signal. The communication arrangement according to Claim 18 wherein the signal processing circuit is configured to output an anti-noise signal to the speaker in response to signals from the pilot environment microphone and the feedback error microphone; and wherein the arrangement further includes a set of one or more language-dependent modules, each of which is configured to enter a power and / or memory saving mode in response to a voice activity detection signal indicative of an absence of user utterances, and in response to presence exiting voice activity detection signal indicative of user utterances from power or memory saving mode; and wherein the set comprises, on one or more voice-dependent modules: a voice microphone output module, which is designed to output a voice microphone signal to the communication device; a keyword recognition module that is designed to recognize one or more keywords or key terms; and a sub-tone module, which is designed to generate a sub-tone signal as a function of the speech microphone signal.

Images