WO2017218128A1 - Far field automatic speech recognition pre-processing - Google Patents
Far field automatic speech recognition pre-processing Download PDFInfo
- Publication number
- WO2017218128A1 WO2017218128A1 PCT/US2017/032913 US2017032913W WO2017218128A1 WO 2017218128 A1 WO2017218128 A1 WO 2017218128A1 US 2017032913 W US2017032913 W US 2017032913W WO 2017218128 A1 WO2017218128 A1 WO 2017218128A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- audio channels
- beams
- partition
- audio
- audio signal
- Prior art date
Links
- 238000007781 pre-processing Methods 0.000 title claims abstract description 34
- 238000005192 partition Methods 0.000 claims abstract description 70
- 230000005236 sound signal Effects 0.000 claims abstract description 57
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000001914 filtration Methods 0.000 claims description 25
- 230000003595 spectral effect Effects 0.000 claims description 25
- 238000000638 solvent extraction Methods 0.000 claims description 24
- 230000009467 reduction Effects 0.000 claims description 18
- 238000002592 echocardiography Methods 0.000 claims description 6
- 230000000875 corresponding effect Effects 0.000 description 16
- 230000015654 memory Effects 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000006872 improvement Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000004807 localization Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000002596 correlated effect Effects 0.000 description 2
- 230000007423 decrease Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000005291 magnetic effect Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000004020 conductor Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007274 generation of a signal involved in cell-cell signaling Effects 0.000 description 1
- 239000012212 insulator Substances 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0324—Details of processing therefor
- G10L21/034—Automatic adjustment
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/02—Casings; Cabinets ; Supports therefor; Mountings therein
- H04R1/04—Structural association of microphone with electric circuitry therefor
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/20—Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L21/0232—Processing in the frequency domain
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/21—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being power information
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
- H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R31/00—Apparatus or processes specially adapted for the manufacture of transducers or diaphragms therefor
- H04R31/006—Interconnection of transducer parts
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L2021/02082—Noise filtering the noise being echo, reverberation of the speech
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02166—Microphone arrays; Beamforming
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/22—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired frequency characteristic only
- H04R1/28—Transducer mountings or enclosures modified by provision of mechanical or acoustic impedances, e.g. resonator, damping means
- H04R1/2869—Reduction of undesired resonances, i.e. standing waves within enclosure, or of undesired vibrations, i.e. of the enclosure itself
- H04R1/2876—Reduction of undesired resonances, i.e. standing waves within enclosure, or of undesired vibrations, i.e. of the enclosure itself by means of damping material, e.g. as cladding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
- H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
- H04R2201/401—2D or 3D arrays of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
- H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
- H04R2201/403—Linear arrays of transducers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2420/00—Details of connection covered by H04R, not provided for in its groups
- H04R2420/07—Applications of wireless loudspeakers or wireless microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
- H04R3/005—Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
Definitions
- Embodiments described herein generally relate to automatic speech recognition (ASR) and more specifically to improving ASR pre-processing.
- ASR automatic speech recognition
- ASR involves a machine-based collection of techniques to understand human languages.
- ASR is interdisciplinary, often involving
- Far field ASR involves techniques to decrease a word error rate (WER) in utterances made a greater distance to a microphone, or microphone array, than traditionally accounted for in ASR processing pipelines. Such distance often decreases the signal to noise (SNR) ratio and thus increases WER in traditional ASR systems.
- WER word error rate
- SNR signal to noise
- FIG. 1 is an example of a smart home gateway housing, according to an embodiment.
- FIG. 2 is a block diagram of an example of a system for far field automatic speech recognition pre-processing, according to an embodiment.
- FIG. 3 illustrates phase-based beam forming (PBF) directivity patterns, according to an embodiment.
- FIG. 4 is a plot of far field ASR WER improvements for different types of noise, according to an embodiment.
- FIG. 5 illustrates an example of a method for automatic speech recognition pre-processing, according to an embodiment.
- FIG. 6 is a block diagram illustrating an example of a machine upon which one or more embodiments may be implemented.
- Embodiments and examples herein general described a number of systems, devices, and techniques for automatic speech recognition pre-processing. It is understood, however, that the systems, devices, and techniques are examples illustrating the underlying concepts.
- FIG. 1 is an example of a smart home gateway 105, according to an embodiment. As illustrated, the circles atop the housing are lumens 110 behind which are housed microphones (as illustrated there are eight microphones). The dashed lines illustrate microphones in a linear arrangement 115 as well as in a circular arrangement 120. Many of the examples described herein operate with these dual arrangements (e.g., linear 115 and circular 120) with respect to a device 105. Although the device 105 here takes the form of the smart home gateway, other configurations are contemplated, such as in a desktop or laptop computer configuration, a refrigerator or other appliance, etc.
- a factor contributing to the far field performance drop for ASR may include speech signal quality degradation due to some or all of reverberations, echo, noise, or amplitude loss. For example, from several experiments, four issues related to far field ASR where found: reverberation; echo; noise; and amplitude losses. The influence of one or all of these factors may be mitigated by intelligently ordering a variety of processing techniques. For example, reverberation (e.g., reverb) reduction enables use of beam-formers and noise reduction (NR) techniques that were not designed to work in reverberant conditions. In another example, acoustic echo cancelation (AEC) reduces echo generated by internal loudspeakers.
- reverberation e.g., reverb
- NR noise reduction
- AEC acoustic echo cancelation
- beam-formers and additional post-filtering modules reduce noise level.
- An automatic gain control (AGC) device counteracts amplitude losses. Overall the unique combination and order of the processing used in the described far field preprocessing pipeline enables accurate far field ASR.
- An example of just such a pipeline in the device 105 may include a sampler 125, a de-reverberator 127, a beam-former processor 130, a stream selector 135, a filter 140, and a controller 145.
- Each of these components are implemented in electronic hardware, such as that described below (e.g., circuits).
- the sampler 125 is arranged to obtain a plurality of audio channels.
- the sampler 125 may be a part of a microphone array, have a tap on microphone output, or have the plurality of audio channels delivered via another component of the device 105.
- an audio channel is audio from a single microphone.
- an audio channel is audio from a plurality of microphones wherein the signal from these microphones is correlated based on a physical arrangement of the microphones, such as a spacing, linear or circular relationship, etc.
- the de-reverberator 127 removes reverberation prior to the beam- former processor partitioning the plurality of audio channels into beams.
- Removing the reverberation may be accomplished using a variety of techniques, such as short- time Fourier transform (STFT) domain inverse filtering methods, non-negative room impulse response (RIR) modeling, statistical RIR modeling, or nonlinear mapping (e.g., denoising auto-encoder using a deep neural network or bidirectional long short-term memory (BLSTM) recurrent neural network).
- STFT short- time Fourier transform
- RIR room impulse response
- BLSTM bidirectional long short-term memory
- the beam-former processor 130 is arranged to partition the plurality of audio channels into beams.
- beams refer to energy received from a specific direction.
- the frequency and amplitude of sound energy may be determined but there is not enough information to also determine a direction.
- the addition of a second microphone (e.g., analogous to two human ear) provides two signals that may be correlated in frequency and amplitude but which may vary in time. With a known and fixed relationship between these microphones, the variations of the audio signal in time may provide a relative direction of the energy.
- the beam-former processor 130 is arranged to obtain (e.g., receive or retrieve) the plurality of audio channels, partition the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels, and provide each partition to the phase-based beam-former.
- the audio channel partitioning allows the beam-former processor 130 or the phased-based beam-former to ascertain the time variance (e.g., a measure of how in-phase the signals are) with a known physical arrangement of microphones. As explained earlier, this provides the information to ascertain from what direction the energy (e.g., sound) came from Beamforming provides another level of control in finding a clean signal from which to process ASR.
- the stream selector 135 is arranged to select a partition
- the stream selector 135 is arranged to compare noise levels between the beams and select the beam based on having the lowest noise levels determined from the comparison.
- the stream selector 135 uses a phrase quality scorer of the stream selector to compare the noise levels across the beams.
- an SNR meter of the stream selector provides a noise level for each beam. The stream selector 135 thus discriminates amongst a variety of possible input sources to provide (e.g., to send or make available) a better signal to downstream processors.
- the filter 140 is arranged to reduce the level of noise in an audio signal from the selected partition. In an example, to reduce the level of noise in the audio signal from the selected partition, the filter 140 applies noise reduction to the audio signal. In an example, to enhance the speech signal from the selected partition, the filter applies a spectral profile matching (SPM) to the audio signal. In an example, the spectral profile matching is applied after noise reduction is applied to the audio signal.
- SPM spectral profile matching
- the filter 140 applies an automated gain control to the audio signal.
- the automated gain control is applied after a spectral matching profile is applied to the audio signal.
- the pipeline may optionally include a second filter
- the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
- the second filter is part of the de-reverberator 127.
- the controller 145 is arranged to provide the audio signal to an external entity via an output interface of the pre-processing pipeline.
- the controller 145 interfaces with downstream components to further process the semantic content in an ASR system
- FIG. 2 is a block diagram of an example of a system 200 for far field automatic speech recognition pre-processing, according to an embodiment.
- the system 200 includes additional examples of the components discussed above.
- the components of the system 200 are implemented in electronic hardware, such as that described above or below (e.g., circuits).
- the system 200 includes a pipeline 205 for real-time far field ASR.
- ASR techniques that previously have been discarded in far field ASR due to reverberations may be reintroduced, such as:
- the far field pre-processing pipeline 205 may be composed of six processing blocks: a de-reverberator 210; an optional AEC 215; a beam-former 220; a stream selector 230; a post-filtering block 245; and a content analysis block 265.
- the far field pre-processing pipeline 205 may operate on a multichannel input.
- the multichannel input may be obtained from a microphone array containing at least two microphones.
- there are no limitations for the microphone array geometry e.g., linear, circular, etc.
- the number of microphones are an even number (e.g., the modulus of the number of microphones and two is zero).
- de-reverb block 210 reverberations are removed from the multichannel input. Parameters of the de-reverberation block 210 may be adjusted to balance computational complexity and performance. Techniques to remove reverberation may include pre-configured room impulse models, or others, as described above.
- the far field pre-processing pipeline 205 may be used with the device containing internal loudspeakers.
- acoustical leakage from the loudspeakers to the microphones may be reduced by the optional multichannel AEC block 215.
- the AEC block 215 includes one or more of the following properties:
- the beam- former block 220 may use phase-based beam formers (PBFs) 225, or other beam forming techniques such as the Minimum Variance Distortionless Response beam formers, to process the multichannel stream.
- PBFs 225 phase-based beam formers
- the beam- former block 220 may use phase-based beam formers (PBFs) 225, or other beam forming techniques such as the Minimum Variance Distortionless Response beam formers, to process the multichannel stream.
- PBFs 225 cannot be used without removing the echo and reverb because the PBF 225 generally requires direct sound in the microphone signals. In reverberant conditions this requirement is not met because reflections (e.g., none-direct signals) would also be captured. Consequently, the precise detection of user position— an important feature in PBF 225 processing— will not be possible.
- the PBFs 225 use two signals coming from a microphone pair. Therefore, for microphone arrays with more than two microphones, multiple instances of PBFs 225 may be used (e.g., one PBF 225 for each exclusive pair). Each PBF 225 instance may be steered toward different directions (e.g., relative to the device).
- FIG. 3 illustrates directivity patterns of four PBF 225 instances when used together with the microphone board described herein. In FIG.
- the PBF 225 processing is bi-directional— e.g., the same beam pattern for front and back facing directions relative to a microphone pair, these directions being perpendicular to a line drawn between the two microphones— the combined solution provides 360 degrees coverage (e.g., the circular long and short dashed lines in FIG. 3).
- the stream selector 230 may assess each directional stream against selected localization criteria, such as highest Signal-to-Noise Ratio (SNR)— e.g., calculated using the Signal Level Measurement (SLM) 270 or highest score of the Voice Activity Detector (VAD) 275 in the content analysis block 265— and select a stream more conducive to ASR.
- SNR Signal-to-Noise Ratio
- SLM Signal Level Measurement
- VAD Voice Activity Detector
- the stream selector 230 may include one or more of a phrase quality scorer 235 or SNR meter 240 to provide localization criteria scores on the streams.
- only one of the PBF- processed streams may be selected for further processing (e.g., the stream with the highest SNR), by the stream selector 230. Because the selected stream (e.g., for further processing) is beam-formed, the influence of noise coming from all directions (e.g., areas not covered by the formed beam) is reduced and the user's speech is better exposed (e.g., more clear or less obstructed by that noise). This improves SNR leading to better far field ASR performance.
- one or more post-filtering operations may be applied to the streams by the post filtering block 245.
- Example post-filtering operations may include:
- SPM Spectral Profile Matching
- AGC 260 used to normalize signal level.
- the NR 250 may accept a reference stream containing PBF-processed signals that were classified by the stream selector block 230 as noisy, at least compared to the other available streams (e.g., beams pointing in a direction that is different than that of the user).
- noisy streams may be used to calculate a robust estimation of the noise floor that the NR 250 will remove.
- the AGC block 260 uses a reference signal.
- the reference signal may be a typical loopback signal from the playback path.
- the SPM block 255 helps some ASR engines and the NR 250 helps for some other (e.g., different) ASR engines.
- the inclusion of one or more of these components is optional, providing further customization for performance, effectiveness, power use, design complexity, etc.
- Output of the far field pre-processing pipeline may be provided to a client 280 that may implement an ASR engine 285.
- the client 280 may implement a wake on voice (WoV) engine 290 or in a VoIP communication channel 295.
- FIG. 4 illustrates far field ASR WER improvements obtained using the far field pre-processing pipeline 205.
- FIG. 4 illustrates far field ASR WER improvements for different noise types— LiRo: living room; SiSp: side speaker; Public: public place; and Work: work place— obtained using the far field pre-processing pipeline; unprocessed signals are the dashed line (on top) and processed signals are the short dash-double dotted line (on bottom).
- FIG. 5 illustrates an example of a method 500 for automatic speech recognition pre-processing, according to an embodiment.
- the operations of the method 500 are implemented in electronic hardware, such as that described above or below (e.g., circuits).
- a plurality of audio channels is obtained.
- obtaining the plurality of audio channels includes removing reverberation prior to a beam-former processor partitioning the plurality of audio channels into beams.
- the plurality of audio channels are partitioned into beams.
- partitioning the plurality of audio channels into beams includes receiving the plurality of audio channels at a beam-former processor, partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels, and providing each partition to a phase-based beam-former.
- a partition corresponding to a beam in the beams is selected based on a noise level.
- selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
- a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
- a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
- SNR signal-to-noise
- an speech signal is filtered from the selected partition.
- the filtering includes applying noise reduction to the audio signal.
- the filtering includes applying a spectral matching profile (SPM) to the audio signal.
- SPM is applied after noise reduction is applied to the audio signal.
- the filtering includes applying an automated gain control to the audio signal.
- the automated gain control is applied after a spectral matching profile is applied to the audio signal
- the method 500 may be extended by optionally performing acoustic echo cancellation to the plurality of audio channels.
- the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
- the filtered audio signal is provided to an external entity via an output interface of the pre-processing pipeline.
- FIG. 6 illustrates a block diagram of an example machine 600 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform
- the machine 600 may operate as a standalone device or may be connected (e.g., networked) to other machines.
- the machine 600 may operate in the capacity of a server machine, a client machine, or both in server-client network environments.
- the machine 600 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment.
- P2P peer-to-peer
- the machine 600 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine.
- PC personal computer
- PDA personal digital assistant
- STB set-top box
- PDA personal digital assistant
- mobile telephone a web appliance
- network router, switch or bridge or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine.
- machine shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations.
- SaaS software as a service
- Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms.
- Circuitry is a collection of circuits implemented in tangible entities that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time and underlying hardware variability. Circuitries include members that may, alone or in
- hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired).
- the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation.
- variably connected physical components e.g., execution units, transistors, simple circuits, etc.
- a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation.
- the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa.
- the instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation.
- the computer readable medium is communicatively coupled to the other components of the circuitry when the device is operating.
- any of the physical components may be used in more than one member of more than one circuitry.
- execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time.
- Machine 600 may include a hardware processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 604 and a static memory 606, some or all of which may communicate with each other via an interlink (e.g., bus) 608.
- the machine 600 may further include a display unit 610, an alphanumeric input device 612 (e.g., a keyboard), and a user interface (UI) navigation device 614 (e.g., a mouse).
- the display unit 610, input device 612 and UI navigation device 614 may be a touch screen display.
- the machine 600 may additionally include a storage device (e.g., drive unit) 616, a signal generation device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 621 , such as a global positioning system (GPS) sensor, compass, accelero meter, or other sensor.
- the machine 600 may include an output controller 628, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).
- a serial e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).
- USB universal serial bus
- the storage device 616 may include a machine readable medium 622 on which is stored one or more sets of data structures or instructions 624 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein.
- the instructions 624 may also reside, completely or at least partially, within the main memory 604, within static memory 606, or within the hardware processor 602 during execution thereof by the machine 600.
- one or any combination of the hardware processor 602, the main memory 604, the static memory 606, or the storage device 616 may constitute machine readable media.
- machine readable medium 622 is illustrated as a single medium, the term “machine readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) configured to store the one or more instructions 624.
- machine readable medium may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) configured to store the one or more instructions 624.
- machine readable medium may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine 600 and that cause the machine 600 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions.
- Non-limiting machine readable medium examples may include solid-state memories, and optical and magnetic media.
- a massed machine readable medium comprises a machine readable medium with a plurality of particles having invariant (e.g., rest) mass. Accordingly, massed machine-readable media are not transitory propagating signals.
- Specific examples of massed machine readable media may include: nonvolatile memory, such as semiconductor memory devices (e.g.. Electrically
- EPROM Programmable Read-Only Memory
- EEPROM Electrically Erasable Programmable Read-Only Memory
- flash memory devices such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
- the instructions 624 may further be transmitted or received over a communications network 626 using a transmission medium via the network interface device 620 utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.).
- transfer protocols e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.
- communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as Wi-Fi®, IEEE 802.16 family of standards known as WiMax®), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others.
- the network interface device 620 may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas to connect to the communications network 626.
- the network interface device 620 may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques.
- SIMO single-input multiple-output
- MIMO multiple-input multiple-output
- MISO multiple-input single-output
- transmission medium shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 600, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.
- Example 1 is a system for automatic speech recognition preprocessing, the system comprising: a sampler to obtain a plurality of audio channels; a de-reverberator to remove reverberations from the plurality of audio channels; a beam-former processor to partition the plurality of audio channels into beams after reverberations are removed; a stream selector to select a partition corresponding to a beam in the beams based on a noise level; a filter to reduce a noise level in a speech signal from the selected partition; and a controller to provide the audio signal to an external entity via an output interface of the pre-processing pipeline.
- Example 2 the subject matter of Example 1 optionally includes an echo cancelation block disposed between the de-reverberator and the beam-former processor to cancel echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
- an echo cancelation block disposed between the de-reverberator and the beam-former processor to cancel echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
- Example 3 the subject matter of any one or more of Examples 1-2 optionally include wherein, to partition the plurality of audio channels into beams, the beam-former processor is to: receive the plurality of audio channels; partition the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and provide each partition to a phase-based beam-former.
- Example 4 the subject matter of any one or more of Examples 1-3 optionally include wherein, to select the partition corresponding to the beam based on the noise level, the stream selector is to: compare speech levels between the beams; and select the beam based on having the highest speech levels determined from the comparison.
- Example S the subject matter of any one or more of Examples 1-4 optionally include wherein, to select the partition corresponding to the beam based on the noise level, the stream selector is to: compare noise levels between the beams; and select the beam based on having the lowest noise levels determined from the comparison.
- Example 6 the subject matter of Example 5 optionally includes wherein the stream selector uses a phrase quality scorer of the stream selector to compare the noise levels between the beams.
- Example 7 the subject matter of Example 6 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
- SNR signal-to-noise
- Example 8 the subject matter of any one or more of Examples 1-7 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies noise reduction to the audio signal.
- Example 9 the subject matter of any one or more of Examples 1-8 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies a spectral profile matching (SPM) to the audio signal.
- SPM spectral profile matching
- Example 10 the subject matter of Example 9 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
- Example 11 the subject matter of any one or more of Examples 1- 10 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies an automated gain control to the audio signal.
- Example 12 the subject matter of Example 11 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
- Example 13 the subject matter of any one or more of Examples 1-
- a second filter to perform acoustic echo cancellation to the plurality of audio channels.
- Example 14 the subject matter of Example 13 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
- Example 1 is at least machine readable medium including instructions for a pre-processing pipeline, the instructions, when executed by a machine, causing the machine to perform operations comprising: obtaining a plurality of audio channels; removing reverberations from the audio channels;
- Example 16 the subject matter of Example IS optionally includes wherein the operations include canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
- Example 17 the subject matter of any one or more of Examples 15-16 optionally include wherein the partitioning the plurality of audio channels into beams includes: receiving the plurality of audio channels at a beam-former processor; partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase-based beam-former.
- Example 18 the subject matter of any one or more of Examples 15-17 optionally include wherein the selecting the partition corresponding to the beam based on the noise level includes comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
- Example 19 the subject matter of any one or more of Examples 15-18 optionally include wherein the selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
- Example 20 the subject matter of Example 19 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
- Example 21 the subject matter of Example 20 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
- SNR signal-to-noise
- Example 22 the subject matter of any one or more of Examples
- 15-21 optionally include wherein the filtering includes applying noise reduction to the audio signal.
- Example 23 the subject matter of any one or more of Examples
- the filtering includes applying a spectral profile matching (SPM) to the audio signal.
- SPM spectral profile matching
- Example 24 the subject matter of Example 23 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
- Example 25 the subject matter of any one or more of Examples 15-24 optionally include wherein the filtering includes applying an automated gain control to the audio signal.
- Example 26 the subject matter of Example 25 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
- Example 27 the subject matter of any one or more of Examples 15-26 optionally include wherein the operations comprise performing acoustic echo cancellation to the plurality of audio channels.
- Example 28 the subject matter of Example 27 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
- Example 29 is a method for automatic speech recognition preprocessing, the method comprising: obtaining a plurality of audio channels;
- removing reverberations from the audio channels partitioning the plurality of audio channels into beams after the reverberations are removed; selecting a partition corresponding to a beam in the beams based on a noise level; filtering an audio signal from the selected partition; and providing the filtered audio signal to an external entity via an output interface of the pre-processing pipeline.
- Example 30 the subject matter of Example 29 optionally includes canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
- Example 31 the subject matter of any one or more of Examples
- partitioning the plurality of audio channels into beams includes: receiving the plurality of audio channels at a beam-former processor; partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase-based beam-former.
- Example 32 the subject matter of any one or more of Examples
- Example 29-31 optionally include wherein selecting the partition corresponding to the beam based on the noise level includes comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
- 29-32 optionally include wherein selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
- Example 34 the subject matter of Example 33 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
- Example 35 the subject matter of Example 34 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
- SNR signal-to-noise
- Example 36 the subject matter of any one or more of Examples
- 29-35 optionally include wherein the filtering includes applying noise reduction to the audio signal.
- Example 37 the subject matter of any one or more of Examples
- 29-36 optionally include wherein the filtering includes applying a spectral profile matching (SPM) to the audio signal.
- SPM spectral profile matching
- Example 38 the subject matter of Example 37 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
- Example 39 the subject matter of any one or more of Examples
- 29-38 optionally include wherein the filtering includes applying an automated gain control to the audio signal.
- Example 40 the subject matter of Example 39 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
- Example 41 the subject matter of any one or more of Examples
- Example 29-40 optionally include performing acoustic echo cancellation to the plurality of audio channels.
- Example 42 the subject matter of Example 41 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
- Example 43 is a system comprising means to perform any of the methods 29-42.
- Example 44 is at least one machine readable medium including instructions that, when executed by a machine, cause the machine to perform any of the methods 29-42.
- Example 45 is a system for automatic speech recognition preprocessing, the system comprising: means for obtaining a plurality of audio channels; means for removing reverberations from the plurality of audio channels; means for partitioning the plurality of audio channels into beams after the reverberations are removed; means for selecting a partition corresponding to a beam in the beams based on a noise level; means for filtering an audio signal from the selected partition; and means for providing the filtered audio signal to an external entity via an output interface of the pre-processing pipeline.
- Example 46 the subject matter of Example 45 optionally includes means for canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
- Example 47 the subject matter of any one or more of Examples
- the means for partitioning the plurality of audio channels into beams includes: means for receiving the plurality of audio channels at a beam-former processor; means for partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase- based beam-former.
- Example 48 the subject matter of any one or more of Examples
- 45-47 optionally include wherein the means for selecting the partition
- corresponding to the beam based on the noise level includes means for comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
- Example 49 the subject matter of any one or more of Examples
- 45-48 optionally include wherein the means for selecting the partition
- corresponding to the beam based on the noise level includes means for comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
- Example SO the subject matter of Example 49 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
- Example 51 the subject matter of Example 50 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
- SNR signal-to-noise
- Example 52 the subject matter of any one or more of Examples 45-51 optionally include wherein the means for filtering includes means for applying noise reduction to the audio signal.
- Example 53 the subject matter of any one or more of Examples
- the means for filtering includes means for applying a spectral profile matching (SPM) to the audio signal.
- SPM spectral profile matching
- Example 54 the subject matter of Example 53 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
- Example 55 the subject matter of any one or more of Examples
- 45-54 optionally include wherein the means for filtering includes means for applying an automated gain control to the audio signal.
- Example 56 the subject matter of Example 55 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
- Example 57 the subject matter of any one or more of Examples 45-56 optionally include means for performing acoustic echo cancellation to the plurality of audio channels.
- Example 58 the subject matter of Example 57 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
Landscapes
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Signal Processing (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Otolaryngology (AREA)
- Manufacturing & Machinery (AREA)
- Circuit For Audible Band Transducer (AREA)
Abstract
System and techniques for automatic speech recognition pre-processing are described herein. First, a plurality of audio channels may be obtained. Then, reverberations mat be removed from the audio channels. The plurality of audio channels may be partitioned into beams after reverberations are removed. A partition corresponding to a beam in the beams may be selected based on a noise level. An audio signal may be filtered from the selected partition. The filtered audio signal may be provided to an external entity via an output interface of the pre-processing pipeline.
Description
FAR FIELD AUTOMATIC SPEECH RECOGNITION PRE-PROCESSING CLAIM OF PRIORITY
[0001] This patent application claims the benefit of priority to United States
Application Serial No. 15/388,147, filed December 22, 2016, which claims the benefit of priority to United States Provisional Application Serial No. 62/350,507, titled "FAR FIELD AUTOMATIC SPEECH RECOGNITION" and filed on June 15, 2016, the entirety of which are hereby incorporated by reference herein.
TECHNICAL FIELD
[0002] Embodiments described herein generally relate to automatic speech recognition (ASR) and more specifically to improving ASR pre-processing.
BACKGROUND
[0003] ASR involves a machine-based collection of techniques to understand human languages. ASR is interdisciplinary, often involving
microphone, analog to digital conversion, frequency processing, database, and artificial intelligence technologies to convert the spoken word into textual or machine readable representations of not only what said (e.g., a transcript) but also what was meant (e.g., semantic understanding) by a human speaker. Far field ASR involves techniques to decrease a word error rate (WER) in utterances made a greater distance to a microphone, or microphone array, than traditionally accounted for in ASR processing pipelines. Such distance often decreases the signal to noise (SNR) ratio and thus increases WER in traditional ASR systems. As used herein, far field ASR involves distances more than half meter from the microphone.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] In the drawings, which are not necessarily drawn to scale, like numerals may describe similar components in different views. Like numerals
having different letter suffixes may represent different instances of similar components. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in the present document.
[0005] FIG. 1 is an example of a smart home gateway housing, according to an embodiment.
[0006] FIG. 2 is a block diagram of an example of a system for far field automatic speech recognition pre-processing, according to an embodiment.
[0007] FIG. 3 illustrates phase-based beam forming (PBF) directivity patterns, according to an embodiment.
[0008] FIG. 4 is a plot of far field ASR WER improvements for different types of noise, according to an embodiment.
[0009] FIG. 5 illustrates an example of a method for automatic speech recognition pre-processing, according to an embodiment.
[0010] FIG. 6 is a block diagram illustrating an example of a machine upon which one or more embodiments may be implemented.
DETAILED DESCRIPTION
[0011] Embodiments and examples herein general described a number of systems, devices, and techniques for automatic speech recognition pre-processing. It is understood, however, that the systems, devices, and techniques are examples illustrating the underlying concepts.
[0012] FIG. 1 is an example of a smart home gateway 105, according to an embodiment. As illustrated, the circles atop the housing are lumens 110 behind which are housed microphones (as illustrated there are eight microphones). The dashed lines illustrate microphones in a linear arrangement 115 as well as in a circular arrangement 120. Many of the examples described herein operate with these dual arrangements (e.g., linear 115 and circular 120) with respect to a device 105. Although the device 105 here takes the form of the smart home gateway, other configurations are contemplated, such as in a desktop or laptop computer configuration, a refrigerator or other appliance, etc.
[0013] A factor contributing to the far field performance drop for ASR may include speech signal quality degradation due to some or all of reverberations, echo, noise, or amplitude loss. For example, from several experiments, four issues related to far field ASR where found: reverberation; echo; noise; and amplitude losses. The influence of one or all of these factors may be mitigated by intelligently ordering a variety of processing techniques. For example, reverberation (e.g., reverb) reduction enables use of beam-formers and noise reduction (NR) techniques that were not designed to work in reverberant conditions. In another example, acoustic echo cancelation (AEC) reduces echo generated by internal loudspeakers. Also, for example, beam-formers and additional post-filtering modules reduce noise level. An automatic gain control (AGC) device counteracts amplitude losses. Overall the unique combination and order of the processing used in the described far field preprocessing pipeline enables accurate far field ASR.
[0014] An example of just such a pipeline in the device 105 may include a sampler 125, a de-reverberator 127, a beam-former processor 130, a stream selector 135, a filter 140, and a controller 145. Each of these components are implemented in electronic hardware, such as that described below (e.g., circuits).
[0015] The sampler 125 is arranged to obtain a plurality of audio channels. Thus, the sampler 125 may be a part of a microphone array, have a tap on microphone output, or have the plurality of audio channels delivered via another component of the device 105. In an example, an audio channel is audio from a single microphone. In an example, an audio channel is audio from a plurality of microphones wherein the signal from these microphones is correlated based on a physical arrangement of the microphones, such as a spacing, linear or circular relationship, etc. In an example, after obtaining the plurality of audio channels by the sampler 125, the de-reverberator 127 removes reverberation prior to the beam- former processor partitioning the plurality of audio channels into beams. Removing the reverberation may be accomplished using a variety of techniques, such as short- time Fourier transform (STFT) domain inverse filtering methods, non-negative room impulse response (RIR) modeling, statistical RIR modeling, or nonlinear mapping (e.g., denoising auto-encoder using a deep neural network or bidirectional
long short-term memory (BLSTM) recurrent neural network). After obtaining the plurality of audio channels by the sampler 125, and after applying de-reverberation to the audio channels by the de-reverberator 127, an output may be directed to, or retrieved by, the beam-former processor 130.
[0016] The beam-former processor 130 is arranged to partition the plurality of audio channels into beams. Here, beams refer to energy received from a specific direction. Generally, given a single stationary microphone, the frequency and amplitude of sound energy may be determined but there is not enough information to also determine a direction. The addition of a second microphone, (e.g., analogous to two human ear) provides two signals that may be correlated in frequency and amplitude but which may vary in time. With a known and fixed relationship between these microphones, the variations of the audio signal in time may provide a relative direction of the energy. This may then be considered the beam Thus, in an example, to partition the plurality of audio channels into beams, the beam-former processor 130 is arranged to obtain (e.g., receive or retrieve) the plurality of audio channels, partition the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels, and provide each partition to the phase-based beam-former. In this example, the audio channel partitioning allows the beam-former processor 130 or the phased-based beam-former to ascertain the time variance (e.g., a measure of how in-phase the signals are) with a known physical arrangement of microphones. As explained earlier, this provides the information to ascertain from what direction the energy (e.g., sound) came from Beamforming provides another level of control in finding a clean signal from which to process ASR.
[0017] The stream selector 135 is arranged to select a partition
corresponding to a beam in the beams based on a noise level. In an example, to select the partition corresponding to the beam based on the noise level, the stream selector 135 is arranged to compare noise levels between the beams and select the beam based on having the lowest noise levels determined from the comparison. In an example, the stream selector 135 uses a phrase quality scorer of the stream selector to compare the noise levels across the beams. In an example, an SNR meter
of the stream selector provides a noise level for each beam. The stream selector 135 thus discriminates amongst a variety of possible input sources to provide (e.g., to send or make available) a better signal to downstream processors.
[0018] The filter 140 is arranged to reduce the level of noise in an audio signal from the selected partition. In an example, to reduce the level of noise in the audio signal from the selected partition, the filter 140 applies noise reduction to the audio signal. In an example, to enhance the speech signal from the selected partition, the filter applies a spectral profile matching (SPM) to the audio signal. In an example, the spectral profile matching is applied after noise reduction is applied to the audio signal.
[0019] In an example, to boost the speech signal in the selected partition, the filter 140 applies an automated gain control to the audio signal. In an example, the automated gain control is applied after a spectral matching profile is applied to the audio signal.
[0020] In an example, the pipeline may optionally include a second filter
(not illustrated) to perform acoustic echo cancellation to the plurality of audio channels. In an example, the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams. In an example, the second filter is part of the de-reverberator 127.
[0021] The controller 145 is arranged to provide the audio signal to an external entity via an output interface of the pre-processing pipeline. Thus, the controller 145 interfaces with downstream components to further process the semantic content in an ASR system
[0022] FIG. 2 is a block diagram of an example of a system 200 for far field automatic speech recognition pre-processing, according to an embodiment. The system 200 includes additional examples of the components discussed above. The components of the system 200 are implemented in electronic hardware, such as that described above or below (e.g., circuits).
[0023] The system 200 includes a pipeline 205 for real-time far field ASR. By ordering the components of the system 200 as illustrated, ASR techniques that
previously have been discarded in far field ASR due to reverberations may be reintroduced, such as:
• the phase-based beam-former (PBF); and
• the Spectral Profile Matching (SPM)
[0024] The far field pre-processing pipeline 205 may be composed of six processing blocks: a de-reverberator 210; an optional AEC 215; a beam-former 220; a stream selector 230; a post-filtering block 245; and a content analysis block 265. In an example, the order of the far field pre-processing blocks is important (i.e., they must be in the order present in FIG. 2). The far field pre-processing pipeline 205 may operate on a multichannel input. The multichannel input may be obtained from a microphone array containing at least two microphones. In an example, there is no upper limit for the number of microphones that may be used. In an example, there are no limitations for the microphone array geometry (e.g., linear, circular, etc.). In an example, the number of microphones are an even number (e.g., the modulus of the number of microphones and two is zero).
[0025] In the de-reverb block 210, reverberations are removed from the multichannel input. Parameters of the de-reverberation block 210 may be adjusted to balance computational complexity and performance. Techniques to remove reverberation may include pre-configured room impulse models, or others, as described above.
[0026] In an example, the far field pre-processing pipeline 205 may be used with the device containing internal loudspeakers. In this example, acoustical leakage from the loudspeakers to the microphones may be reduced by the optional multichannel AEC block 215. In an example, the AEC block 215 includes one or more of the following properties:
* it is located after the de-reverb block 210, thus the AEC block 215 analyses signals that are not affected by the room reverb;
* it creates a cancelling filter using the multichannel reference signal, which improves AEC performance due to additional information that can be extracted from the different channels; or
• it is positioned before the beam-former block 220, not after the beam-former block 220.
[0027] After the AEC block 215, the multichannel stream has had the room reverb and loudspeaker echo removed (to the extent practical). Thus the beam- former block 220 may use phase-based beam formers (PBFs) 225, or other beam forming techniques such as the Minimum Variance Distortionless Response beam formers, to process the multichannel stream. Generally, for far field ASR, PBFs 225 cannot be used without removing the echo and reverb because the PBF 225 generally requires direct sound in the microphone signals. In reverberant conditions this requirement is not met because reflections (e.g., none-direct signals) would also be captured. Consequently, the precise detection of user position— an important feature in PBF 225 processing— will not be possible. This issue worsens for distances between the user and the device greater than two meters. However, in the illustrated arrangement, nearly all reflections (e.g., most of their energy) are removed before the PBF 225 stage. Thus, it is possible to use PBFs 225 effectively.
[0028] The PBFs 225 use two signals coming from a microphone pair. Therefore, for microphone arrays with more than two microphones, multiple instances of PBFs 225 may be used (e.g., one PBF 225 for each exclusive pair). Each PBF 225 instance may be steered toward different directions (e.g., relative to the device). FIG. 3 illustrates directivity patterns of four PBF 225 instances when used together with the microphone board described herein. In FIG. 3 signals from eight microphones, two blank, two diagonally striped, two diagonally cross- hatched, and two vertically cross-hatched, (grouped pairwise in the center with the center most microphones in a group) are grouped in four steering pairs of covered area [i.e., the groups of 1) dashed with two dots, 2) dashed with one dot, 3) dashed, and 4) dotted]. As illustrated, sounds from each area pair are fed into the separate PBF 225 instances. As a result, the PBF-processed signals point towards four different directions with a 45-degree beam width each. Since the PBF 225 processing is bi-directional— e.g., the same beam pattern for front and back facing directions relative to a microphone pair, these directions being perpendicular to a
line drawn between the two microphones— the combined solution provides 360 degrees coverage (e.g., the circular long and short dashed lines in FIG. 3).
[0029] In an example, owing to four directional streams, user localization is possible. Thus, the stream selector 230 may assess each directional stream against selected localization criteria, such as highest Signal-to-Noise Ratio (SNR)— e.g., calculated using the Signal Level Measurement (SLM) 270 or highest score of the Voice Activity Detector (VAD) 275 in the content analysis block 265— and select a stream more conducive to ASR. The stream selector 230 may include one or more of a phrase quality scorer 235 or SNR meter 240 to provide localization criteria scores on the streams. Based on the localization criteria, only one of the PBF- processed streams may be selected for further processing (e.g., the stream with the highest SNR), by the stream selector 230. Because the selected stream (e.g., for further processing) is beam-formed, the influence of noise coming from all directions (e.g., areas not covered by the formed beam) is reduced and the user's speech is better exposed (e.g., more clear or less obstructed by that noise). This improves SNR leading to better far field ASR performance.
[0030] In an example, one or more post-filtering operations may be applied to the streams by the post filtering block 245. Example post-filtering operations may include:
· NR 250 - used to reduce remaining noise;
• Spectral Profile Matching (SPM) 255 - used to equalize the speech signal to match frequency response of the ASRs training corpora; or
• AGC 260 - used to normalize signal level.
[0031] In an example, the NR 250 may accept a reference stream containing PBF-processed signals that were classified by the stream selector block 230 as noisy, at least compared to the other available streams (e.g., beams pointing in a direction that is different than that of the user). In an example, noisy streams may be used to calculate a robust estimation of the noise floor that the NR 250 will remove.
[0032] In an example, the AGC block 260 uses a reference signal. In an example, the reference signal may be a typical loopback signal from the playback path.
[0033] Some experiments have shown that the SPM block 255 helps some ASR engines and the NR 250 helps for some other (e.g., different) ASR engines. Thus, in an example, the inclusion of one or more of these components is optional, providing further customization for performance, effectiveness, power use, design complexity, etc.
[0034] Output of the far field pre-processing pipeline may be provided to a client 280 that may implement an ASR engine 285. In an example, however, the client 280 may implement a wake on voice (WoV) engine 290 or in a VoIP communication channel 295. FIG. 4 illustrates far field ASR WER improvements obtained using the far field pre-processing pipeline 205. FIG. 4 illustrates far field ASR WER improvements for different noise types— LiRo: living room; SiSp: side speaker; Public: public place; and Work: work place— obtained using the far field pre-processing pipeline; unprocessed signals are the dashed line (on top) and processed signals are the short dash-double dotted line (on bottom).
[0035] All of the blocks illustrated in FIG. 2 were implemented and evaluated to find their influence on far field ASR performance. It was shown that every element of the pipeline introduces improvement. The improvement was illustrated by the lower WERs obtained from multiple ASR engines in far field scenarios. Further, blocks were combined offline to simulate the far field preprocessing pipeline. The simulation demonstrated better ASR performance compared to using the blocks individually. The far field pre-processing pipeline 205 was then ported to a real-time audio stack and used in the mock-up of a smart home gateway device (e.g., intelligent loudspeaker) illustrated in FIG. 1. Real-time demonstrations with the mock-up exhibited the simulated far field ASR
improvements. Although the techniques discussed above are useful in far field applications, they may be applied in near field ASR, or other ASR applications (e.g., distances) as well.
[0036] FIG. 5 illustrates an example of a method 500 for automatic speech recognition pre-processing, according to an embodiment. The operations of the method 500 are implemented in electronic hardware, such as that described above or below (e.g., circuits).
[0037] At operation 505, a plurality of audio channels is obtained. In an example, obtaining the plurality of audio channels includes removing reverberation prior to a beam-former processor partitioning the plurality of audio channels into beams.
[0038] At operation 510, the plurality of audio channels are partitioned into beams. In an example, partitioning the plurality of audio channels into beams includes receiving the plurality of audio channels at a beam-former processor, partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels, and providing each partition to a phase-based beam-former.
[0039] At operation 515, a partition corresponding to a beam in the beams is selected based on a noise level. In an example, selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison. In an example, a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams. In an example, a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
[0040] At operation 520, an speech signal is filtered from the selected partition. In an example, the filtering includes applying noise reduction to the audio signal. In an example, the filtering includes applying a spectral matching profile (SPM) to the audio signal. In an example, the SPM is applied after noise reduction is applied to the audio signal.
[0041] In an example, the filtering includes applying an automated gain control to the audio signal. In an example, the automated gain control is applied after a spectral matching profile is applied to the audio signal
[0042] In an example, the method 500 may be extended by optionally performing acoustic echo cancellation to the plurality of audio channels. In an example, the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
[0043] At operation 525, the filtered audio signal is provided to an external entity via an output interface of the pre-processing pipeline.
[0044] FIG. 6 illustrates a block diagram of an example machine 600 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform In alternative embodiments, the machine 600 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 600 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In an example, the machine 600 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 600 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations.
[0045] Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms. Circuitry is a collection of circuits implemented in tangible entities that include hardware (e.g., simple circuits, gates, logic, etc.). Circuitry membership may be flexible over time and underlying hardware variability. Circuitries include members that may, alone or in
combination, perform specified operations when operating. In an example, hardware of the circuitry may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuitry may include variably connected physical components (e.g., execution units, transistors,
simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuitry in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuitry when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuitry. For example, under operation, execution units may be used in a first circuit of a first circuitry at one point in time and reused by a second circuit in the first circuitry, or by a third circuit in a second circuitry at a different time.
[0046] Machine (e.g., computer system) 600 may include a hardware processor 602 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 604 and a static memory 606, some or all of which may communicate with each other via an interlink (e.g., bus) 608. The machine 600 may further include a display unit 610, an alphanumeric input device 612 (e.g., a keyboard), and a user interface (UI) navigation device 614 (e.g., a mouse). In an example, the display unit 610, input device 612 and UI navigation device 614 may be a touch screen display. The machine 600 may additionally include a storage device (e.g., drive unit) 616, a signal generation device 618 (e.g., a speaker), a network interface device 620, and one or more sensors 621 , such as a global positioning system (GPS) sensor, compass, accelero meter, or other sensor. The machine 600 may include an output controller 628, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).
[0047] The storage device 616 may include a machine readable medium 622 on which is stored one or more sets of data structures or instructions 624 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 624 may also reside, completely or at least partially, within the main memory 604, within static memory 606, or within the hardware processor 602 during execution thereof by the machine 600. In an example, one or any combination of the hardware processor 602, the main memory 604, the static memory 606, or the storage device 616 may constitute machine readable media.
[0048] While the machine readable medium 622 is illustrated as a single medium, the term "machine readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) configured to store the one or more instructions 624.
[0049] The term "machine readable medium" may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine 600 and that cause the machine 600 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Non-limiting machine readable medium examples may include solid-state memories, and optical and magnetic media. In an example, a massed machine readable medium comprises a machine readable medium with a plurality of particles having invariant (e.g., rest) mass. Accordingly, massed machine-readable media are not transitory propagating signals. Specific examples of massed machine readable media may include: nonvolatile memory, such as semiconductor memory devices (e.g.. Electrically
Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
[0050] The instructions 624 may further be transmitted or received over a communications network 626 using a transmission medium via the network interface device 620 utilizing any one of a number of transfer protocols (e.g., frame
relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example
communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as Wi-Fi®, IEEE 802.16 family of standards known as WiMax®), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others. In an example, the network interface device 620 may include one or more physical jacks (e.g., Ethernet, coaxial, or phone jacks) or one or more antennas to connect to the communications network 626. In an example, the network interface device 620 may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term "transmission medium" shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 600, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.
Additional Notes & Examples
[0051] Example 1 is a system for automatic speech recognition preprocessing, the system comprising: a sampler to obtain a plurality of audio channels; a de-reverberator to remove reverberations from the plurality of audio channels; a beam-former processor to partition the plurality of audio channels into beams after reverberations are removed; a stream selector to select a partition corresponding to a beam in the beams based on a noise level; a filter to reduce a noise level in a speech signal from the selected partition; and a controller to provide the audio signal to an external entity via an output interface of the pre-processing pipeline.
[0052] In Example 2, the subject matter of Example 1 optionally includes an echo cancelation block disposed between the de-reverberator and the beam-former processor to cancel echoes from the plurality of audio channels after the
reverberations are removed and before the plurality of audio channels are partitioned into beams.
[0053] In Example 3, the subject matter of any one or more of Examples 1-2 optionally include wherein, to partition the plurality of audio channels into beams, the beam-former processor is to: receive the plurality of audio channels; partition the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and provide each partition to a phase-based beam-former.
[0054] In Example 4, the subject matter of any one or more of Examples 1-3 optionally include wherein, to select the partition corresponding to the beam based on the noise level, the stream selector is to: compare speech levels between the beams; and select the beam based on having the highest speech levels determined from the comparison.
[0055] In Example S, the subject matter of any one or more of Examples 1-4 optionally include wherein, to select the partition corresponding to the beam based on the noise level, the stream selector is to: compare noise levels between the beams; and select the beam based on having the lowest noise levels determined from the comparison.
[0056] In Example 6, the subject matter of Example 5 optionally includes wherein the stream selector uses a phrase quality scorer of the stream selector to compare the noise levels between the beams.
[0057] In Example 7, the subject matter of Example 6 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
[0058] In Example 8, the subject matter of any one or more of Examples 1-7 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies noise reduction to the audio signal.
[0059] In Example 9, the subject matter of any one or more of Examples 1-8 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies a spectral profile matching (SPM) to the audio signal.
[0060] In Example 10, the subject matter of Example 9 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
[0061] In Example 11, the subject matter of any one or more of Examples 1- 10 optionally include wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies an automated gain control to the audio signal.
[0062] In Example 12, the subject matter of Example 11 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
[0063] In Example 13, the subject matter of any one or more of Examples 1-
12 optionally include a second filter to perform acoustic echo cancellation to the plurality of audio channels.
[0064] In Example 14, the subject matter of Example 13 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
[0065] Example 1 is at least machine readable medium including instructions for a pre-processing pipeline, the instructions, when executed by a machine, causing the machine to perform operations comprising: obtaining a plurality of audio channels; removing reverberations from the audio channels;
partitioning the plurality of audio channels into beams after reverberations are removed; selecting a partition corresponding to a beam in the beams based on a noise level; filtering an audio signal from the selected partition; and providing the filtered audio signal to an external entity via an output interface of the preprocessing pipeline.
[0066] In Example 16, the subject matter of Example IS optionally includes wherein the operations include canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
[0067] In Example 17, the subject matter of any one or more of Examples 15-16 optionally include wherein the partitioning the plurality of audio channels into beams includes: receiving the plurality of audio channels at a beam-former
processor; partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase-based beam-former.
[0068] In Example 18, the subject matter of any one or more of Examples 15-17 optionally include wherein the selecting the partition corresponding to the beam based on the noise level includes comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
[0069] In Example 19, the subject matter of any one or more of Examples 15-18 optionally include wherein the selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
[0070] In Example 20, the subject matter of Example 19 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
[0071] In Example 21, the subject matter of Example 20 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
[0072] In Example 22, the subject matter of any one or more of Examples
15-21 optionally include wherein the filtering includes applying noise reduction to the audio signal.
[0073] In Example 23, the subject matter of any one or more of Examples
15-22 optionally include wherein the filtering includes applying a spectral profile matching (SPM) to the audio signal.
[0074] In Example 24, the subject matter of Example 23 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
[0075] In Example 25, the subject matter of any one or more of Examples 15-24 optionally include wherein the filtering includes applying an automated gain control to the audio signal.
[0076] In Example 26, the subject matter of Example 25 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
[0077] In Example 27, the subject matter of any one or more of Examples 15-26 optionally include wherein the operations comprise performing acoustic echo cancellation to the plurality of audio channels.
[0078] In Example 28, the subject matter of Example 27 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
[0079] Example 29 is a method for automatic speech recognition preprocessing, the method comprising: obtaining a plurality of audio channels;
removing reverberations from the audio channels; partitioning the plurality of audio channels into beams after the reverberations are removed; selecting a partition corresponding to a beam in the beams based on a noise level; filtering an audio signal from the selected partition; and providing the filtered audio signal to an external entity via an output interface of the pre-processing pipeline.
[0080] In Example 30, the subject matter of Example 29 optionally includes canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
[0081] In Example 31 , the subject matter of any one or more of Examples
29-30 optionally include wherein partitioning the plurality of audio channels into beams includes: receiving the plurality of audio channels at a beam-former processor; partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase-based beam-former.
[0082] In Example 32, the subject matter of any one or more of Examples
29-31 optionally include wherein selecting the partition corresponding to the beam based on the noise level includes comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
[0083] In Example 33, the subject matter of any one or more of Examples
29-32 optionally include wherein selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
[0084] In Example 34, the subject matter of Example 33 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
[0085] In Example 35, the subject matter of Example 34 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
[0086] In Example 36, the subject matter of any one or more of Examples
29-35 optionally include wherein the filtering includes applying noise reduction to the audio signal.
[0087] In Example 37, the subject matter of any one or more of Examples
29-36 optionally include wherein the filtering includes applying a spectral profile matching (SPM) to the audio signal.
[0088] In Example 38, the subject matter of Example 37 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
[0089] In Example 39, the subject matter of any one or more of Examples
29-38 optionally include wherein the filtering includes applying an automated gain control to the audio signal.
[0090] In Example 40, the subject matter of Example 39 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
[0091] In Example 41 , the subject matter of any one or more of Examples
29-40 optionally include performing acoustic echo cancellation to the plurality of audio channels.
[0092] In Example 42, the subject matter of Example 41 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
[0093] Example 43 is a system comprising means to perform any of the methods 29-42.
[0094] Example 44 is at least one machine readable medium including instructions that, when executed by a machine, cause the machine to perform any of the methods 29-42.
[0095] Example 45 is a system for automatic speech recognition preprocessing, the system comprising: means for obtaining a plurality of audio channels; means for removing reverberations from the plurality of audio channels; means for partitioning the plurality of audio channels into beams after the reverberations are removed; means for selecting a partition corresponding to a beam in the beams based on a noise level; means for filtering an audio signal from the selected partition; and means for providing the filtered audio signal to an external entity via an output interface of the pre-processing pipeline.
[0096] In Example 46, the subject matter of Example 45 optionally includes means for canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
[0097] In Example 47, the subject matter of any one or more of Examples
45-46 optionally include wherein the means for partitioning the plurality of audio channels into beams includes: means for receiving the plurality of audio channels at a beam-former processor; means for partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and providing each partition to a phase- based beam-former.
[0098] In Example 48, the subject matter of any one or more of Examples
45-47 optionally include wherein the means for selecting the partition
corresponding to the beam based on the noise level includes means for comparing
speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
[0099] In Example 49, the subject matter of any one or more of Examples
45-48 optionally include wherein the means for selecting the partition
corresponding to the beam based on the noise level includes means for comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
[0100] In Example SO, the subject matter of Example 49 optionally includes wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
[0101] In Example 51 , the subject matter of Example 50 optionally includes wherein a signal-to-noise (SNR) meter of the stream selector provides a noise level for each beam.
[0102] In Example 52, the subject matter of any one or more of Examples 45-51 optionally include wherein the means for filtering includes means for applying noise reduction to the audio signal.
[0103] In Example 53, the subject matter of any one or more of Examples
45-52 optionally include wherein the means for filtering includes means for applying a spectral profile matching (SPM) to the audio signal.
[0104] In Example 54, the subject matter of Example 53 optionally includes wherein the spectral profile matching is applied after noise reduction is applied to the audio signal.
[0105] In Example 55, the subject matter of any one or more of Examples
45-54 optionally include wherein the means for filtering includes means for applying an automated gain control to the audio signal.
[0106] In Example 56, the subject matter of Example 55 optionally includes wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
[0107] In Example 57, the subject matter of any one or more of Examples 45-56 optionally include means for performing acoustic echo cancellation to the plurality of audio channels.
[0108] In Example 58, the subject matter of Example 57 optionally includes wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
[0109] The above detailed description includes references to the
accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments that may be practiced. These embodiments are also referred to herein as "examples." Such examples may include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.
[0110] All publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated reference(s) should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.
[0111] In this document, the terms "a" or "an" are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of "at least one" or "one or more." In this document, the term "or" is used to refer to a nonexclusive or, such that "A or B" includes "A but not B," "B but not A," and "A and B," unless otherwise indicated. In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Also, in the following claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the
following claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.
[0112] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments may be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is to allow the reader to quickly ascertain the nature of the technical disclosure and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment. The scope of the embodiments should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.
Claims
What is claimed is
1. A system for automatic speech recognition pre-processing, the system comprising:
a sampler to obtain a plurality of audio channels;
a de-reverberator to remove reverberations from the plurality of audio channels;
a beam-former processor to partition the plurality of audio channels into beams after reverberations are removed;
a stream selector to select a partition corresponding to a beam in the beams based on a noise level;
a filter to reduce a noise level in a speech signal from the selected partition; and
a controller to provide the audio signal to an external entity via an output interface of the pre-processing pipeline.
2. The system of claim 1, comprising an echo cancelation block disposed between the de-reverberator and the beam-former processor to cancel echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
3. The system of claim 1 , wherein, to partition the plurality of audio channels into beams, the beam-former processor is to:
receive the plurality of audio channels;
partition the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and
provide each partition to a phase-based beam-former. 4. The system of claim 1, wherein, to select the partition corresponding to the beam based on the noise level, the stream selector is to:
compare noise levels between the beams; and
select the beam based on having the lowest noise levels determined from the comparison. S. The system of claim 4, wherein the stream selector uses a phrase quality scorer of the stream selector to compare the noise levels between the beams.
6. The system of claim 1, wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies noise reduction to the audio signal
7. The system of claim 1, wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies a spectral profile matching (SPM) to the audio signal
8. The system of claim 7, wherein the spectral profile matching is applied after noise reduction is applied to the audio signal
9. The system of claim 1, wherein, to reduce the noise level in the speech signal from the selected partition, the filter applies an automated gain control to the audio signal.
10. The system of claim 9, wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
11. A method for automatic speech recognition pre-processing, the method comprising:
obtaining a plurality of audio channels;
removing reverberations from the audio channels;
partitioning the plurality of audio channels into beams after the
reverberations are removed;
selecting a partition corresponding to a beam in the beams based on a noise level;
filtering an audio signal from the selected partition; and
providing the filtered audio signal to an external entity via an output interface of the pre-processing pipeline.
12. The method of claim 11, comprising canceling echoes from the plurality of audio channels after the reverberations are removed and before the plurality of audio channels are partitioned into beams.
13. The method of claim 11 , wherein partitioning the plurality of audio channels into beams includes:
receiving the plurality of audio channels at a beam-former processor;
partitioning the plurality of audio channels into partitions of two audio channels based on a relationship between microphones producing the plurality of audio channels; and
providing each partition to a phase-based beam-former.
14. The method of claim 11, wherein selecting the partition corresponding to the beam based on the noise level includes comparing speech levels between the beams and selecting the beam based on having the highest speech levels determined from the comparison.
15. The method of claim 11, wherein selecting the partition corresponding to the beam based on the noise level includes comparing noise levels between the beams and selecting the beam based on having the lowest noise levels determined from the comparison.
16. The method of claim IS, wherein a phrase quality scorer of a stream selector performing the partition selection compares the noise levels between the beams.
17. The method of claim 11, wherein the filtering includes applying noise reduction to the audio signal.
18. The method of claim 11, wherein the filtering includes applying a spectral profile matching (SPM) to the audio signal.
19. The method of claim 18, wherein the spectral profile matching is applied after noise reduction is applied to the audio signal. 20. The method of claim 11, wherein the filtering includes applying an automated gain control to the audio signal.
21. The method of claim 20, wherein the automated gain control is applied after a spectral profile matching is applied to the audio signal.
22. The method of claim 11 , comprising performing acoustic echo cancellation to the plurality of audio channels.
23. The method of claim 22, wherein the acoustic echo cancellation is performed prior to partitioning the plurality of audio channels into beams.
24. A system comprising means to perform any of the methods 11-23.
25. At least one machine readable medium including instructions that, when executed by a machine, cause the machine to perform any of the methods 11-23.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP17813753.5A EP3472834A4 (en) | 2016-06-15 | 2017-05-16 | Far field automatic speech recognition pre-processing |
CN201780029587.0A CN109074816B (en) | 2016-06-15 | 2017-05-16 | Far field automatic speech recognition preprocessing |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201662350507P | 2016-06-15 | 2016-06-15 | |
US62/350,507 | 2016-06-15 | ||
US15/388,147 | 2016-12-22 | ||
US15/388,147 US20170365255A1 (en) | 2016-06-15 | 2016-12-22 | Far field automatic speech recognition pre-processing |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2017218128A1 true WO2017218128A1 (en) | 2017-12-21 |
Family
ID=60659998
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2017/032932 WO2017218129A1 (en) | 2016-06-15 | 2017-05-16 | Automatic speech recognition de-reverberation |
PCT/US2017/032913 WO2017218128A1 (en) | 2016-06-15 | 2017-05-16 | Far field automatic speech recognition pre-processing |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2017/032932 WO2017218129A1 (en) | 2016-06-15 | 2017-05-16 | Automatic speech recognition de-reverberation |
Country Status (4)
Country | Link |
---|---|
US (4) | US20170366897A1 (en) |
EP (1) | EP3472834A4 (en) |
CN (1) | CN109074816B (en) |
WO (2) | WO2017218129A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109597022A (en) * | 2018-11-30 | 2019-04-09 | 腾讯科技(深圳)有限公司 | The operation of sound bearing angle, the method, apparatus and equipment for positioning target audio |
US10657983B2 (en) | 2016-06-15 | 2020-05-19 | Intel Corporation | Automatic gain control for speech recognition |
Families Citing this family (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107290711A (en) * | 2016-03-30 | 2017-10-24 | 芋头科技(杭州)有限公司 | A kind of voice is sought to system and method |
US11037330B2 (en) * | 2017-04-08 | 2021-06-15 | Intel Corporation | Low rank matrix compression |
US10403299B2 (en) * | 2017-06-02 | 2019-09-03 | Apple Inc. | Multi-channel speech signal enhancement for robust voice trigger detection and automatic speech recognition |
CN109979476B (en) * | 2017-12-28 | 2021-05-14 | 电信科学技术研究院 | Method and device for removing reverberation of voice |
USD920137S1 (en) * | 2018-03-07 | 2021-05-25 | Intel Corporation | Acoustic imaging device |
US10313786B1 (en) * | 2018-03-20 | 2019-06-04 | Cisco Technology, Inc. | Beamforming and gainsharing mixing of small circular array of bidirectional microphones |
US20190324117A1 (en) * | 2018-04-24 | 2019-10-24 | Mediatek Inc. | Content aware audio source localization |
US10586538B2 (en) * | 2018-04-25 | 2020-03-10 | Comcast Cable Comminications, LLC | Microphone array beamforming control |
US10573301B2 (en) | 2018-05-18 | 2020-02-25 | Intel Corporation | Neural network based time-frequency mask estimation and beamforming for speech pre-processing |
US10667071B2 (en) * | 2018-05-31 | 2020-05-26 | Harman International Industries, Incorporated | Low complexity multi-channel smart loudspeaker with voice control |
US11902758B2 (en) * | 2018-12-21 | 2024-02-13 | Gn Audio A/S | Method of compensating a processed audio signal |
CN109524004B (en) * | 2018-12-29 | 2022-03-08 | 思必驰科技股份有限公司 | Method for realizing parallel transmission of multi-channel audio and data, external voice interaction device and system |
CN109767769B (en) | 2019-02-21 | 2020-12-22 | 珠海格力电器股份有限公司 | Voice recognition method and device, storage medium and air conditioner |
GB201902812D0 (en) * | 2019-03-01 | 2019-04-17 | Nokia Technologies Oy | Wind noise reduction in parametric audio |
CN110310655B (en) * | 2019-04-22 | 2021-10-22 | 广州视源电子科技股份有限公司 | Microphone signal processing method, device, equipment and storage medium |
KR20200132613A (en) | 2019-05-16 | 2020-11-25 | 삼성전자주식회사 | Method and apparatus for speech recognition with wake on voice |
KR20210017252A (en) | 2019-08-07 | 2021-02-17 | 삼성전자주식회사 | Method for processing audio sound based on multi-channel and an electronic device |
CN111341345B (en) * | 2020-05-21 | 2021-04-02 | 深圳市友杰智新科技有限公司 | Control method and device of voice equipment, voice equipment and storage medium |
CN112887877B (en) * | 2021-01-28 | 2023-09-08 | 歌尔科技有限公司 | Audio parameter setting method and device, electronic equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080292108A1 (en) * | 2006-08-01 | 2008-11-27 | Markus Buck | Dereverberation system for use in a signal processing apparatus |
US20110158418A1 (en) * | 2009-12-25 | 2011-06-30 | National Chiao Tung University | Dereverberation and noise reduction method for microphone array and apparatus using the same |
US20120106749A1 (en) * | 2003-05-13 | 2012-05-03 | Nuance Communications, Inc. | Microphone Non-Uniformity Compensation System |
WO2016056683A1 (en) * | 2014-10-07 | 2016-04-14 | 삼성전자 주식회사 | Electronic device and reverberation removal method therefor |
Family Cites Families (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH07109560B2 (en) * | 1990-11-30 | 1995-11-22 | 富士通テン株式会社 | Voice recognizer |
US6453289B1 (en) * | 1998-07-24 | 2002-09-17 | Hughes Electronics Corporation | Method of noise reduction for speech codecs |
US6314396B1 (en) * | 1998-11-06 | 2001-11-06 | International Business Machines Corporation | Automatic gain control in a speech recognition system |
JP3180786B2 (en) * | 1998-11-27 | 2001-06-25 | 日本電気株式会社 | Audio encoding method and audio encoding device |
US6314394B1 (en) * | 1999-05-27 | 2001-11-06 | Lear Corporation | Adaptive signal separation system and method |
US6122331A (en) * | 1999-06-14 | 2000-09-19 | Atmel Corporation | Digital automatic gain control |
KR100304666B1 (en) * | 1999-08-28 | 2001-11-01 | 윤종용 | Speech enhancement method |
US6636829B1 (en) * | 1999-09-22 | 2003-10-21 | Mindspeed Technologies, Inc. | Speech communication system and method for handling lost frames |
JP2004537233A (en) | 2001-07-20 | 2004-12-09 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Acoustic reinforcement system with echo suppression circuit and loudspeaker beamformer |
KR20040019362A (en) | 2001-07-20 | 2004-03-05 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Sound reinforcement system having an multi microphone echo suppressor as post processor |
JP3984842B2 (en) * | 2002-03-12 | 2007-10-03 | 松下電器産業株式会社 | Howling control device |
DK174558B1 (en) | 2002-03-15 | 2003-06-02 | Bruel & Kjaer Sound & Vibratio | Transducers two-dimensional array, has set of sub arrays of microphones in circularly symmetric arrangement around common center, each sub-array with three microphones arranged in straight line |
CA2388439A1 (en) * | 2002-05-31 | 2003-11-30 | Voiceage Corporation | A method and device for efficient frame erasure concealment in linear predictive based speech codecs |
US6798380B2 (en) * | 2003-02-05 | 2004-09-28 | University Of Florida Research Foundation, Inc. | Robust capon beamforming |
US7039200B2 (en) * | 2003-03-31 | 2006-05-02 | Microsoft Corporation | System and process for time delay estimation in the presence of correlated noise and reverberation |
US7415117B2 (en) | 2004-03-02 | 2008-08-19 | Microsoft Corporation | System and method for beamforming using a microphone array |
JP2005333180A (en) | 2004-05-18 | 2005-12-02 | Audio Technica Corp | Boundary microphone |
WO2009009568A2 (en) * | 2007-07-09 | 2009-01-15 | Mh Acoustics, Llc | Augmented elliptical microphone array |
EP1965603B1 (en) | 2005-12-19 | 2017-01-11 | Yamaha Corporation | Sound emission and collection device |
DE602007011594D1 (en) * | 2006-04-27 | 2011-02-10 | Dolby Lab Licensing Corp | SOUND AMPLIFICATION WITH RECORDING OF PUBLIC EVENTS ON THE BASIS OF SPECIFIC VOLUME |
CN101192862B (en) * | 2006-11-30 | 2013-01-16 | 昂达博思公司 | Method and device for automatic gain control in wireless communication system |
GB0703275D0 (en) * | 2007-02-20 | 2007-03-28 | Skype Ltd | Method of estimating noise levels in a communication system |
US8488803B2 (en) | 2007-05-25 | 2013-07-16 | Aliphcom | Wind suppression/replacement component for use with electronic systems |
KR100905586B1 (en) * | 2007-05-28 | 2009-07-02 | 삼성전자주식회사 | System and method of estimating microphone performance for recognizing remote voice in robot |
JP5075664B2 (en) * | 2008-02-15 | 2012-11-21 | 株式会社東芝 | Spoken dialogue apparatus and support method |
JP5102165B2 (en) | 2008-09-22 | 2012-12-19 | 株式会社デンソー | Radar equipment |
US8032236B2 (en) * | 2008-09-30 | 2011-10-04 | Rockwell Automation Technologies, Inc. | Asymetrical process parameter control system and method |
CN101986386B (en) * | 2009-07-29 | 2012-09-26 | 比亚迪股份有限公司 | Method and device for eliminating voice background noise |
US20110188671A1 (en) * | 2009-10-15 | 2011-08-04 | Georgia Tech Research Corporation | Adaptive gain control based on signal-to-noise ratio for noise suppression |
JP5423370B2 (en) | 2009-12-10 | 2014-02-19 | 船井電機株式会社 | Sound source exploration device |
US8861756B2 (en) * | 2010-09-24 | 2014-10-14 | LI Creative Technologies, Inc. | Microphone array system |
US8798278B2 (en) * | 2010-09-28 | 2014-08-05 | Bose Corporation | Dynamic gain adjustment based on signal to ambient noise level |
US9100734B2 (en) | 2010-10-22 | 2015-08-04 | Qualcomm Incorporated | Systems, methods, apparatus, and computer-readable media for far-field multi-source tracking and separation |
KR20120128542A (en) | 2011-05-11 | 2012-11-27 | 삼성전자주식회사 | Method and apparatus for processing multi-channel de-correlation for cancelling multi-channel acoustic echo |
FR2976710B1 (en) | 2011-06-20 | 2013-07-05 | Parrot | DEBRISING METHOD FOR MULTI-MICROPHONE AUDIO EQUIPMENT, IN PARTICULAR FOR A HANDS-FREE TELEPHONY SYSTEM |
WO2013166080A1 (en) * | 2012-04-30 | 2013-11-07 | Creative Technology Ltd | A universal reconfigurable echo cancellation system |
IN2015DN00484A (en) | 2012-07-27 | 2015-06-26 | Sony Corp | |
US9584642B2 (en) * | 2013-03-12 | 2017-02-28 | Google Technology Holdings LLC | Apparatus with adaptive acoustic echo control for speakerphone mode |
US9860439B2 (en) * | 2013-02-15 | 2018-01-02 | Panasonic Intellectual Property Management Co., Ltd. | Directionality control system, calibration method, horizontal deviation angle computation method, and directionality control method |
US9338551B2 (en) * | 2013-03-15 | 2016-05-10 | Broadcom Corporation | Multi-microphone source tracking and noise suppression |
US9449594B2 (en) | 2013-09-17 | 2016-09-20 | Intel Corporation | Adaptive phase difference based noise reduction for automatic speech recognition (ASR) |
EP2866465B1 (en) * | 2013-10-25 | 2020-07-22 | Harman Becker Automotive Systems GmbH | Spherical microphone array |
US9571930B2 (en) * | 2013-12-24 | 2017-02-14 | Intel Corporation | Audio data detection with a computing device |
US9124234B1 (en) * | 2014-04-11 | 2015-09-01 | Entropic Communications, LLC. | Method and apparatus for adaptive automatic gain control |
JP2016169358A (en) * | 2014-07-24 | 2016-09-23 | セントラル硝子株式会社 | Curable silicone resin composition and cured product of the same, and optical semiconductor device using them |
US9456276B1 (en) | 2014-09-30 | 2016-09-27 | Amazon Technologies, Inc. | Parameter selection for audio beamforming |
US20160150315A1 (en) * | 2014-11-20 | 2016-05-26 | GM Global Technology Operations LLC | System and method for echo cancellation |
US9390723B1 (en) * | 2014-12-11 | 2016-07-12 | Amazon Technologies, Inc. | Efficient dereverberation in networked audio systems |
WO2016098315A1 (en) * | 2014-12-15 | 2016-06-23 | パナソニックIpマネジメント株式会社 | Microphone array, monitoring system, and sound pickup setting method |
US9800279B2 (en) * | 2015-02-16 | 2017-10-24 | Samsung Electronics Co., Ltd. | Method and apparatus for automatic gain control in wireless receiver |
CN104810021B (en) * | 2015-05-11 | 2017-08-18 | 百度在线网络技术(北京)有限公司 | The pre-treating method and device recognized applied to far field |
US9843851B2 (en) * | 2015-05-22 | 2017-12-12 | Amazon Technologies, Inc. | Portable speaker system |
US10028051B2 (en) | 2015-08-31 | 2018-07-17 | Panasonic Intellectual Property Management Co., Ltd. | Sound source localization apparatus |
CN105355210B (en) * | 2015-10-30 | 2020-06-23 | 百度在线网络技术(北京)有限公司 | Preprocessing method and device for far-field speech recognition |
US9754605B1 (en) * | 2016-06-09 | 2017-09-05 | Amazon Technologies, Inc. | Step-size control for multi-channel acoustic echo canceller |
US20170366897A1 (en) * | 2016-06-15 | 2017-12-21 | Robert Azarewicz | Microphone board for far field automatic speech recognition |
-
2016
- 2016-12-22 US US15/388,275 patent/US20170366897A1/en not_active Abandoned
- 2016-12-22 US US15/388,147 patent/US20170365255A1/en not_active Abandoned
- 2016-12-22 US US15/388,107 patent/US10657983B2/en not_active Expired - Fee Related
- 2016-12-22 US US15/388,323 patent/US20170365271A1/en not_active Abandoned
-
2017
- 2017-05-16 WO PCT/US2017/032932 patent/WO2017218129A1/en active Application Filing
- 2017-05-16 WO PCT/US2017/032913 patent/WO2017218128A1/en unknown
- 2017-05-16 CN CN201780029587.0A patent/CN109074816B/en active Active
- 2017-05-16 EP EP17813753.5A patent/EP3472834A4/en not_active Withdrawn
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120106749A1 (en) * | 2003-05-13 | 2012-05-03 | Nuance Communications, Inc. | Microphone Non-Uniformity Compensation System |
US20080292108A1 (en) * | 2006-08-01 | 2008-11-27 | Markus Buck | Dereverberation system for use in a signal processing apparatus |
US20110158418A1 (en) * | 2009-12-25 | 2011-06-30 | National Chiao Tung University | Dereverberation and noise reduction method for microphone array and apparatus using the same |
WO2016056683A1 (en) * | 2014-10-07 | 2016-04-14 | 삼성전자 주식회사 | Electronic device and reverberation removal method therefor |
Non-Patent Citations (2)
Title |
---|
ARMIN SEHR ET AL.: "Towards Robust Distant-Talking Automatic Speech Recognition in Reverberant Environments", SPEECH AND AUDIO PROCESSING IN ADVERSE ENVIRONMENTS, 2008, pages 679 - 728, XP009514718 * |
See also references of EP3472834A4 * |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10657983B2 (en) | 2016-06-15 | 2020-05-19 | Intel Corporation | Automatic gain control for speech recognition |
CN109597022A (en) * | 2018-11-30 | 2019-04-09 | 腾讯科技(深圳)有限公司 | The operation of sound bearing angle, the method, apparatus and equipment for positioning target audio |
EP3822654A4 (en) * | 2018-11-30 | 2021-10-13 | Tencent Technology (Shenzhen) Company Limited | Audio recognition method, and target audio positioning method, apparatus and device |
CN109597022B (en) * | 2018-11-30 | 2023-02-17 | 腾讯科技(深圳)有限公司 | Method, device and equipment for calculating azimuth angle of sound source and positioning target audio |
US11967316B2 (en) | 2018-11-30 | 2024-04-23 | Tencent Technology (Shenzhen) Company Limited | Audio recognition method, method, apparatus for positioning target audio, and device |
Also Published As
Publication number | Publication date |
---|---|
WO2017218129A1 (en) | 2017-12-21 |
CN109074816B (en) | 2023-11-28 |
US10657983B2 (en) | 2020-05-19 |
US20170365255A1 (en) | 2017-12-21 |
US20170365274A1 (en) | 2017-12-21 |
US20170365271A1 (en) | 2017-12-21 |
US20170366897A1 (en) | 2017-12-21 |
CN109074816A (en) | 2018-12-21 |
EP3472834A1 (en) | 2019-04-24 |
EP3472834A4 (en) | 2020-02-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20170365255A1 (en) | Far field automatic speech recognition pre-processing | |
JP7434137B2 (en) | Speech recognition method, device, equipment and computer readable storage medium | |
US10546593B2 (en) | Deep learning driven multi-channel filtering for speech enhancement | |
Wang et al. | Deep learning based target cancellation for speech dereverberation | |
CN107221336B (en) | Device and method for enhancing target voice | |
Schwarz et al. | Coherent-to-diffuse power ratio estimation for dereverberation | |
CN102324237B (en) | Microphone-array speech-beam forming method as well as speech-signal processing device and system | |
CN112017681B (en) | Method and system for enhancing directional voice | |
CN108447496B (en) | Speech enhancement method and device based on microphone array | |
CN102164328A (en) | Audio input system used in home environment based on microphone array | |
AU2011334840A1 (en) | Apparatus and method for spatially selective sound acquisition by acoustic triangulation | |
CN110610718B (en) | Method and device for extracting expected sound source voice signal | |
Papp et al. | Hands-free voice communication with TV | |
US11240621B2 (en) | Three-dimensional audio systems | |
CN113470685A (en) | Training method and device of voice enhancement model and voice enhancement method and device | |
CN106161820B (en) | A kind of interchannel decorrelation method for stereo acoustic echo canceler | |
Maas et al. | A two-channel acoustic front-end for robust automatic speech recognition in noisy and reverberant environments | |
Koldovský et al. | Noise reduction in dual-microphone mobile phones using a bank of pre-measured target-cancellation filters | |
CN110169082B (en) | Method and apparatus for combining audio signal outputs, and computer readable medium | |
Zwyssig et al. | Recognition of overlapping speech using digital MEMS microphone arrays | |
Liu et al. | Head‐related transfer function–reserved time‐frequency masking for robust binaural sound source localization | |
Kim | Hearing aid speech enhancement using phase difference-controlled dual-microphone generalized sidelobe canceller | |
US11528571B1 (en) | Microphone occlusion detection | |
Geng et al. | A speech enhancement method based on the combination of microphone array and parabolic reflector | |
CN112731291B (en) | Binaural sound source localization method and system for collaborative two-channel time-frequency mask estimation task learning |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17813753 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2017813753 Country of ref document: EP Effective date: 20190115 |