WO2011071484A1 - Guest speaker robust adapted speech recognition - Google Patents
Guest speaker robust adapted speech recognition Download PDFInfo
- Publication number
- WO2011071484A1 WO2011071484A1 PCT/US2009/067055 US2009067055W WO2011071484A1 WO 2011071484 A1 WO2011071484 A1 WO 2011071484A1 US 2009067055 W US2009067055 W US 2009067055W WO 2011071484 A1 WO2011071484 A1 WO 2011071484A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- recognition
- speech
- pass
- generic
- class
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 50
- 230000008569 process Effects 0.000 claims abstract description 17
- 239000013598 vector Substances 0.000 claims abstract description 9
- 230000001419 dependent effect Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 2
- 238000011867 re-evaluation Methods 0.000 claims 1
- 230000006978 adaptation Effects 0.000 description 11
- 238000010606 normalization Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000013518 transcription Methods 0.000 description 1
- 230000035897 transcription Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/065—Adaptation
- G10L15/07—Adaptation to the speaker
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
- G10L15/32—Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
Definitions
- the invention generally relates to computer processor implemented systems for automatic speech recognition (ASR), and more specifically to adaptation of ASR systems.
- ASR automatic speech recognition
- An automatic speech recognition (ASR) system tries to determine a
- each frame can be thought of as a multi-dimensional vector that represents various characteristics of the speech signal present during a short time window of the speech.
- variable numbers of frames are organized as "utterances" representing a period of speech followed by a pause which in real life loosely corresponds to a spoken sentence or phrase.
- the system compares the input utterances to find acoustic models that best match the vector sequence characteristics and determines corresponding representative text associated with the acoustic models.
- Modern acoustic models typically use state sequence models such as Hidden Markov Models that model speech sounds (usually phonemes) using mixtures of probability distribution functions, typically Gaussians.
- Phoneme models often represent phonemes in specific contexts, referred to as context-dependent phonemes, e.g. triphones or phonemes with known left and/or right contexts.
- State sequence models can be scaled up to represent words as connected sequences of acoustically modeled phonemes, and phrases or sentences as connected sequences of words.
- the models are organized together as words, phrases, and sentences, additional language-related information is also typically incorporated into the models in the form of language modeling.
- a system may produce a single best recognition candidate - the recognition result - or multiple recognition hypotheses in various forms such as an N-best list, a recognition lattice, or a confusion network. Further details regarding continuous speech recognition are provided in U.S. Patent No. 5,794, 189, entitled “Continuous Speech Recognition,” and U.S. Patent No. 6,167,377, entitled “Speech Recognition Language Models,” the contents of which are incorporated herein by reference.
- Some ASR arrangements are generic to a broad class of multiple speakers and input channels (e.g. microphone type) and acoustic environments, which is referred to as speaker independent (SI) and channel/environment independent speech recognition.
- SI speaker independent
- Such systems can be specialized to a specific speaker, speaker group, speaking style, channel or environment by means of adaptation.
- an SI arrangement can be adapted for a specific individual speaker to create a speaker dependent (SD) arrangement.
- SD speaker dependent
- the acoustic model component of the recognizer is adapted, but some language model adaptation techniques also have been proposed and successfully deployed.
- Embodiments of the present invention are directed to a method for speech recognition which is implemented in the specific form of computer processes that function in a computer processor. That is, one or more computer processes: process a speech input to produce a sequence of representative speech vectors and perform multiple recognition passes to determine a recognition output corresponding to the speech input. At least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech. And at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech.
- the generic recognition pass and the adapted recognition pass may be performed in parallel. Or, the generic recognition pass and the adapted recognition pass may be performed in series. In the latter case, the adapted recognition pass may be performed before the generic recognition pass or vice versa.
- the output of the first recognition pass may be a confusion network of recognition hypotheses, or a lattice or N-best list of recognition hypotheses that the following pass reevaluates.
- the generic recognition pass may use speaker independent recognition and the adapted recognition pass may use speaker dependent recognition.
- the final recognition result may be the result of the last recognition pass in the series; it may also be a system combination result incorporating results from intermediate passes.
- the specific sub-class may be a specific prime user of the speech recognition method such that the adapted recognition pass uses speaker dependent recognition and the generic recognition pass uses speaker independent recognition.
- the specific sub-class may be a specific speaker group, a specific speaking style, or a specific speech channel.
- the speech adapted arrangement may be based on adapted feature transforms so that the multiple recognition passes may use the same acoustic models.
- Embodiments of the present invention also include a speech recognition engine implemented as a computer processes in a computer processor and using a method according to any of the arrangements described above. And embodiments also include a computer program product implemented in a computer readable storage medium for speech recognition which includes program code for performing a method according to any of the arrangements described above.
- Figure 1 is a block diagram representing various hardware components of an embodiment of a speech recognition system according to the present invention.
- Figure 2 shows functional blocks in a series multi-pass recognition arrangement according to one specific embodiment of the present invention.
- Various embodiments of the present invention are directed to techniques for the combined application of adapted and un-adapted speech recognition arrangements to improve recognition performance for guest speakers using a speaker adapted recognition arrangement adapted for another main user. More broadly, this can be thought of as improving recognition performance for a broad general class of input speech in a recognizer that is pre-adapted to a specific sub-class of the general class.
- the specific sub-class may be a specific prime user of the speech recognition method such that the adapted recognition pass uses speaker dependent recognition and the generic recognition pass uses speaker independent recognition.
- the specific sub-class may be a specific speaker group, a specific speaking style, or a specific speech channel.
- FIG. 1 shows various hardware components of an embodiment of an ASR system according to the present invention.
- a computer system 10 includes a speech input microphone 11 which is connected through a suitable preamplifier 13 to an analog-to- digital (A/D) converter 15.
- a front-end speech pre-processor 17 typically performs a Fourier transform so as to extract spectral features to characterize the input speech as a sequence of representative multi-dimensional vectors.
- a speech recognition processor 12, e.g., an Intel Core 2 processor or the like, is programmed to run one or more specialized computer software processes to determine a recognition output corresponding to the speech input.
- processor memory 120 e.g., random access memory (RAM) and/or read-only memory (ROM) stores the speech processing software routines, the speech recognition models and data for use by the speech recognition processor 12.
- the recognition output may be displayed, for example, as representative text on computer workstation display 14.
- Such a computer workstation would also typically include a keyboard 16 and a mouse 18 for user interaction with the system 10.
- ASR implemented for a mobile device such as a cell phone
- ASR for the cabin of an automobile
- client-server based ASR etc.
- a method for speech recognition is implemented in the specific form of computer processes that function in a computer processing environment such as described in the preceding paragraph.
- one or more computer processes operate in an ASR system to process a speech input to produce a sequence of representative speech vectors. Multiple recognition passes are performed on the vector sequence to determine a recognition output that corresponds to the speech input. Of these multiple recognition passes, at least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech. And at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech.
- the multi-pass recognition arrangement provides a speech recognition system that is pre-adapted for improved recognition performance for a specific prime or main user, while having only minor degradation for general speakers ("guest speakers”) despite the adaptation to the prime user.
- Figure 2 shows an example of a specific series multi-pass recognition arrangement where a first recognition pass is initially performed on the speech input using of pre-adapted speaker dependent (SD) acoustic models to produce intermediate results of recognition hypotheses.
- SD speaker dependent
- these recognition hypotheses may be in the specific form of a confusion network, a recognition lattice, or an N-best list.
- SI generic speaker independent
- subsequent recognition passes can use the output of the previous pass for adaptation and/or constraining the search space.
- the recognition passes may partially use a speaker-independent (SI) configuration and partially a speaker-dependent (SD) configuration. Or, some embodiments may select between SI and SD configurations based on model matching criteria.
- the re-scoring pass can improve the final recognition result for many different reasons. It is possible to apply different, higher resolution acoustic models than in the first pass (e.g., a selected gender-dependent model, higher dimensional feature space, etc.).
- the speech adapted SD arrangement may be based on adapted feature transforms so that the multiple recognition passes may use the same acoustic models.
- batch-mode feature- and channel-normalization schemes may be used such as batch-mode Channel (or Cepstral) Mean/Variance Normalization (CMN/CVN), batch- mode spectral subtraction, and other mean and variance normalization schemes.
- first and second pass model complementariness contributes to improving the final recognition result.
- An SD recognition system can be initially constructed and adapted from the start to a specific speaker.
- an arrangement that starts with SI models can be adapted to become SD models based on either off-line or on-line adaptation over time. And either batch or incremental adaptation techniques may be applicable.
- the generic recognition pass and the adapted recognition pass may be performed in parallel rather than in series.
- the decoding searches can be mutually pruned or results can be combined by means of system combination such as ROVER (Recognizer Output Voting for Error Reduction), see e.g., J. G. Fiscus, A Post-Processing System To Yield Reduced Word Error Rates : Recognizer Output Voting Error Reduction (ROVER), In Proc. ASRU, pages 347-354, 1997; incorporated herein by reference, or CNC
- ROVER Recognition Output Voting for Error Reduction
- Embodiments of the present invention also include a speech recognition engine implemented as a computer processes in a computer processor and using a method according to any of the arrangements described above. And embodiments also include a computer program product implemented in a computer readable storage medium for speech recognition which includes program code for performing a method according to any of the arrangements described above.
- Embodiments of the invention may be implemented using any conventional computer programming language. For example, preferred embodiments may be implemented in a procedural programming language (e.g., "C") or an object oriented programming language (e.g. , "C++", Python). Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.
- a procedural programming language e.g., "C”
- object oriented programming language e.g. , "C++”, Python
- Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.
- Embodiments can be implemented as a computer program product for use with a computer system.
- Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette, CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium.
- the medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques).
- the series of computer instructions embodies all or part of the functionality previously described herein with respect to the system.
- Such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g. , a computer program product) and hardware. Still other embodiments of the invention are implemented as entirely hardware, or entirely software (e.g., a computer program product).
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Machine Translation (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A method for speech recognition is implemented in the specific form of computer processes that function in a computer processor. That is, one or more computer processes: process a speech input to produce a sequence of representative speech vectors and perform multiple recognition passes to determine a recognition output corresponding to the speech input. At least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech. And at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech.
Description
TITLE
Guest Speaker Robust Adapted Speech Recognition
FIELD OF THE INVENTION
[0001] The invention generally relates to computer processor implemented systems for automatic speech recognition (ASR), and more specifically to adaptation of ASR systems.
BACKGROUND ART
[0002] An automatic speech recognition (ASR) system tries to determine a
representative meaning (e.g., text) corresponding to input speech. Typically, the input speech is processed into a sequence of digital frames. Each frame can be thought of as a multi-dimensional vector that represents various characteristics of the speech signal present during a short time window of the speech. In a continuous recognition system, variable numbers of frames are organized as "utterances" representing a period of speech followed by a pause which in real life loosely corresponds to a spoken sentence or phrase.
[0003] The system compares the input utterances to find acoustic models that best match the vector sequence characteristics and determines corresponding representative text associated with the acoustic models. Modern acoustic models typically use state sequence models such as Hidden Markov Models that model speech sounds (usually phonemes) using mixtures of probability distribution functions, typically Gaussians. Phoneme models often represent phonemes in specific contexts, referred to as context-dependent phonemes, e.g. triphones or phonemes with known left and/or right contexts.
[0004] State sequence models can be scaled up to represent words as connected sequences of acoustically modeled phonemes, and phrases or sentences as connected sequences of words. When the models are organized together as words, phrases, and sentences, additional language-related information is also typically incorporated into the models in the form of language modeling.
[0005] The words or phrases associated with the best matching model structures are referred to as recognition candidates or hypotheses. A system may produce a single best
recognition candidate - the recognition result - or multiple recognition hypotheses in various forms such as an N-best list, a recognition lattice, or a confusion network. Further details regarding continuous speech recognition are provided in U.S. Patent No. 5,794, 189, entitled "Continuous Speech Recognition," and U.S. Patent No. 6,167,377, entitled "Speech Recognition Language Models," the contents of which are incorporated herein by reference.
[0006] Some ASR arrangements are generic to a broad class of multiple speakers and input channels (e.g. microphone type) and acoustic environments, which is referred to as speaker independent (SI) and channel/environment independent speech recognition. Such systems can be specialized to a specific speaker, speaker group, speaking style, channel or environment by means of adaptation. For example, an SI arrangement can be adapted for a specific individual speaker to create a speaker dependent (SD) arrangement. Typically, the acoustic model component of the recognizer is adapted, but some language model adaptation techniques also have been proposed and successfully deployed.
[0007] It is well known, however, that the performance of an adapted speech recognizer degrades whenever there is a mismatch between the adaptation data (speaker, speaker group, channel, speaking style, acoustic environment) that the recognizer has been adapted to and the input speech data that it actually faces in application. An adapted SD
recognizer, for example, strongly degrades in performance when being applied on a different speaker than the one it was adapted to.
[0008] In many speech recognition applications such as mobile handset ASR or server- based speech recognition, a speaker often cannot be correctly identified. Still, the applications usually do have a main or prime user and adaptation of the application to the prime user is desirable to improve recognition accuracy. Hence, an adapted system should have some means of achieving only minor degradation for general speakers ("guest speakers") despite the adaptation to the prime user.
SUMMARY OF THE INVENTION
[0009] Embodiments of the present invention are directed to a method for speech
recognition which is implemented in the specific form of computer processes that function in a computer processor. That is, one or more computer processes: process a speech input to produce a sequence of representative speech vectors and perform multiple recognition passes to determine a recognition output corresponding to the speech input. At least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech. And at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech.
[0010] In further specific embodiments, the generic recognition pass and the adapted recognition pass may be performed in parallel. Or, the generic recognition pass and the adapted recognition pass may be performed in series. In the latter case, the adapted recognition pass may be performed before the generic recognition pass or vice versa. And the output of the first recognition pass may be a confusion network of recognition hypotheses, or a lattice or N-best list of recognition hypotheses that the following pass reevaluates. The generic recognition pass may use speaker independent recognition and the adapted recognition pass may use speaker dependent recognition. The final recognition result may be the result of the last recognition pass in the series; it may also be a system combination result incorporating results from intermediate passes.
[0011] In specific embodiments, the specific sub-class may be a specific prime user of the speech recognition method such that the adapted recognition pass uses speaker dependent recognition and the generic recognition pass uses speaker independent recognition. Or the specific sub-class may be a specific speaker group, a specific speaking style, or a specific speech channel. In some embodiments, the speech adapted arrangement may be based on adapted feature transforms so that the multiple recognition passes may use the same acoustic models.
[0012] Embodiments of the present invention also include a speech recognition engine implemented as a computer processes in a computer processor and using a method according to any of the arrangements described above. And embodiments also include a computer program product implemented in a computer readable storage medium for
speech recognition which includes program code for performing a method according to any of the arrangements described above.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 is a block diagram representing various hardware components of an embodiment of a speech recognition system according to the present invention.
[0014] Figure 2 shows functional blocks in a series multi-pass recognition arrangement according to one specific embodiment of the present invention.
DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS
[0015] Various embodiments of the present invention are directed to techniques for the combined application of adapted and un-adapted speech recognition arrangements to improve recognition performance for guest speakers using a speaker adapted recognition arrangement adapted for another main user. More broadly, this can be thought of as improving recognition performance for a broad general class of input speech in a recognizer that is pre-adapted to a specific sub-class of the general class. In specific embodiments, the specific sub-class may be a specific prime user of the speech recognition method such that the adapted recognition pass uses speaker dependent recognition and the generic recognition pass uses speaker independent recognition. Or the specific sub-class may be a specific speaker group, a specific speaking style, or a specific speech channel.
[0016] Figure 1 shows various hardware components of an embodiment of an ASR system according to the present invention. A computer system 10 includes a speech input microphone 11 which is connected through a suitable preamplifier 13 to an analog-to- digital (A/D) converter 15. A front-end speech pre-processor 17 typically performs a Fourier transform so as to extract spectral features to characterize the input speech as a sequence of representative multi-dimensional vectors. A speech recognition processor 12, e.g., an Intel Core 2 processor or the like, is programmed to run one or more specialized computer software processes to determine a recognition output corresponding to the speech input. To that end, processor memory 120, e.g., random access memory (RAM)
and/or read-only memory (ROM) stores the speech processing software routines, the speech recognition models and data for use by the speech recognition processor 12. The recognition output may be displayed, for example, as representative text on computer workstation display 14. Such a computer workstation would also typically include a keyboard 16 and a mouse 18 for user interaction with the system 10. Of course, many other typical arrangements are also familiar such as an ASR implemented for a mobile device such as a cell phone, ASR for the cabin of an automobile, client-server based ASR, etc.
[0017] According to embodiments of the present invention, a method for speech recognition is implemented in the specific form of computer processes that function in a computer processing environment such as described in the preceding paragraph.
Specifically, one or more computer processes operate in an ASR system to process a speech input to produce a sequence of representative speech vectors. Multiple recognition passes are performed on the vector sequence to determine a recognition output that corresponds to the speech input. Of these multiple recognition passes, at least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech. And at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech. Altogether, the multi-pass recognition arrangement provides a speech recognition system that is pre-adapted for improved recognition performance for a specific prime or main user, while having only minor degradation for general speakers ("guest speakers") despite the adaptation to the prime user.
[0018] Figure 2 shows an example of a specific series multi-pass recognition arrangement where a first recognition pass is initially performed on the speech input using of pre-adapted speaker dependent (SD) acoustic models to produce intermediate results of recognition hypotheses. For example, these recognition hypotheses may be in the specific form of a confusion network, a recognition lattice, or an N-best list. Then a second recognition pass is performed to rescore the intermediate recognition hypotheses using unadapted generic speaker independent (SI) acoustic models to produce the final
recognition result. A pseudo code representation of such an embodiment might be set forth as follows:
Process MultipassHybridAdaptedRecognition
convert speech input to sequence of representative speech
vectors,
first recognition pass with pre-adapted speaker dependent (SD) acoustic models to produce intermediate recognition results,
second recognition pass with un-adapted speaker dependent (SI) acoustic models to produce intermediate recognition results .
[0019] In such an arrangement, subsequent recognition passes can use the output of the previous pass for adaptation and/or constraining the search space. In some embodiments, the recognition passes may partially use a speaker-independent (SI) configuration and partially a speaker-dependent (SD) configuration. Or, some embodiments may select between SI and SD configurations based on model matching criteria.
[0020] The re-scoring pass can improve the final recognition result for many different reasons. It is possible to apply different, higher resolution acoustic models than in the first pass (e.g., a selected gender-dependent model, higher dimensional feature space, etc.). In some embodiments, the speech adapted SD arrangement may be based on adapted feature transforms so that the multiple recognition passes may use the same acoustic models. For example, batch-mode feature- and channel-normalization schemes may be used such as batch-mode Channel (or Cepstral) Mean/Variance Normalization (CMN/CVN), batch- mode spectral subtraction, and other mean and variance normalization schemes. Also, first and second pass model complementariness contributes to improving the final recognition result.
[0021] Of course, there are various different ways to construct an adapted arrangement. An SD recognition system can be initially constructed and adapted from the start to a specific speaker. Or an arrangement that starts with SI models can be adapted to become SD models based on either off-line or on-line adaptation over time. And either batch or incremental adaptation techniques may be applicable. Some experiments were performed where it was found that applying a pre-adapted SD acoustic model in the first pass and an
SI acoustic model configuration in the second pass yielded a favorable trade-off between adaptation gain for the prime speaker while preserving acceptable performance for guest speakers.
[0022] In other specific embodiments, the generic recognition pass and the adapted recognition pass may be performed in parallel rather than in series. With parallel decoding using an SI and an SD arrangement, the decoding searches can be mutually pruned or results can be combined by means of system combination such as ROVER (Recognizer Output Voting for Error Reduction), see e.g., J. G. Fiscus, A Post-Processing System To Yield Reduced Word Error Rates : Recognizer Output Voting Error Reduction (ROVER), In Proc. ASRU, pages 347-354, 1997; incorporated herein by reference, or CNC
(Confusion Network Combination), see. e.g., G. Evermann and P. Woodland, Posterior Probability Decoding, Confidence Estimation And System Combination, In Proc. NIST Speech Transcription Workshop, 2000; incorporated herein by reference.
[0023] Embodiments of the present invention also include a speech recognition engine implemented as a computer processes in a computer processor and using a method according to any of the arrangements described above. And embodiments also include a computer program product implemented in a computer readable storage medium for speech recognition which includes program code for performing a method according to any of the arrangements described above.
[0024] Embodiments of the invention may be implemented using any conventional computer programming language. For example, preferred embodiments may be implemented in a procedural programming language (e.g., "C") or an object oriented programming language (e.g. , "C++", Python). Alternative embodiments of the invention may be implemented as pre-programmed hardware elements, other related components, or as a combination of hardware and software components.
[0025] Embodiments can be implemented as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable medium (e.g., a diskette,
CD-ROM, ROM, or fixed disk) or transmittable to a computer system, via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein with respect to the system. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web). Of course, some embodiments of the invention may be implemented as a combination of both software (e.g. , a computer program product) and hardware. Still other embodiments of the invention are implemented as entirely hardware, or entirely software (e.g., a computer program product).
[0026] Although various exemplary embodiments of the invention have been disclosed, it should be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the true scope of the invention.
Claims
1. A method for speech recognition implemented as a plurality of computer processes functioning in a computer processor, the method comprising:
in one or more computer processes:
processing a speech input to produce a sequence of representative speech
vectors; and
performing multiple recognition passes to determine a recognition output corresponding to the speech input;
wherein at least one generic recognition pass is based on a generic speech recognition arrangement using generic modeling of a broad general class of input speech, and
wherein at least one adapted recognition pass is based on a speech adapted arrangement using pre-adapted modeling of a specific sub-class of the general class of input speech.
2. A method according to claim 1, wherein the at least one generic recognition pass and the at least one adapted recognition pass are performed in parallel to determine a combined recognition result.
3. A method according to claim 2, where the combined recognition result is determined by selection of a recognition result of one of the passes.
4. A method according to claim 3, where decoding search processes in the parallel passes are mutually pruned.
5. A method according to claim 2, where the combined recognition result is determined by means of system combination.
6. A method according to claim 2, where the combined recognition result is further refined by following re-evaluation passes.
7. A method according to claim I, wherein the at least one generic recognition pass and the at least one adapted recognition pass are performed in series.
8. A method according to claim 7, wherein the at least one adapted recognition pass is performed before the at least one generic recognition pass.
9. A method according to claim 7, wherein the output of the first recognition pass is a confusion network of recognition hypotheses and the following pass re-evaluates this confusion network.
10. A method according to claim 7, wherein the output of the first recognition pass is a lattice or N-best list of recognition hypotheses and the following pass re-evaluates this lattice of N-best list.
11. A method according to claim 7, wherein the at least one generic recognition pass uses speaker independent recognition and the at least one adapted recognition pass uses speaker dependent recognition.
12. A method according to claim 1, wherein the specific sub-class is a specific prime user of the speech recognition method such that the at least one adapted recognition pass uses speaker dependent recognition and the at least one generic recognition pass uses speaker independent recognition.
13. A method according to claim 1, wherein the specific sub-class is a specific speaker group.
14. A method according to claim 1, wherein the specific sub-class is a specific speaking style.
15. A method according to claim 1, wherein the specific sub-class is a specific speech channel.
16. A method according to claim 1, wherein the speech adapted arrangement is based on adapted feature transforms and the multiple recognition passes use the same acoustic models.
17. A speech recognition engine implemented as a plurality of computer processes functioning in a computer processor and using a method according to any of claims 1-16.
18. A computer program product implemented in a computer readable storage medium for speech recognition, the product comprising program code for performing a method according to any of claims 1-16.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2009/067055 WO2011071484A1 (en) | 2009-12-08 | 2009-12-08 | Guest speaker robust adapted speech recognition |
US13/514,458 US9478216B2 (en) | 2009-12-08 | 2009-12-08 | Guest speaker robust adapted speech recognition |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2009/067055 WO2011071484A1 (en) | 2009-12-08 | 2009-12-08 | Guest speaker robust adapted speech recognition |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2011071484A1 true WO2011071484A1 (en) | 2011-06-16 |
Family
ID=42358148
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2009/067055 WO2011071484A1 (en) | 2009-12-08 | 2009-12-08 | Guest speaker robust adapted speech recognition |
Country Status (2)
Country | Link |
---|---|
US (1) | US9478216B2 (en) |
WO (1) | WO2011071484A1 (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2877992A1 (en) * | 2012-07-24 | 2015-06-03 | Nuance Communications, Inc. | Feature normalization inputs to front end processing for automatic speech recognition |
US8645138B1 (en) * | 2012-12-20 | 2014-02-04 | Google Inc. | Two-pass decoding for speech recognition of search and action requests |
US9269355B1 (en) * | 2013-03-14 | 2016-02-23 | Amazon Technologies, Inc. | Load balancing for automatic speech recognition |
EP3485491A1 (en) * | 2016-08-26 | 2019-05-22 | Mitsubishi Electric Corporation | Factory automation system and programmable logic controller |
CN113223516B (en) * | 2021-04-12 | 2022-11-29 | 北京百度网讯科技有限公司 | Speech recognition method and device |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999018566A2 (en) * | 1997-10-07 | 1999-04-15 | Koninklijke Philips Electronics N.V. | A method and device for activating a voice-controlled function in a multi-station network through using both speaker-dependent and speaker-independent speech recognition |
US20020143540A1 (en) * | 2001-03-28 | 2002-10-03 | Narendranath Malayath | Voice recognition system using implicit speaker adaptation |
EP1394770A1 (en) * | 2001-06-08 | 2004-03-03 | Sony Corporation | Voice recognition apparatus and voice recognition method |
EP1548705A1 (en) * | 2003-12-23 | 2005-06-29 | AT&T Corp. | System and method for latency reduction for automatic speech recognition using partial multi-pass results |
Family Cites Families (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5864810A (en) | 1995-01-20 | 1999-01-26 | Sri International | Method and apparatus for speech recognition adapted to an individual speaker |
US5960397A (en) * | 1997-05-27 | 1999-09-28 | At&T Corp | System and method of recognizing an acoustic environment to adapt a set of based recognition models to the current acoustic environment for subsequent speech recognition |
US6249761B1 (en) * | 1997-09-30 | 2001-06-19 | At&T Corp. | Assigning and processing states and arcs of a speech recognition model in parallel processors |
US6073096A (en) | 1998-02-04 | 2000-06-06 | International Business Machines Corporation | Speaker adaptation system and method based on class-specific pre-clustering training speakers |
AU2901299A (en) * | 1998-03-09 | 1999-09-27 | Lernout & Hauspie Speech Products N.V. | Apparatus and method for simultaneous multimode dictation |
US6442519B1 (en) | 1999-11-10 | 2002-08-27 | International Business Machines Corp. | Speaker model adaptation via network of similar users |
DE10047723A1 (en) | 2000-09-27 | 2002-04-11 | Philips Corp Intellectual Pty | Method for determining an individual space for displaying a plurality of training speakers |
US7502737B2 (en) * | 2002-06-24 | 2009-03-10 | Intel Corporation | Multi-pass recognition of spoken dialogue |
US20090030552A1 (en) * | 2002-12-17 | 2009-01-29 | Japan Science And Technology Agency | Robotics visual and auditory system |
US20040186819A1 (en) * | 2003-03-18 | 2004-09-23 | Aurilab, Llc | Telephone directory information retrieval system and method |
US7895039B2 (en) | 2005-02-04 | 2011-02-22 | Vocollect, Inc. | Methods and systems for optimizing model adaptation for a speech recognition system |
US7881928B2 (en) * | 2006-09-01 | 2011-02-01 | International Business Machines Corporation | Enhanced linguistic transformation |
US8090570B2 (en) * | 2006-10-26 | 2012-01-03 | Mobile Technologies, Llc | Simultaneous translation of open domain lectures and speeches |
JP4410265B2 (en) * | 2007-02-19 | 2010-02-03 | 株式会社東芝 | Speech recognition apparatus and method |
US20090030691A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using an unstructured language model associated with an application of a mobile communication facility |
US20090030685A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using speech recognition results based on an unstructured language model with a navigation system |
US20090030697A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using contextual information for delivering results generated from a speech recognition facility using an unstructured language model |
US20090030687A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Adapting an unstructured language model speech recognition system based on usage |
US20090030688A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Tagging speech recognition results based on an unstructured language model for use in a mobile communication facility application |
US8843370B2 (en) * | 2007-11-26 | 2014-09-23 | Nuance Communications, Inc. | Joint discriminative training of multiple speech recognizers |
US8386251B2 (en) * | 2009-06-08 | 2013-02-26 | Microsoft Corporation | Progressive application of knowledge sources in multistage speech recognition |
US9892730B2 (en) * | 2009-07-01 | 2018-02-13 | Comcast Interactive Media, Llc | Generating topic-specific language models |
US8468012B2 (en) | 2010-05-26 | 2013-06-18 | Google Inc. | Acoustic model adaptation using geographic information |
-
2009
- 2009-12-08 WO PCT/US2009/067055 patent/WO2011071484A1/en active Application Filing
- 2009-12-08 US US13/514,458 patent/US9478216B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO1999018566A2 (en) * | 1997-10-07 | 1999-04-15 | Koninklijke Philips Electronics N.V. | A method and device for activating a voice-controlled function in a multi-station network through using both speaker-dependent and speaker-independent speech recognition |
US20020143540A1 (en) * | 2001-03-28 | 2002-10-03 | Narendranath Malayath | Voice recognition system using implicit speaker adaptation |
EP1394770A1 (en) * | 2001-06-08 | 2004-03-03 | Sony Corporation | Voice recognition apparatus and voice recognition method |
EP1548705A1 (en) * | 2003-12-23 | 2005-06-29 | AT&T Corp. | System and method for latency reduction for automatic speech recognition using partial multi-pass results |
Also Published As
Publication number | Publication date |
---|---|
US20120245940A1 (en) | 2012-09-27 |
US9478216B2 (en) | 2016-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9406299B2 (en) | Differential acoustic model representation and linear transform-based adaptation for efficient user profile update techniques in automatic speech recognition | |
US10847137B1 (en) | Trigger word detection using neural network waveform processing | |
Li et al. | An overview of noise-robust automatic speech recognition | |
Hain et al. | New features in the CU-HTK system for transcription of conversational telephone speech | |
US8386254B2 (en) | Multi-class constrained maximum likelihood linear regression | |
US9280979B2 (en) | Online maximum-likelihood mean and variance normalization for speech recognition | |
US9037463B2 (en) | Efficient exploitation of model complementariness by low confidence re-scoring in automatic speech recognition | |
US20080004876A1 (en) | Non-enrolled continuous dictation | |
US20150206527A1 (en) | Feature normalization inputs to front end processing for automatic speech recognition | |
US6148284A (en) | Method and apparatus for automatic speech recognition using Markov processes on curves | |
US9478216B2 (en) | Guest speaker robust adapted speech recognition | |
Nakagawa | A survey on automatic speech recognition | |
EP2867890B1 (en) | Meta-data inputs to front end processing for automatic speech recognition | |
Shinozaki et al. | Unsupervised acoustic model adaptation based on ensemble methods | |
US8768695B2 (en) | Channel normalization using recognition feedback | |
KR20050101695A (en) | A system for statistical speech recognition using recognition results, and method thereof | |
Gunasekara et al. | Real-time translation of discrete sinhala speech to unicode text | |
Sankar et al. | Noise-resistant feature extraction and model training for robust speech recognition | |
JPH0981177A (en) | Voice recognition device, dictionary for work constitution elements and method for learning imbedded markov model | |
Cloarec et al. | Modeling inter-speaker variability in speech recognition | |
Stouten et al. | Evaluation of model-based feature enhancement on the AURORA-4 task | |
Müller et al. | Robust continuous speech recognition through combination of invariant-feature based systems | |
Ivanecký et al. | French–German bilingual acoustic modeling for embedded voice driven applications | |
Jayasena et al. | Kathana-fluent speech recognition system based on hidden markov model for Sinhala language | |
Gao et al. | 15.5 Large Vocabulary Continuous Speech Recognition |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09764431 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 13514458 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 09764431 Country of ref document: EP Kind code of ref document: A1 |