EP0621531B1 - Interaktives Rechnersystem, das gesprochene Befehle erkennt - Google Patents

Interaktives Rechnersystem, das gesprochene Befehle erkennt Download PDF

Info

Publication number
EP0621531B1
EP0621531B1 EP94105293A EP94105293A EP0621531B1 EP 0621531 B1 EP0621531 B1 EP 0621531B1 EP 94105293 A EP94105293 A EP 94105293A EP 94105293 A EP94105293 A EP 94105293A EP 0621531 B1 EP0621531 B1 EP 0621531B1
Authority
EP
European Patent Office
Prior art keywords
active
state
vocabulary
computer program
time period
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP94105293A
Other languages
English (en)
French (fr)
Other versions
EP0621531A1 (de
Inventor
Joseph Charles Andreshak
Gregg H. Daggett
John Karat
John Lucassen
Stephen Eric Levy
Robert Lawrence Mack
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Publication of EP0621531A1 publication Critical patent/EP0621531A1/de
Application granted granted Critical
Publication of EP0621531B1 publication Critical patent/EP0621531B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
    • G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context

Definitions

  • the invention relates to interactive computer systems in which a user provides commands to a target computer program executing on the computer system by way of an input device.
  • the input device may be, for example, a keyboard, a mouse device, or a speech recognizer.
  • an input signal generated by the input device is translated into a form usable by the target computer program.
  • An interactive computer system in which the user can provide commands by speaking the commands may consist of a processor executing a target computer program having commands identifying functions which can be performed by the target computer program.
  • the computer system further includes a speech recognizer for recognizing the spoken commands and for outputting command signals corresponding to the recognized commands.
  • the speech recognizer recognizes a spoken command by measuring the value of at least one feature of an utterance during each of a series of successive time intervals to produce a series of feature signals, comparing the measured featured signals to each of a plurality of acoustic command models to generate a match score for the utterance and each acoustic command model, and outputting a command signal corresponding to the command model having the best match score.
  • the set of utterance models and words represented by the utterance models which the speech recognizer can recognize is referred to as the system vocabulary.
  • the system vocabulary is finite and may, for example, range from one utterance model to thousands of utterance models.
  • Each utterance model may represent one word, or may represent a combination of two or more words spoken continuously (without a pause between the words).
  • the system vocabulary may contain, for example, utterance models of all of the commands to which the target computer program is capable of responding. However, as the number of utterance models increases, the time required to perform utterance recognition using the entire system vocabulary increases, and the recognition accuracy decreases.
  • a target computer program has a series of active states occurring over a series of time periods. For each active state, there may be a list of active state commands identifying functions which can be performed in the active state.
  • the active state commands may be a small subset of the system vocabulary.
  • the translation of an uttered command to a form usable by the target computer program in one state of the target computer program may be different from the translation of the same command in another state of the target computer program.
  • the speech recognizer may be provided with a finite state machine which duplicates the active states and transitions between active states of the target computer program.
  • a command to load a file will cause a computer program to make a transition to one state if the file exists, or to a different state if the file does not exist.
  • the speech recognizer finite state machine must be built with some assumption that the file exists or does not exist. If a command to load a file is spoken to the com- puter program using the speech recognizer, then the speech recognizer finite state machine may or may not track the computer program state correctly, depending on whether that file exists or does not exist. If the speech recognizer finite state machine assumes that the file exists, but in fact the file does not exist, then the speech recognizer state machine will enter a state different from the state of the target computer program. As a result, the target computer program can no longer receive valid input from the speech recognizer.
  • EP-A-414238 discloses a voice-controlled filing system using the prior knowledge of the functional cycle of the filing and retrieval program to limit the speech recognition to the comparison of a spoken command to a reference model corresponding to what the system can expect as a command, based on the previous command entered by the user.
  • Another example is the article entitled «Integrated Audio-Graphics User Interface» published in the IBM Technical Disclosure Bulletin, Vol. 33, No. 11, April 1991, on pages 368-371, which suggests to reduce the complexity of the speech recognition by limiting the options opened to the user of a program by devising special, carefully chosen, menus.
  • Both systems rely on special designs for the computer systems and/or programs specifically intended for making it possible to predict in advance the evolution of the program states and transitions between states.
  • an interactive computer system is as set out in claim 1.
  • the display means displays at least a second active-state image different from the first active-state image for a second active state occurring during a second time period different from the first time period.
  • the object identifying means identifies at least one object displayed in the second active-state image, and generates a list of one or more second active-state commands identifying functions which can be performed in the second active state of the target computer program.
  • the active-state vocabulary identifying means identifies a second active-state vocabulary of acoustic command models for the second active state.
  • the second active-state vocabulary comprises the acoustic command models from the system vocabulary representing the second active-state commands.
  • the second active-state vocabulary is at least partly different from the first active-state vocabulary.
  • the speech recognizer measures the value of at least one feature of an utterance during each of a series of successive time intervals within the second time period to produce a series of feature signals.
  • the speech recognizer compares the measured feature signals for the second time period to each of the acoustic command models in the second active-state vocabulary to generate a match score for the utterance and each acoustic command model.
  • the speech recognizer then outputs a command signal corresponding to the command model from the second active-state vocabulary having the best match score.
  • the target computer program may, for example, have only one active state occurring during each time period.
  • the target computer program may comprise an operating system program alone, an application program and an operating system program combined, or two or more application programs and an operating system program.
  • At least some of the commands for an active-state identify functions which can be performed on the identified objects in the active-state image for the state.
  • the identified object in an active-state image may, for example, comprise one or more of a character, a word, an icon, a button, a scroll bar, a slider, a list box, a menu, a check box, a container, or a notebook.
  • the speech recognizer may output two or more command signals corresponding to the command models from the active-state vocabulary having the best match scores for a given time period.
  • the vocabulary of acoustic command models for each active state may further comprise a set of global acoustic command models representing global commands identifying functions which can be performed in each active state of the target computer program.
  • the display means may comprise, for example, a cathode ray tube display, a liquid crystal display, or a printer.
  • the display means may display both an active-state image for an active state occurring during a time period, and at least a portion of one or more images for program states not occurring during the time period.
  • a method of computer interaction according to the invention is as set out in claim 14.
  • the active-state vocabulary of the speech recognizer can be limited to a small subset of the system vocabulary representing active-state commands, without having to predict in advance the states and transitions between states of the target computer program which will occur under all possible circumstances.
  • Figure 1 is a block diagram of an example of an interactive computer system according to the invention.
  • Figure 2 shows an example of a first active-state image for a first active state of a target computer program.
  • Figure 3 is a block diagram of an example of a speech recognizer for an interactive computer system according to the invention.
  • Figure 4 shows an example of a second active-state image for a second active state of a target computer program.
  • Figure 5 is a block diagram of an example of an acoustic command model store for the system vocabulary of an interactive computer system according to the invention.
  • Figure 6 is a block diagram of an acoustic processor for the speech recognizer of Figure 3.
  • Figure 7 schematically shows an example of an acoustic command model.
  • Figure 8 schematically shows an example of an acoustic model of a phoneme for constructing an acoustic command model.
  • Figure 9 schematically shows an example of paths through the acoustic model of Figure 7.
  • FIG. 1 is a block diagram of an example of an interactive computer system according to the invention.
  • the interactive computer system comprises a processor 10 executing a target computer program having a series of active program states occurring over a series of time periods.
  • the target computer program generates active state image data signals representing an active state image for the active state of the target computer program occurring during each time period.
  • Each active state image contains one or more objects.
  • the processor may be, for example, a personal computer, a computer work station, or any other microcomputer, minicomputer, or main frame computer.
  • the target computer program may be an operating system program such as DOS, Microsoft Windows (trademark), OS/2 (trademark), AIX (trademark), UNIX (trademark), X-Windows, or any other operating system.
  • the target computer program may comprise one or more application programs executing with an operating system program.
  • Application programs include spreadsheet programs, word processing programs, database programs, educational programs, recreational programs, communication programs, and many more.
  • Objects in an active-state image may comprise one or more of a character, a word, an icon, a button, a scroll bar, a slider, a list box, a menu, a check box, a container, a notebook, or some other items.
  • the interactive computer system further comprises display means 12 for displaying at least a first active-state image for a first active state occurring during a first time period.
  • the display means may be, for example, a cathode ray tube display, a liquid crystal display, or a printer.
  • Figure 2 shows an example of a hypothetical first active-state image for a first active state occurring during a first time period.
  • the active-state image includes a frame object 14 containing a title bar object 16, a menu bar object 18, a list box object 20, and a push button object 22.
  • the menu bar object 18 includes an "items” object, an "options” object, and an "exit” object.
  • the list box object 20 includes a vertical scroll bar object 24, and "blue”, “green”, “red”, “orange”, “black”, “white”, and “purple” objects. In the list box 20, only the “blue”, “green”, “red”, “orange”, and “black” objects are shown in Figure 2.
  • the "white” and “purple” objects are contained in the list box and could be made visible by scrolling with the vertical scroll bar 24.
  • the active state image data signals may be generated by the target computer program, for example, by using operating system interrupts, function calls, or application program interface calls.
  • Example I illustrates C programming language source code for creating active state image data signals.
  • the interactive computer system further comprises an image object identifier 26 for identifying at least one object displayed in the first active-state image, and for generating from the identified object a list of one or more first active-state commands identifying functions which can be performed in the first active-state of the target computer program.
  • the image object identifier 26 may comprise computer program subroutines designed to intercept (hook) operating system function calls, and application program interface calls provided by one or more target computer programs, and/or may comprise computer program subroutines for using operating system interrupts, function calls, or application program interface calls for identifying objects displayed in the first active-state image of the target computer program.
  • Example II illustrates C programming language source code for identifying at least one object displayed in an active state image.
  • Table 1 shows a hypothetical example of a list of first active-state commands identifying functions which can be performed in the first active-state of the target computer program for the objects displayed in the first active-state image of Figure 2.
  • each object may have zero or more commands identifying functions which can be performed in the first active state of the target computer program. At least some commands identify functions which can be performed on the identified object in the active-state image for the state.
  • the command "FRAME” changes the focus to the entire frame object 14 of Figure 2. With the focus on the entire frame object 14, the spoken command “LEFT” operates on the frame object by moving it to the left on the display screen.
  • the interactive computer system comprises a system acoustic command model vocabulary store 28 for storing a system vocabulary of acoustic command models.
  • Each acoustic command model represents one or more series of acoustic feature values representing an utterance of one or more words associated with the acoustic command model.
  • the stored acoustic command models may be, for example, Markov models or other dynamic programming models.
  • the parameters of the acoustic command models may be estimated from a known uttered training text (for example, 257 sentences) by, for example, smoothing parameters obtained by the forward-backward algorithm. (See, for example, Jelinek. "Continuous Speech Recognition By Statistical Methods", Proceedings of the IEEE, Volume 64, No. 4, April 1976 pages 532-536.)
  • each acoustic command model represents a command spoken in isolation (that is, independent of the context of prior and subsequent utterances).
  • Context-independent acoustic command models can be produced, for example, either manually from models of phonemes or automatically, for example, by the method described by Lalit R. Bahl et al in U.S. Patent 4,759,068 entitled “Constructing Markov Models of Words from Multiple Utterances", or by any other known method of generating context-independent models.
  • context-dependent models may be produced from context-independent models by grouping utterances of a command into context-dependent categories.
  • a context can be, for example, manually selected, or automatically selected by tagging each feature signal corresponding to a command with its context, and by grouping the feature signals according to their context to optimize a selected evaluation function.
  • the interactive computer system comprises an active-state command model vocabulary identifier 30 for identifying a first active-state vocabulary of acoustic command models for the first active state.
  • the first active-state vocabulary comprises the acoustic command models from the system vocabulary 28 representing the first active-state commands from the image object identifier 26.
  • Example III illustrates C programming language source code for identifying an active-state vocabulary.
  • Example IV illustrates C programming language source code for defining the active-state vocabulary to the speech recognizer.
  • the active-state vocabulary comprises substantially less than all of the acoustic command models in the system vocabulary.
  • each active-state vocabulary may comprise 50 to 200 commands.
  • the entire system command vocabulary may comprise 500 to 700 or more commands.
  • the speech recognizer does not compare the measured feature signals for a time period to any acoustic command model which is not in the active-state vocabulary for that time period.
  • a speech recognizer 32 measures the value of at least one feature of an utterance during each of a series of successive time intervals within the first time period to produce a series of feature signals.
  • the speech recognizer 32 compares the measured feature signal to each of the acoustic command models in the first active-state vocabulary to generate a match score for the utterance and each acoustic command model.
  • the speech recognizer 32 outputs a command signal corresponding to the command model from the first active-state vocabulary having the best match score.
  • Example V illustrates C programming language source code for outputting a command signal corresponding to the command model from an active-state vocabulary having the best match score.
  • FIG. 3 is a block diagram of an example of a speech recognizer for an interactive computer system according to the invention.
  • the speech recognizer 32 comprises an active-state acoustic command model store 34 for storing the active-state vocabulary comprising the acoustic command models from the system vocabulary store 28 representing the active-state commands identified in active state command model vocabulary identifier 30.
  • the speech recognizer 32 further comprises an acoustic processor 36 for measuring the value of at least one feature of an utterance during each of a series of successive time intervals within each active-state time period to produce a series of feature signals.
  • An acoustic match score processor 38 compares the measured feature signals from acoustic processor 36 to each of the acoustic command models in the active-state acoustic command models store 34 to generate a match score for the utterance and each acoustic command model.
  • An output 40 outputs one or more command signals corresponding to the command models from the active state vocabulary having the best match scores for a given time period.
  • only one command signal corresponding to the command model from the first active-state vocabulary having the best match score is output.
  • the one output command may be immediately executed. If two or more command signals corresponding to the command models from the active-state vocabulary having the best match scores for a given time period are output, then the recognized commands may be displayed for the user to select one for execution.
  • the speech recognizer may be a publicly available product such as the IBM Voice Type II (trademark) or the IBM Speech Server Series (trademark). In products containing a fast acoustic match and a detailed acoustic match, both acoustic matches may be used in the invention. Alternatively, since the image object identifier 26 and the active state command model vocabulary identifier 30 select only a small subset of the system vocabulary in store 28 for the acoustic match, the fast acoustic match can be omitted.
  • the language model can be omitted.
  • all of the words in the active-state vocabulary can be assigned equal language model probabilities.
  • each command is independent of successive commands.
  • both the target computer program and the speech recognizer are executed on the same central processing unit in a time sharing manner.
  • the target computer program and the speech recognizer can be executed on different central processing units, for example using a client-server architecture.
  • the display means may further display at least a second active-state image different from the first active-state image for a second active state occurring during a second time period different from the first time period.
  • Figure 4 shows an example of a second active-state image for a second active state of the target computer program.
  • the second active-state image shown in Figure 4 contains a frame object 42, a title bar object 44, a system menu object 46, a vertical scroll bar object 48, a horizontal scroll bar object 50, and a container object 52.
  • the container object 52 contains an "editor” object, a "phone book” object, a "spreadsheet” object, a "mail” object, and a "solitaire” object.
  • the object identifying means identifies at least one object displayed in the second active-state image, and generates from the identified object a list of one or more second active-state commands identifying functions which can be performed in the second active-state of the target computer program.
  • Table 2 is an example of a hypothetical list of commands for each object shown in Figure 4 identifying functions which can be performed in the second active-state of the target computer program.
  • the first active-state image differs from the second active-state image by providing menu bar object 18, list box object 20, and push button object 22 in the first active-state image but not in the second active-state image.
  • the horizontal scroll bar 50, and the editor, phone book, mail, spreadsheet, and solitaire objects are provided in the second active-state image, but not in the first active-state image.
  • the active-state vocabulary identifying means further identifies a second active-state vocabulary of acoustic command models for the second active state.
  • the second active state vocabulary comprises the acoustic command models from the system vocabulary representing the second active-state commands.
  • the second active-state vocabulary is at least partly different from the first active-state vocabulary.
  • the first active-state vocabulary comprises the spoken commands listed in Table 1.
  • the second active-state vocabulary comprises the spoken commands listed in Table 2.
  • the first active-state vocabulary is at least partly different from the second active-state vocabulary as shown therein.
  • the speech recognizer measures the value of at least one feature of an utterance during each of a series of successive time intervals within the second time period to produce a series of feature signals.
  • the speech recognizer compares the measured feature signals for the second time period to each of the acoustic command models in the second active-state vocabulary to generate a match score for the utterance and each acoustic command model.
  • the speech recognizer outputs a command signal corresponding to the command model from the second active-state vocabulary having the best match score.
  • the target computer program has only one active state occurring during each time period.
  • Figure 5 is a block diagram of an example of the acoustic command model vocabulary store 28 of Figure 1.
  • the system vocabulary may comprise, for example, a set of global acoustic command models representing global commands identifying functions which can be performed in every active state of the target computer program.
  • Table 3 lists some examples of global commands represented by global acoustic command models.
  • Global Commands Spoken Command Function MICROPHONE OFF turns the microphone off ENTER sends "ENTER” keystroke to keyboard input buffer LEFT sends "LEFT ARROW” keystroke to keyboard input buffer RIGHT sends "RIGHT ARROW” keystroke to keyboard input buffer PASTE inserts contents of clipboard into application with the focus WINDOW LIST displays a list of running programs EDITOR executes the editor program DESK top makes the desk top window active
  • the system vocabulary may further comprise object type acoustic command models associated with different types of objects.
  • object type acoustic command models associated with different types of objects.
  • frame object type acoustic commands include “frame”, “top border”, “bottom border”, “left border”, “right border”, “left”, “right”, “up”, and “down”.
  • Vertical scroll bar object type acoustic commands include “scroll bar”, “up”, “down”, “top”, “bottom”, “page up”, “page down”.
  • Push button object type acoustic command models include “press” and "push button”.
  • system vocabulary includes application-specific acoustic command models representing application-specific objects.
  • application-specific objects include the words “items”, “colors”, “names”, “addresses”, “phone book”, “spreadsheet”, “mail” and "solitaire”.
  • the display means 12 of Figure 1 may display both an active-state image for an active state occurring during a time period, and at least a portion of one or more images for program states not occurring during the time period.
  • the acoustic processor 36 of Figure 3 comprises a microphone 54 for generating an analog electrical signal corresponding to the utterance.
  • the analog electrical signal from microphone 54 is converted to a digital electrical signal by analog to digital converter 56.
  • the analog signal may be sampled, for example, at a rate of twenty kilohertz by the analog to digital converter 56.
  • a window generator 58 obtains, for example, a twenty millisecond duration sample of the digital signal from analog to digital converter 56 every ten milliseconds (one centisecond). Each twenty millisecond sample of the digital signal is analyzed by spectrum analyzer 60 in order to obtain the amplitude of the digital signal sample in each of, for example, twenty frequency bands. Preferably, spectrum analyzer 60 also generates a twenty-first dimension signal representing the total amplitude or total power of the twenty millisecond digital signal sample.
  • the spectrum analyzer 60 may be, for example, a fast Fourier transform processor. Alternatively, it may be a bank of twenty band pass filters.
  • the twenty-one dimension vector signals produced by spectrum analyzer 60 may be adapted to remove background noise by an adaptive noise cancellation processor 62.
  • Noise cancellation processor 62 subtracts a noise vector N(t) from the feature vector F(t) input into the noise cancellation processor to produce an output feature vector F '( t ).
  • the noise cancellation processor 62 adapts to changing noise levels by periodically updating the noise vector N(t) whenever the prior feature vector F(t-1) is identified as noise or silence.
  • the prior feature vector F(t-1) is recognized as noise or silence if either (a) the total energy of the vector is below a threshold, or (b) the closest prototype vector in adaptation prototype vector store 66 to the feature vector is a prototype representing noise or silence.
  • the threshold may be, for example, the fifth percentile of all feature vectors (corresponding to both speech and silence) produced in the two seconds prior to the feature vector being evaluated.
  • the feature vector F' ( t ) is normalized to adjust for variations in the loudness of the input speech by short term mean normalization processor 68.
  • Normalization processor 68 normalizes the twenty-one dimension feature vector F' ( t ) to produce a twenty dimension normalized feature vector X(t).
  • the normalized twenty dimension feature vector X(t) may be further processed by an adaptive labeler 70 to adapt to variations in pronunciation of speech sounds.
  • An adapted twenty dimension feature vector X '( t ) is generated by subtracting a twenty dimension adaptation vector A(t) from the twenty dimension feature vector X(t) provided to the input of the adaptive labeler 70.
  • the twenty dimension adapted feature vector signal X' ( t ) from the adaptive labeler 70 is preferably provided to an auditory model 72.
  • Auditory model 72 may, for example, provide a model of how the human auditory system perceives sound signals.
  • An example of an auditory model is described in U.S. Patent 4,980,918 to Bahl et al entitled "Speech Recognition System with Efficient Storage and Rapid Assembly of Phonological Graphs".
  • the output of the auditory model 72 is a modified twenty dimension feature vector signal.
  • This feature vector is augmented by a twenty-first dimension having a value equal to the square root of the sum of the squares of the values of the other twenty dimensions.
  • a concatenator 74 For each centisecond time interval, a concatenator 74 preferably concatenates nine twenty-one dimension feature vectors representing the one current centisecond time interval, the four preceding centisecond time intervals, and the four following centisecond time intervals to form a single spliced vector of 189 dimensions.
  • Each 189 dimension spliced vector is preferably multiplied in a rotator 76 by a rotation matrix to rotate the spliced vector and to reduce the spliced vector to fifty dimensions.
  • the rotation matrix used in rotator 76 may be obtained, for example, by classifying into M classes a set of 189 dimension spliced vectors obtained during a training session.
  • the covariance matrix for all of the spliced vectors in the training set is multiplied by the inverse of the within-class covariance matrix for all of the spliced vectors in all M classes.
  • the first fifty eigenvectors of the resulting matrix form the rotation matrix.
  • Window generator 58 may be suitably programmed special purpose or general purpose digital signal processors.
  • Prototype stores 64 and 66 may be electronic computer memory of the types discussed above.
  • the prototype vectors in prototype store 54 may be obtained, for example, by clustering feature vector signals from a training set into a plurality of clusters, and then calculating the mean and standard deviation for each cluster to form the parameter values of the prototype vector.
  • the training script comprises a series of word-segment models (forming a model of a series of words)
  • each word-segment model comprises a series of elementary models having specified locations in the word-segment models
  • the feature vector signals may be clustered by specifying that each cluster corresponds to a single elementary model in a single location in a single word-segment model.
  • all acoustic feature vectors generated by the utterance of a training text and which correspond to a given elementary model may be clustered by K-means Euclidean clustering or K-means Gaussian clustering, or both.
  • K-means Euclidean clustering or K-means Gaussian clustering, or both.
  • Figure 7 schematically shows a hypothetical example of an acoustic command model.
  • the hypothetical model shown in Figure 7 has a starting state S1, an ending state S4, and a plurality of paths from the starting state S1 to the ending state S4.
  • Figure 8 schematically shows a hypothetical example of an acoustic Markov model of a phoneme.
  • the acoustic phoneme model comprises three occurrences of transition T1, four occurrences of transition T2, and three occurrences of transition T3.
  • the transitions shown in dotted lines are null transitions.
  • Each solid-line transition in the acoustic models of Figures 7 and 8 has at least one model output comprising an acoustic feature value.
  • Each model output has an output probability.
  • Each null transition has no output.
  • Each solid line transition and each dotted line transition from a state has a probability of occurrence when the model is in that state.
  • Figure 9 shows a hypothetical example of paths through the acoustic model of Figure 7.
  • the match score for an utterance and an acoustic command model is the sum of the probabilities of the measured features of the utterance for all paths through the acoustic command model.
  • the probability of the measured features of the utterance is equal to the product of the probabilities of the transitions along the path times the probabilities of the measured features at each transition along the path.
  • the interactive computer system may be made by suitably programming a general purpose digital computer system.
  • the processor 10, the image object identifier 26, and the active-state command model vocabulary identifier 30 may be made by suitably programming a general purpose digital processor.
  • the system acoustic command model vocabulary store 28 and the active-state acoustic command models store 34 may be electronic computer memory.
  • the display means 12 may comprise a video display such as a cathode ray tube, a liquid crystal display, or a printer.
  • the target computer program may be one or more application programs and an operating system program.
  • the target computer program may be IBM OS/2 (trademark) version 2.0, and Presentation Manager (trademark).
  • IBM's OS/2 version 2.0 operating system and Presentation Manager have application program interface calls in various languages, including the C programming language, the assembly programming language, and the REXX programming language.
  • the complete collection of application program interface calls is part of the OS/2 2.0 Technical Library.
  • the syntax for the application program interface calls in a language is compatible with how standard calls operate in the language.
  • the name of a particular application program interface call may be different for different languages. Also, some aspects of the application program interface in one language may not be supported from another language.
  • the application program interface consists of many library calls.
  • C programming language source code be compiled with the IBM C Set/2 compiler.
  • Examples I to V illustrate C programming language source code for OS/2 and Presentation Manager for (a) creating and displaying an image, (b) reading the active state image to identify at least one object displayed in the active state image, (c) creating the vocabulary from the active state image, (d) defining the vocabulary to the speech recognizer, and (e) outputting a command signal corresponding to the command model from an active-state vocabulary having the best match score.
  • Example I illustrates C programming language source code for creating the hypothetical first active-state image shown in Figure 2.
  • a standard window is a combination of several commonly-used windows.
  • the frame window, title bar, system menu and menu bar can be considered to be part of a standard window.
  • the standard window is created with the following C programming language source code using the OS/2 application program interface call WinCreateStdWindow(). The comments following the double slashes (//) describe the operation of the source code.
  • Example II illustrates C programming language source code for reading an active-state image.
  • Presentation Manager provides an application program interface call for any application to put a "hook" into the queues of messages which are passed back and forth between windows.
  • a hook is installed with a call back function which gets called with every message which is sent.
  • Call back functions for hooks must reside in a presentation manager dynamic link library. The required procedure is to load the dynamic link library which contains the call back function and then load the hook.
  • Example III illustrates C programming language source code for identifying the list of active-state commands from the active-state image.
  • the procedure for creating the list of active-state commands from the image is as follows. (1) Create a list of all the windows which are children (direct or indirect) of the active frame found above. (2) Identify all windows in the list by their window class. (3) For windows from window classes which display text to the user, query all the window text (hidden and visible). (4) Combine a global list of words with a standard list of words for each window type and with the words which were queried from the application in step (3).
  • Step (4) merely involves combining multiple arrays of words into one array of words. Therefore, source code for Step (4) is not illustrated.
  • Example IV illustrates C programming language source code for defining the active-state vocabulary to the speech recognizer.
  • An application program interface for the speech recognizer is used to set it up for recognition.
  • a possible application program interface which can be used is the Speech Manager (trademark) application program interface that comes with the IBM Speech Server Series (trademark) product. Source code for a similar application program interface will be shown below.
  • Example V illustrates C programming language source code for outputting a command signal corresponding to the command model from an active-state vocabulary having the best match score.
  • the action defined by "RIGHT” for this target is to move the target to the right by a previously-defined increment, for example 10 picture elements (pels). #define INCREMENT_RIGHT 10
  • the command is performed on the target using the OS/2 Presentation Manager application program interface call named WinSetWindowPos().
  • the current window position must be queried first so that the new position can be determined. Instead, assume the word, "ORANGE," is recognized. From the list of command-object associations, the target for the command is known. This is hwndTarget in the example. HWND hwndTarget;
  • the action defined by "ORANGE" for this target is to select the entry in the listbox.
  • the command is performed on the target by sending a message, LM_SELECTITEM, to the listbox using the OS/2 Presentation Manager application interface call named WinSendMsg(). First the index of the item has to be found.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Acoustics & Sound (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Digital Computer Display Output (AREA)
  • Selective Calling Equipment (AREA)
  • Debugging And Monitoring (AREA)

Claims (25)

  1. Interaktives Rechnersystem, das folgendes umfasst:
    einen Prozessor (10), der ein Zielrechnerprogramm ausführt, das eine Reihe von aktiven Programmzuständen hat, die über eine Folge von Zeiträumen auftreten, wobei das Zielrechnerprogramm Datensignale eines Bildes eines aktiven Zustands erzeugt, die ein Bild des aktiven Zustands für einen aktiven Zustand des Zielrechnerprogramms darstellen, der während eines jeden Zeitraums auftritt, wobei jedes Bild des aktiven Zustands ein oder mehrere Objekte enthält;
    ein Mittel (12), um mindestens ein Bild des ersten aktiven Zustands für einen ersten aktiven Zustand anzuzeigen, der während eines ersten Zeitraums auftritt;
    ein Mittel (26), um ein Objekt zu kennzeichnen, das in dem Bild des ersten aktiven Zustands angezeigt wird, und um von einem gekennzeichneten Objekt, das in dem Bild des ersten aktiven Zustands angezeigt wird, eine Liste von einem oder mehreren Befehlen des ersten aktiven Zustands zu erzeugen, die eine Funktion des ersten aktiven Zustands kennzeichnen, welche in dem ersten aktiven Zustand des Zielrechnerprogramms ausgeführt werden kann;
    ein Mittel (28), um ein Systemvokabular von akustischen Befehlsmodellen zu speichern, wobei jedes akustische Befehlsmodell eine oder mehrere Reihen von akustischen Merkmalswerten darstellt, die eine Äußerung von einem oder mehreren Wörtern darstellen, welche zu dem akustischen Befehlsmodell gehören;
    ein Mittel (30), um ein Vokabular von akustischen Befehlsmodellen eines ersten aktiven Zustands für den ersten aktiven Zustand zu kennzeichnen, wobei das Vokabular des ersten aktiven Zustands die akustischen Befehlsmodelle von dem Systemvokabular umfasst, das die Befehle des ersten aktiven Zustands darstellt, wobei sich das Vokabular des ersten aktiven Zustands in Abhängigkeit von sowohl der Kennung des Zielrechnerprogramms als auch der Datensignale des Bildes des aktiven Zustands dynamisch ändert, die einen aktiven Zustand des Zielrechnerprogramms kennzeichnen; und
    eine Spracherkennungseinheit (32), um einen Wert von mindestens einem Merkmal einer Äußerung während eines jeden einer ersten Folge von aufeinanderfolgenden Zeitintervallen innerhalb des ersten Zeitraums zu messen, um eine erste Reihe von Merkmalssignalen zu erzeugen, wobei die Spracherkennungseinheit die erste Reihe der Merkmalssignale mit jedem der akustischen Befehlsmodelle in dem Vokabular des ersten aktiven Zustands vergleicht, um ein Übereinstimmungsergebnis für die Äußerung und jedes akustische Befehlsmodell zu erzeugen, und wobei die Spracherkennungseinheit ein Befehlssignal ausgibt, das dem Befehlsmodell von dem Vokabular des ersten aktiven Zustands entspricht, welches das beste Übereinstimmungsergebnis aufweist.
  2. Interaktives Rechnersystem nach Anspruch 1, dadurch gekennzeichnet, dass:
    das Vokabular des ersten aktiven Zustands wesentlich weniger als alle der akustischen Befehlsmodelle von dem Systemvokabular umfasst; und
    die Spracherkennungseinheit die gemessenen Merkmalssignale für den ersten Zeitraum nicht mit einem akustischen Befehlsmodell vergleicht, das sich nicht im Vokabular des ersten aktiven Zustands befindet.
  3. Interaktives Rechnersystem nach Anspruch 2, dadurch gekennzeichnet, dass:
    das Anzeigemittel mindestens ein Bild eines zweiten aktiven Zustands, das sich von dem Bild des ersten aktiven Zustands unterscheidet, für einen zweiten aktiven Zustand anzeigt, der während eines zweiten Zeitraums auftritt, welcher sich von dem ersten Zeitraum unterscheidet;
    das Mittel zur Kennzeichnung eines Objekts ein Objekt kennzeichnet, das in dem Bild des zweiten aktiven Zustands angezeigt wird, und von einem gekennzeichneten Objekt, das in dem Bild des zweiten aktiven Zustands angezeigt wird, einen Katalog von einem oder mehreren Befehlen des zweiten aktiven Zustands erzeugt, die eine Funktion des zweiten aktiven Zustands kennzeichnen, welche in dem zweiten aktiven Zustand des Zielrechnerprogramms ausgeführt werden kann;
    das Mittel zur Kennzeichnung eines Vokabulars eines ersten aktiven Zustands ein Vokabular von akustischen Befehlsmodellen eines zweiten aktiven Zustands für den zweiten aktiven Zustand kennzeichnet, wobei das Vokabular des zweiten aktiven Zustands die akustischen Befehlsmodelle von dem Systemvokabular umfasst, das die Befehle des zweiten aktiven Zustands darstellt, wobei sich das Vokabular des zweiten aktiven Zustands mindestens teilweise von dem Vokabular des ersten aktiven Zustands unterscheidet; und
    die Spracherkennungseinheit den Wert von mindestens einem Merkmal einer Äußerung während eines jeden einer zweiten Folge von Zeitintervallen innerhalb des zweiten Zeitraums misst, um eine zweite Reihe von Merkmalssignalen zu erzeugen, wobei die Spracherkennungseinheit die zweite Reihe der Merkmalssignale für den zweiten Zeitraum mit jedem der akustischen Befehlsmodelle in dem Vokabular des zweiten aktiven Zustands vergleicht, um das Übereinstimmungsergebnis für die Äußerung und jedes akustische Befehlsmodell zu erzeugen, und wobei die Spracherkennungseinheit das Befehlssignal ausgibt, das dem Befehlsmodell von dem Vokabular des zweiten aktiven Zustands entspricht, welches das beste Übereinstimmungsergebnis aufweist.
  4. Interaktives Rechnersystem nach Anspruch 3, dadurch gekennzeichnet, dass bei dem Zielrechnerprogramm während eines jeden Zeitraums nur ein aktiver Zustand auftritt.
  5. Interaktives Rechnersystem nach Anspruch 4, dadurch gekennzeichnet, dass das Zielrechnerprogramm ein Betriebssystemprogramm umfasst.
  6. Interaktives Rechnersystem nach Anspruch 5, dadurch gekennzeichnet, dass das Zielrechnerprogramm ein Anwendungsprogramm und ein Betriebssystemprogramm umfasst.
  7. Interaktives Rechnersystem nach Anspruch 6, dadurch gekennzeichnet, dass das Zielrechnerprogramm zwei oder mehr Anwendungsprogramme und ein Betriebssystemprogramm umfasst.
  8. Interaktives Rechnersystem nach Anspruch 6, dadurch gekennzeichnet, dass mindestens einige Befehle für den aktiven Zustand Funktionen kennzeichnen, die an dem gekennzeichneten Objekt in dem Bild des aktiven Zustands für den aktiven Zustand ausgeführt werden können.
  9. Interaktives Rechnersystem nach Anspruch 8, dadurch gekennzeichnet, dass das gekennzeichnete Objekt in einem Bild eines aktiven Zustands ein oder mehrere Zeichen, Wörter, Symbole, Schaltflächen, Schiebeleisten, Schiebefelder, Listenfenster, Menüs, Markierungsfelder, Behälter oder Notizbücher umfasst.
  10. Interaktives Rechnersystem nach Anspruch 9, dadurch gekennzeichnet, dass die Spracherkennungseinheit zwei oder mehr Befehlssignale ausgibt, die den Befehlsmodellen von dem Vokabular des aktiven Zustands entsprechen, welche die besten Übereinstimmungsergebnisse für einen bestimmten Zeitraum aufweisen.
  11. Interaktives Rechnersystem nach Anspruch 10, dadurch gekennzeichnet, dass das Vokabular von akustischen Befehlsmodellen des aktiven Zustands für jeden aktiven Zustand des Weiteren eine Gruppe von globalen akustischen Befehlsmodellen umfasst, die globale Befehle darstellen, welche Funktionen kennzeichnen, die in jedem aktiven Zustand des Zielrechnerprogramms ausgeführt werden können.
  12. Interaktives Rechnersystem nach Anspruch 11, dadurch gekennzeichnet, dass das Anzeigemittel einen Bildschirm umfasst.
  13. Interaktives Rechnersystem nach Anspruch 11, dadurch gekennzeichnet, dass das Anzeigemittel sowohl das Bild des aktiven Zustands für den aktiven Zustand, der während eines Zeitraums auftritt, als auch mindestens einen Teil von einem oder mehreren Bildern für Programmzustände, die während des Zeitraums nicht auftreten, anzeigt.
  14. Verfahren zur Interaktion mit einem Rechner, das folgendes umfasst:
    Ausführen eines Zielrechnerprogramms auf einem Prozessor, das eine Reihe von aktiven Programmzuständen hat, die über eine Folge von Zeiträumen auftreten, wobei das Zielrechnerprogramm Datensignale eines Bildes eines aktiven Zustands erzeugt, die ein Bild des aktiven Zustands für einen aktiven Zustand des Zielrechnerprogramms darstellen, der während eines jeden Zeitraums auftritt, wobei jedes Bild des aktiven Zustands ein oder mehrere Objekte enthält;
    Anzeigen von mindestens einem Bild des ersten aktiven Zustands für einen ersten aktiven Zustand, der während eines ersten Zeitraums auftritt;
    Kennzeichnen eines Objekts, das in dem Bild des ersten aktiven Zustands angezeigt wird, und Erzeugen einer Liste von einem oder mehreren Befehlen des ersten aktiven Zustands von einem gekennzeichneten Objekt, die eine Funktion des ersten aktiven Zustands kennzeichnen, welche in dem ersten aktiven Zustand des Zielrechnerprogramms ausgeführt werden kann;
    Speichern eines Systemvokabulars von akustischen Befehlsmodellen, wobei jedes akustische Befehlsmodell eine oder mehrere Reihen von akustischen Merkmalswerten darstellt, die eine Äußerung von einem oder mehreren Wörtern darstellen, welche zu dem akustischen Befehlsmodell gehören;
    Kennzeichnen eines Vokabulars von akustischen Befehlsmodellen des ersten aktiven Zustands für den ersten aktiven Zustand, wobei das Vokabular des ersten aktiven Zustands die akustischen Befehlsmodelle von dem Systemvokabular umfasst, das die Befehle des ersten aktiven Zustands darstellt, wobei sich das Vokabular des ersten aktiven Zustands in Abhängigkeit von sowohl der Kennung des Zielrechnerprogramms als auch der Datensignale des Bildes des aktiven Zustands dynamisch ändert, die einen aktiven Zustand des Zielrechnerprogramms kennzeichnen; und
    Messen eines Werts von mindestens einem Merkmal einer Äußerung während eines jeden einer ersten Folge von Zeitintervallen innerhalb des ersten Zeitraums, um eine erste Reihe von Merkmalssignalen zu erzeugen;
    Vergleichen der ersten Reihe der Merkmalssignale mit jedem der akustischen Befehlsmodelle in dem Vokabular des ersten aktiven Zustands, um ein Übereinstimmungsergebnis für die Äußerung und jedes akustische Befehlsmodell zu erzeugen; und
    Ausgeben eines Befehlssignals, das dem Befehlsmodell von dem Vokabular des ersten aktiven Zustands entspricht, welches das beste Übereinstimmungsergebnis aufweist.
  15. Verfahren zur Interaktion mit einem Rechner nach Anspruch 14, dadurch gekennzeichnet, dass:
    das Vokabular des ersten aktiven Zustands wesentlich weniger als alle der akustischen Befehlsmodelle von dem Systemvokabular umfasst; und
    der Vergleichsschritt die gemessenen Merkmalssignale für den ersten Zeitraum nicht mit einem akustischen Befehlsmodell vergleicht, das sich nicht im Vokabular des ersten aktiven Zustands befindet.
  16. Verfahren zur Interaktion mit einem Rechner nach Anspruch 15, das des Weiteren die folgenden Schritte umfasst:
    Anzeigen von mindestens einem Bild eines zweiten aktiven Zustands, das sich von dem Bild des ersten aktiven Zustands unterscheidet, für einen zweiten aktiven Zustand, der während eines zweiten Zeitraums auftritt, welcher sich von dem ersten Zeitraum unterscheidet;
    Kennzeichnen eines Objekts, das in dem Bild des zweiten aktiven Zustands angezeigt wird, und Erzeugen eines Katalogs von einem oder mehreren Befehlen des zweiten aktiven Zustands von einem gekennzeichneten Objekt, die eine Funktion des zweiten aktiven Zustands kennzeichnen, welche in dem zweiten aktiven Zustand des Zielrechnerprogramms ausgeführt werden kann;
    Kennzeichnen eines Vokabulars von akustischen Befehlsmodellen eines zweiten aktiven Zustands für den zweiten aktiven Zustand, wobei das Vokabular des zweiten aktiven Zustands die akustischen Befehlsmodelle von dem Systemvokabular umfasst, das die Befehle des zweiten aktiven Zustands darstellt, wobei sich das Vokabular des zweiten aktiven Zustands mindestens teilweise von dem Vokabular des ersten aktiven Zustands unterscheidet;
    Messen des Werts von mindestens einem Merkmal einer Äußerung während eines jeden einer zweiten Folge von Zeitintervallen innerhalb des zweiten Zeitraums, um eine zweite Reihe von Merkmalssignalen zu erzeugen;
    Vergleichen der zweiten Reihe der Merkmalssignale für den zweiten Zeitraum mit jedem der akustischen Befehlsmodelle in dem Vokabular des zweiten aktiven Zustands, um das Übereinstimmungsergebnis für die Äußerung und jedes akustische Befehlsmodell zu erzeugen; und
    Ausgeben eines Befehlssignals, das dem Befehlsmodell von dem Vokabular des zweiten aktiven Zustands entspricht, welches das beste Übereinstimmungsergebnis aufweist.
  17. Verfahren zur Interaktion mit einem Rechner nach Anspruch 16, dadurch gekennzeichnet, dass bei dem Zielrechnerprogramm während eines jeden Zeitraums nur ein aktiver Zustand auftritt.
  18. Verfahren zur Interaktion mit einem Rechner nach Anspruch 17, dadurch gekennzeichnet, dass das Zielrechnerprogramm ein Betriebssystemprogramm umfasst.
  19. Verfahren zur Interaktion mit einem Rechner nach Anspruch 18, dadurch gekennzeichnet, dass das Zielrechnerprogramm ein Anwendungsprogramm und ein Betriebssystemprogramm umfasst.
  20. Verfahren zur Interaktion mit einem Rechner nach Anspruch 19, dadurch gekennzeichnet, dass das Zielrechnerprogramm zwei oder mehr Anwendungsprogramme und ein Betriebssystemprogramm umfasst.
  21. Verfahren zur Interaktion mit einem Rechner nach Anspruch 19, dadurch gekennzeichnet, dass mindestens einige Befehle für den aktiven Zustand Funktionen kennzeichnen, die an den gekennzeichneten Objekten in dem Bild des aktiven Zustands für den aktiven Zustand ausgeführt werden können.
  22. Verfahren zur Interaktion mit einem Rechner nach Anspruch 21, dadurch gekennzeichnet, dass das gekennzeichnete Objekt in dem Bild des aktiven Zustands ein oder mehrere Zeichen, Wörter, Symbole, Schaltflächen, Schiebeleisten, Schiebefelder, Listenfenster, Menüs, Markierungsfelder, Behälter oder Notizbücher umfasst.
  23. Verfahren zur Interaktion mit einem Rechner nach Anspruch 22, dadurch gekennzeichnet, dass der Schritt der Ausgabe eines Befehlssignals die Ausgabe von zwei oder mehr Befehlssignalen umfasst, die den Befehlsmodellen von dem Vokabular des aktiven Zustands entsprechen, welche die besten Übereinstimmungsergebnisse für einen bestimmten Zeitraum aufweisen.
  24. Verfahren zur Interaktion mit einem Rechner nach Anspruch 23, dadurch gekennzeichnet, dass das Vokabular von akustischen Befehlsmodellen für jeden aktiven Zustand des Weiteren eine Gruppe von globalen akustischen Befehlsmodellen umfasst, die globale Befehle darstellen, welche Funktionen kennzeichnen, die in jedem aktiven Zustand des Zielrechnerprogramms ausgeführt werden können.
  25. Vorfahren zur Interaktion mit einem Rechner nach Anspruch 24, das des Weiteren den Schritt der Anzeige von sowohl dem Bild des aktiven Zustands für den aktiven Zustand, der während eines Zeitraums auftritt, als auch von mindestens einem Teil von einem oder mehreren Bildern für Programmzustände, die während des Zeitraums nicht auftreten, umfasst.
EP94105293A 1993-04-21 1994-04-06 Interaktives Rechnersystem, das gesprochene Befehle erkennt Expired - Lifetime EP0621531B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US5095093A 1993-04-21 1993-04-21
US50950 1993-04-21

Publications (2)

Publication Number Publication Date
EP0621531A1 EP0621531A1 (de) 1994-10-26
EP0621531B1 true EP0621531B1 (de) 1999-09-29

Family

ID=21968512

Family Applications (1)

Application Number Title Priority Date Filing Date
EP94105293A Expired - Lifetime EP0621531B1 (de) 1993-04-21 1994-04-06 Interaktives Rechnersystem, das gesprochene Befehle erkennt

Country Status (8)

Country Link
US (1) US5664061A (de)
EP (1) EP0621531B1 (de)
JP (1) JP2856671B2 (de)
KR (1) KR970006403B1 (de)
CN (1) CN1086484C (de)
AT (1) ATE185203T1 (de)
CA (1) CA2115210C (de)
DE (1) DE69420888T2 (de)

Families Citing this family (103)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5909666A (en) * 1992-11-13 1999-06-01 Dragon Systems, Inc. Speech recognition system which creates acoustic models by concatenating acoustic models of individual words
US5764852A (en) * 1994-08-16 1998-06-09 International Business Machines Corporation Method and apparatus for speech recognition for distinguishing non-speech audio input events from speech audio input events
JPH09510803A (ja) * 1995-01-18 1997-10-28 フィリップス エレクトロニクス ネムローゼ フェンノートシャップ オペレータ介入により支援しうるヒューマン−マシンダイアログを提供する方法及び装置
JP3750150B2 (ja) * 1995-03-30 2006-03-01 三菱電機株式会社 移動体通信端末
JPH09149157A (ja) * 1995-11-24 1997-06-06 Casio Comput Co Ltd 通信端末装置
GB9602701D0 (en) * 1996-02-09 1996-04-10 Canon Kk Image manipulation
US5960395A (en) 1996-02-09 1999-09-28 Canon Kabushiki Kaisha Pattern matching method, apparatus and computer readable memory medium for speech recognition using dynamic programming
US5819225A (en) * 1996-05-30 1998-10-06 International Business Machines Corporation Display indications of speech processing states in speech recognition system
US5867817A (en) * 1996-08-19 1999-02-02 Virtual Vision, Inc. Speech recognition manager
US6654955B1 (en) * 1996-12-19 2003-11-25 International Business Machines Corporation Adding speech recognition libraries to an existing program at runtime
US5897618A (en) * 1997-03-10 1999-04-27 International Business Machines Corporation Data processing system and method for switching between programs having a same title using a voice command
US6192338B1 (en) * 1997-08-12 2001-02-20 At&T Corp. Natural language knowledge servers as network resources
EP0962014B1 (de) * 1997-12-30 2003-11-12 Koninklijke Philips Electronics N.V. Spracherkennungsvorrichtung unter verwendung eines befehlslexikons
US6301560B1 (en) * 1998-01-05 2001-10-09 Microsoft Corporation Discrete speech recognition system with ballooning active grammar
US6298324B1 (en) * 1998-01-05 2001-10-02 Microsoft Corporation Speech recognition system with changing grammars and grammar help command
US6671745B1 (en) * 1998-03-23 2003-12-30 Microsoft Corporation Application program interfaces and structures in a resource limited operating system
US6144938A (en) 1998-05-01 2000-11-07 Sun Microsystems, Inc. Voice user interface with personality
FI981154A (fi) * 1998-05-25 1999-11-26 Nokia Mobile Phones Ltd Menetelmä ja laite puheen tunnistamiseksi
US7082391B1 (en) * 1998-07-14 2006-07-25 Intel Corporation Automatic speech recognition
US6243076B1 (en) 1998-09-01 2001-06-05 Synthetic Environments, Inc. System and method for controlling host system interface with point-of-interest data
FR2783625B1 (fr) * 1998-09-21 2000-10-13 Thomson Multimedia Sa Systeme comprenant un appareil telecommande et un dispositif de telecommande vocale de l'appareil
US6928614B1 (en) 1998-10-13 2005-08-09 Visteon Global Technologies, Inc. Mobile office with speech recognition
US6240347B1 (en) 1998-10-13 2001-05-29 Ford Global Technologies, Inc. Vehicle accessory control with integrated voice and manual activation
US6230129B1 (en) * 1998-11-25 2001-05-08 Matsushita Electric Industrial Co., Ltd. Segment-based similarity method for low complexity speech recognizer
US7206747B1 (en) 1998-12-16 2007-04-17 International Business Machines Corporation Speech command input recognition system for interactive computer display with means for concurrent and modeless distinguishing between speech commands and speech queries for locating commands
US6192343B1 (en) 1998-12-17 2001-02-20 International Business Machines Corporation Speech command input recognition system for interactive computer display with term weighting means used in interpreting potential commands from relevant speech terms
US6233560B1 (en) 1998-12-16 2001-05-15 International Business Machines Corporation Method and apparatus for presenting proximal feedback in voice command systems
US8275617B1 (en) 1998-12-17 2012-09-25 Nuance Communications, Inc. Speech command input recognition system for interactive computer display with interpretation of ancillary relevant speech query terms into commands
US6937984B1 (en) 1998-12-17 2005-08-30 International Business Machines Corporation Speech command input recognition system for interactive computer display with speech controlled display of recognized commands
US7567677B1 (en) * 1998-12-18 2009-07-28 Gateway, Inc. Noise reduction scheme for a computer system
US6230135B1 (en) 1999-02-02 2001-05-08 Shannon A. Ramsay Tactile communication apparatus and method
US6408301B1 (en) 1999-02-23 2002-06-18 Eastman Kodak Company Interactive image storage, indexing and retrieval system
US6345254B1 (en) * 1999-05-29 2002-02-05 International Business Machines Corp. Method and apparatus for improving speech command recognition accuracy using event-based constraints
US6421655B1 (en) * 1999-06-04 2002-07-16 Microsoft Corporation Computer-based representations and reasoning methods for engaging users in goal-oriented conversations
US6308157B1 (en) * 1999-06-08 2001-10-23 International Business Machines Corp. Method and apparatus for providing an event-based “What-Can-I-Say?” window
US6871179B1 (en) * 1999-07-07 2005-03-22 International Business Machines Corporation Method and apparatus for executing voice commands having dictation as a parameter
US6374226B1 (en) * 1999-08-06 2002-04-16 Sun Microsystems, Inc. System and method for interfacing speech recognition grammars to individual components of a computer program
US6510414B1 (en) 1999-10-05 2003-01-21 Cisco Technology, Inc. Speech recognition assisted data entry system and method
US6594630B1 (en) * 1999-11-19 2003-07-15 Voice Signal Technologies, Inc. Voice-activated control for electrical device
US7319962B2 (en) * 1999-12-24 2008-01-15 Medtronic, Inc. Automatic voice and data recognition for implanted medical device instrument systems
FR2803927B1 (fr) * 2000-01-14 2002-02-22 Renault Procede et dispositif de commande d'equipements embarques sur un vehicule utilisant la reconnaissance vocale
US7047196B2 (en) 2000-06-08 2006-05-16 Agiletv Corporation System and method of voice recognition near a wireline node of a network supporting cable television and/or video delivery
US6408277B1 (en) 2000-06-21 2002-06-18 Banter Limited System and method for automatic task prioritization
US9699129B1 (en) 2000-06-21 2017-07-04 International Business Machines Corporation System and method for increasing email productivity
US8290768B1 (en) 2000-06-21 2012-10-16 International Business Machines Corporation System and method for determining a set of attributes based on content of communications
US6510410B1 (en) * 2000-07-28 2003-01-21 International Business Machines Corporation Method and apparatus for recognizing tone languages using pitch information
JP3774698B2 (ja) * 2000-10-11 2006-05-17 キヤノン株式会社 情報処理装置、情報処理方法及び記憶媒体
US7644057B2 (en) 2001-01-03 2010-01-05 International Business Machines Corporation System and method for electronic communication management
US8095370B2 (en) 2001-02-16 2012-01-10 Agiletv Corporation Dual compression voice recordation non-repudiation system
DE10115899B4 (de) * 2001-03-30 2005-04-14 Siemens Ag Verfahren zur Erstellung von Computer-Programmen mittels Spracherkennung
US7409349B2 (en) * 2001-05-04 2008-08-05 Microsoft Corporation Servers for web enabled speech recognition
US7610547B2 (en) * 2001-05-04 2009-10-27 Microsoft Corporation Markup language extensions for web enabled recognition
US20020178182A1 (en) * 2001-05-04 2002-11-28 Kuansan Wang Markup language extensions for web enabled recognition
US7506022B2 (en) * 2001-05-04 2009-03-17 Microsoft.Corporation Web enabled recognition architecture
US7203188B1 (en) 2001-05-21 2007-04-10 Estara, Inc. Voice-controlled data/information display for internet telephony and integrated voice and data communications using telephones and computing devices
US7020841B2 (en) 2001-06-07 2006-03-28 International Business Machines Corporation System and method for generating and presenting multi-modal applications from intent-based markup scripts
KR101049748B1 (ko) * 2001-07-03 2011-07-19 코닌클리케 필립스 일렉트로닉스 엔.브이. 인터액티브 디스플레이 및 메시지를 디스플레이하는 방법
US8229753B2 (en) 2001-10-21 2012-07-24 Microsoft Corporation Web server controls for web enabled recognition and/or audible prompting
US7711570B2 (en) 2001-10-21 2010-05-04 Microsoft Corporation Application abstraction with dialog purpose
US20040034529A1 (en) * 2002-08-14 2004-02-19 Hooper Howard Gaines Multifunction printer that converts and prints voice data
US7421390B2 (en) * 2002-09-13 2008-09-02 Sun Microsystems, Inc. Method and system for voice control of software applications
US7389230B1 (en) 2003-04-22 2008-06-17 International Business Machines Corporation System and method for classification of voice signals
US20040230637A1 (en) * 2003-04-29 2004-11-18 Microsoft Corporation Application controls for speech enabled recognition
US7363060B2 (en) * 2003-05-02 2008-04-22 Nokia Corporation Mobile telephone user interface
US20050187913A1 (en) 2003-05-06 2005-08-25 Yoram Nelken Web-based customer service interface
US8495002B2 (en) 2003-05-06 2013-07-23 International Business Machines Corporation Software tool for training and testing a knowledge base
US20050009604A1 (en) * 2003-07-11 2005-01-13 Hsien-Ta Huang Monotone voice activation device
US7552055B2 (en) * 2004-01-10 2009-06-23 Microsoft Corporation Dialog component re-use in recognition systems
US8160883B2 (en) 2004-01-10 2012-04-17 Microsoft Corporation Focus tracking in dialogs
CN1691581B (zh) * 2004-04-26 2010-04-28 彭诗力 基于特征值的多模式匹配方法
CN100403255C (zh) * 2005-03-17 2008-07-16 英华达(上海)电子有限公司 一种利用声音操作游戏的方法
JP4667138B2 (ja) * 2005-06-30 2011-04-06 キヤノン株式会社 音声認識方法及び音声認識装置
KR100632400B1 (ko) * 2005-11-11 2006-10-11 한국전자통신연구원 음성 인식을 이용한 입출력 장치 및 그 방법
US8229733B2 (en) * 2006-02-09 2012-07-24 John Harney Method and apparatus for linguistic independent parsing in a natural language systems
WO2008024800A2 (en) * 2006-08-21 2008-02-28 Western Slope Utilities, Inc. Systems and methods for swab transport in pipeline rehabilitation
WO2008136081A1 (ja) * 2007-04-20 2008-11-13 Mitsubishi Electric Corporation ユーザインタフェース装置及びユーザインタフェース設計装置
US8620652B2 (en) * 2007-05-17 2013-12-31 Microsoft Corporation Speech recognition macro runtime
US8538757B2 (en) * 2007-05-17 2013-09-17 Redstart Systems, Inc. System and method of a list commands utility for a speech recognition command system
US8150699B2 (en) * 2007-05-17 2012-04-03 Redstart Systems, Inc. Systems and methods of a structured grammar for a speech recognition command system
US20080312929A1 (en) * 2007-06-12 2008-12-18 International Business Machines Corporation Using finite state grammars to vary output generated by a text-to-speech system
US7962344B2 (en) * 2007-06-29 2011-06-14 Microsoft Corporation Depicting a speech user interface via graphical elements
US8165886B1 (en) 2007-10-04 2012-04-24 Great Northern Research LLC Speech interface system and method for control and interaction with applications on a computing system
US8595642B1 (en) 2007-10-04 2013-11-26 Great Northern Research, LLC Multiple shell multi faceted graphical user interface
CN101436404A (zh) * 2007-11-16 2009-05-20 鹏智科技(深圳)有限公司 可会话的类生物装置及其会话方法
US8958848B2 (en) 2008-04-08 2015-02-17 Lg Electronics Inc. Mobile terminal and menu control method thereof
KR20090107365A (ko) * 2008-04-08 2009-10-13 엘지전자 주식회사 이동 단말기 및 그 메뉴 제어방법
US8762963B2 (en) * 2008-12-04 2014-06-24 Beck Fund B.V. L.L.C. Translation of programming code
KR101528266B1 (ko) * 2009-01-05 2015-06-11 삼성전자 주식회사 휴대 단말기 및 그의 응용프로그램 제공 방법
US8606578B2 (en) * 2009-06-25 2013-12-10 Intel Corporation Method and apparatus for improving memory locality for real-time speech recognition
CN101976186B (zh) * 2010-09-14 2013-04-03 方正科技集团苏州制造有限公司 一种计算机语音识别方法及其计算机
US20120089392A1 (en) * 2010-10-07 2012-04-12 Microsoft Corporation Speech recognition user interface
US20120155663A1 (en) * 2010-12-16 2012-06-21 Nice Systems Ltd. Fast speaker hunting in lawful interception systems
WO2012169679A1 (ko) * 2011-06-10 2012-12-13 엘지전자 주식회사 디스플레이 장치, 디스플레이 장치의 제어 방법 및 디스플레이 장치의 음성인식 시스템
WO2013022135A1 (en) * 2011-08-11 2013-02-14 Lg Electronics Inc. Electronic device and method of controlling the same
US10186262B2 (en) * 2013-07-31 2019-01-22 Microsoft Technology Licensing, Llc System with multiple simultaneous speech recognizers
US9653073B2 (en) * 2013-11-26 2017-05-16 Lenovo (Singapore) Pte. Ltd. Voice input correction
US9589564B2 (en) 2014-02-05 2017-03-07 Google Inc. Multiple speech locale-specific hotword classifiers for selection of a speech locale
KR102281178B1 (ko) * 2014-07-09 2021-07-23 삼성전자주식회사 멀티-레벨 음성 인식 방법 및 장치
US11741951B2 (en) * 2019-02-22 2023-08-29 Lenovo (Singapore) Pte. Ltd. Context enabled voice commands
CN110598671B (zh) * 2019-09-23 2022-09-27 腾讯科技（深圳）有限公司 基于文本的虚拟形象行为控制方法、设备和介质
EP4302178A1 (de) * 2021-03-01 2024-01-10 Apple Inc. Virtuelle objektplatzierung auf der basis von referenzausdrücken
CN113590360A (zh) * 2021-08-03 2021-11-02 北京博睿宏远数据科技股份有限公司 一种实现函数hook的方法、装置、计算机设备及存储介质
US12100395B2 (en) * 2021-11-30 2024-09-24 Google Llc Dynamic assistant suggestions during assistant browsing

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CH644246B (fr) * 1981-05-15 1900-01-01 Asulab Sa Dispositif d'introduction de mots a commande par la parole.
JPS58195957A (ja) * 1982-05-11 1983-11-15 Casio Comput Co Ltd 音声によるプログラム実行方式
US4704696A (en) * 1984-01-26 1987-11-03 Texas Instruments Incorporated Method and apparatus for voice control of a computer
US4980918A (en) * 1985-05-09 1990-12-25 International Business Machines Corporation Speech recognition system with efficient storage and rapid assembly of phonological graphs
US4759068A (en) * 1985-05-29 1988-07-19 International Business Machines Corporation Constructing Markov models of words from multiple utterances
US4776016A (en) * 1985-11-21 1988-10-04 Position Orientation Systems, Inc. Voice control system
US4839634A (en) * 1986-12-01 1989-06-13 More Edward S Electro-optic slate for input/output of hand-entered textual and graphic information
PH24865A (en) * 1987-03-24 1990-12-26 Ibm Mode conversion of computer commands
US4931950A (en) * 1988-07-25 1990-06-05 Electric Power Research Institute Multimedia interface and method for computer system
US5157384A (en) * 1989-04-28 1992-10-20 International Business Machines Corporation Advanced user interface
DE3928049A1 (de) * 1989-08-25 1991-02-28 Grundig Emv Sprachgesteuertes archivsystem
EP0438662A2 (de) * 1990-01-23 1991-07-31 International Business Machines Corporation Einrichtung und Verfahren zur Gruppierung von Äusserungen eines Phonemen in von Kontexten abhängigen Kategorien, die auf Tonähnlichkeit basiert sind für automatische Spracherkennung
JPH04163618A (ja) * 1990-10-26 1992-06-09 Oki Electric Ind Co Ltd 音声操作コンピュータ
US5182773A (en) * 1991-03-22 1993-01-26 International Business Machines Corporation Speaker-independent label coding apparatus
WO1993003453A1 (en) * 1991-08-02 1993-02-18 Broderbund Software, Inc. System for interactve performance and animation of prerecorded audiovisual sequences

Also Published As

Publication number Publication date
CN1105464A (zh) 1995-07-19
CN1086484C (zh) 2002-06-19
US5664061A (en) 1997-09-02
JPH06348452A (ja) 1994-12-22
KR970006403B1 (ko) 1997-04-28
CA2115210C (en) 1997-09-23
CA2115210A1 (en) 1994-10-22
DE69420888T2 (de) 2000-04-27
DE69420888D1 (de) 1999-11-04
JP2856671B2 (ja) 1999-02-10
ATE185203T1 (de) 1999-10-15
EP0621531A1 (de) 1994-10-26

Similar Documents

Publication Publication Date Title
EP0621531B1 (de) Interaktives Rechnersystem, das gesprochene Befehle erkennt
US8645122B1 (en) Method of handling frequently asked questions in a natural language dialog service
US7869998B1 (en) Voice-enabled dialog system
US5615296A (en) Continuous speech recognition and voice response system and method to enable conversational dialogues with microprocessors
US5920837A (en) Word recognition system which stores two models for some words and allows selective deletion of one such model
Rudnicky et al. Survey of current speech technology
US5333236A (en) Speech recognizer having a speech coder for an acoustic match based on context-dependent speech-transition acoustic models
EP0974141B1 (de) Erweiterbares spracherkennungssystem mit einer audio-rückkopplung
US6910012B2 (en) Method and system for speech recognition using phonetically similar word alternatives
US6526380B1 (en) Speech recognition system having parallel large vocabulary recognition engines
US5465317A (en) Speech recognition system with improved rejection of words and sounds not in the system vocabulary
US20090070113A1 (en) System for handling frequently asked questions in a natural language dialog service
US20170169815A1 (en) System and methods for adapting neural network acoustic models
US20020123894A1 (en) Processing speech recognition errors in an embedded speech recognition system
CN109313892A (zh) 稳健的语言识别方法和系统
CN111177350A (zh) 智能语音机器人的话术形成方法、装置和系统
US7461000B2 (en) System and methods for conducting an interactive dialog via a speech-based user interface
CN113223560A (zh) 情绪识别方法、装置、设备及存储介质
US6345254B1 (en) Method and apparatus for improving speech command recognition accuracy using event-based constraints
US20040044528A1 (en) Method and apparatus for generating decision tree questions for speech processing
Di Fabbrizio et al. AT&t help desk.
CN115985320A (zh) 智能设备控制方法、装置、电子设备及存储介质
CN113555016A (zh) 语音交互方法、电子设备及可读存储介质
Helander et al. Systems design for automated speech recognition
US20240257804A1 (en) Language model customization techniques and applications thereof

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE CH DE ES FR GB IT LI NL SE

17P Request for examination filed

Effective date: 19950227

17Q First examination report despatched

Effective date: 19980126

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAG Despatch of communication of intention to grant

Free format text: ORIGINAL CODE: EPIDOS AGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH DE ES FR GB IT LI NL SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 19990929

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19990929

Ref country code: LI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19990929

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRE;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.SCRIBED TIME-LIMIT

Effective date: 19990929

Ref country code: ES

Free format text: THE PATENT HAS BEEN ANNULLED BY A DECISION OF A NATIONAL AUTHORITY

Effective date: 19990929

Ref country code: CH

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19990929

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19990929

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 19990929

REF Corresponds to:

Ref document number: 185203

Country of ref document: AT

Date of ref document: 19991015

Kind code of ref document: T

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REF Corresponds to:

Ref document number: 69420888

Country of ref document: DE

Date of ref document: 19991104

ET Fr: translation filed
NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed
REG Reference to a national code

Ref country code: GB

Ref legal event code: IF02

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20100617 AND 20100623

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20130403

Year of fee payment: 20

Ref country code: DE

Payment date: 20130403

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20130625

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 69420888

Country of ref document: DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20140405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20140405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20140408