Connect public, paid and private patent data with Google Patents Public Datasets

Method and apparatus for providing a human-machine dialog supportable by operator intervention

Download PDF

Info

Publication number
US5745877A
US5745877A US08587190 US58719096A US5745877A US 5745877 A US5745877 A US 5745877A US 08587190 US08587190 US 08587190 US 58719096 A US58719096 A US 58719096A US 5745877 A US5745877 A US 5745877A
Authority
US
Grant status
Grant
Patent type
Prior art keywords
system
user
dialog
operator
verbal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US08587190
Inventor
Aloijsius J. Nijmam
Harald Aust
Peter Besting
Martin Oerder
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
North American Philips Lighting Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Grant date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRICAL DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
    • G09B7/04Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question, supplying a further explanation
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Successive sections of a computer program of a user interactive system are carried out. A user presents successive verbal items such as answers, to verbal items such as questions presented by the system. Each question corresponds to a respective program section. The user item is recognized and a next machine item is presented, if possible and appropriate. The recognizing is evaluated, and on the basis of the evaluating a progress score is kept. If the progress score attains a particular level, the system enables an operator to intervene in the dialogue in an unobtrusive manner. If a plurality of dialogs are supervised in parallel, the enabling is on the basis of a progress ranking among the dialogs. The intervention may be realized by emulating a user item in the form of speech or in the form of a mechanically entered user item.

Description

BACKGROUND TO THE INVENTION

The invention relates to a method of carrying out a human-machine dialog whilst executing successive sections of a computer program of a user interactive system in accordance with successive verbal user items presented by a user as reacting to successive verbal system items presented by the system to the user, each such verbal system item corresponding to a respective program section, said method comprising successive steps of:

a. presentation by the system to the user of a verbal system item corresponding to a program section;

b. recognition by the system of a verbal user item presented by the user to the system.

A method of this kind has been disclosed in U.S. Pat. 4,959,864 (PHN11266) assigned to the present assignee. The known art, during said dialog, records the number of user protests and when scoring of said protests attains a particular score level, the system automatically and unattendedly transits to a more explicit level of reacting. The present inventors have found that in many instances, the user's inability to correctly interact with the machine will render the dialog ineffective, regardless of the care taken in developing the dialogue's organization, as represented by the program. On the other hand, switch-over to a direct dialog with a human operator is sometimes felt by the user person as overbearing. On the other hand, intervention by an operator often necessitates the operator to carry on the dialog up to its end. In this respect, the present invention should allow the operator not only to enter into the dialog, but also to leave the dialog again without the latter's structure becoming disagreeable to the user person. A major cause for the dialog's insufficient progress is that in certain instances the machine has great difficulty to understand the user, either because the user does not understand the machine questions, or more frequently, because the speech recognition proves ineffective viz a viz the particular user. The latter problem can be caused by properties of the user's speech such as a foreign or other accent, or by system problems such as bad line quality in case of a telephone answering system.

SUMMARY TO THE INVENTION

In consequence, amongst other things, it is an object of the present invention to provide a method according to the preamble, where the transition of the system to another level of human interaction is kept more or less invisible. Now, according to one of its aspects, the invention is characterized in that evaluation of a result of said recognition, and reverting to part a. whilst on the basis of said recognition, selecting among a plurality of such program sections each associated to a subsequent verbal system item, and on the basis of said evaluation keeping a progress score pertaining to said dialog, and under control of a particular progress score level enabling a system operator to intervene in said dialog in an unobtrusive manner. In this way, the machine as it were masks the actions by the operator, whilst at the same time allowing the operator intervention to produce either correctly recognizable entries, or such entries that are based on correct understanding of the dialog process.

Advantageously, the method comprises the carrying out in parallel of a plurality of dialogs, whilst comparing and ranking separate progress score levels each pertaining to a respective said dialog, and under control of a particular progress score ranking among said dialogs enabling a system operator to intervene in the associated dialog in an unobtrusive manner. In this way, one or more human operators can supervise a plurality of dialogs that are carried on in parallel, whilst the machine indicates the dialog or dialogs that is or are most in need for human intervention, such as through a relatively slow progress. Alternatively, in certain dialogs, such as those pertaining to a learning system, rather the dialog progressing fastest could warrant intervention.

Advantageously, said enabling allows the system operator to emulate a user person by means of an emulated verbal user item presented to the system. The intervention may be made by keyboard, but it is often faster and more effective to replace a user-provided item, such as an answer to a question, directly by a spoken answer by the operator. The advantage comes about especially if the user is mispronouncing a key word of the answer, such as the name of a town, or generally if the user has impaired speech. On the other hand, mechanical input such as by mouse or keyboard can sometimes be more suitable.

The invention also relates to an apparatus for implementing the method. Further advantageous aspects of the invention are recited in dependent claims.

BRIEF DESCRIPTION OF THE DRAWING

These and other aspects and advantages of the invention will be discussed more in detail hereinafter with reference to disclosure of preferred embodiments, and in particular with reference to the appended Figures, that show:

FIG. 1, a multi-dialog system according to the invention;

FIG. 2, a flow chart according to the invention;

FIG. 3, a functional block diagram of the invention.

DETAILED DISCLOSURE OF PREFERRED EMBODIMENTS

Hereinafter, a preferred embodiment of the invention will be disclosed with respect to an enquiry system for train connections. However, many other dialog environments are feasible, such as other enquiry systems, e.g. a telephone directory enquiring system, or a help desk for problems with household appliances or with the filling out of a tax form. Another environment would be a learning system based on a dialog structure. Whereas in an enquiry system, the machine usually poses a series of questions to find out what information the user person eventually wants, in other environments the distribution between questions and answers is less unsymmetric.

Now, FIG. 1 shows a multi-dialog system according to the invention for implementing an enquiry system. The system is able to maintain three dialogues in parallel under supervision by a single operator. In practice, these numbers are non-limitative. The system need not be uniform in that other aspects, such as geographical distances, user properties and different operator skills may play a part. The system has three bidirectional user interface devices 20-24, that operate exclusively with an audio channel. An interface device may have a loudspeaker and a microphone. Another solution that these interface devices couple to a telephone line, that may be an ISDN line accommodating digital data transfer. Alternatively, they may be supplemented by a video output channel for more or less persistently displaying key information, such as a list of departure times, recognized town names or fare amounts. Elements 26-30 interface the analog audio channels to the digital data processing in the remainder of the system; in an ISDN-environment, these would not be necessary. Now, the audio input is sampled and digitized. The output packages are converted to analog and if necessary, filtered. Blocks 31, 32, 33 are speech recognizers. For the system according to the embodiment, that was built for the country of Germany, these may recognize a vocabulary of 1100 names of stations and 900 hundred other words, such as times and time qualifiers of day (such as `early`, `afternoon`, name of day and name of month), a set of terms that are privy to the application, such as `express train`, `quick`, `first class`, etcetera, and finally a set of common words. These recognized terms are outputted in the sequence of uttering, and in the form of a bit string code. Blocks 34, 36, 38 represent the respective dialog control that interpret results recognized on various different levels as will be discussed hereinafter, and determine the status of the dialog. If further questions are necessary, these are generated and outputted to associated blocks 26, 28, 30 for presentation to the user. If the control decides that it knows the item that the user person actually wants, this item is also presented to the user in question. Blocks 40, 42, 44 keep track of the recognition processes so far. For example, they may store verbal user items, either in the form as presented for listening in by the operator, or in the form as recognized text for viewing by the operator on a visual operator display not shown explicitly. Also, each block 40-44 keeps a score of the associated progress; this may be in the form of a number or otherwise. Blocks 40-44 output to the operator control system 52. The operator control system may be a full-fledged work station with bidirectional supplementary audio channel not shown, or with only a subset of the facilities that commonly are provided in such work station. In the work station, the progress scores for the respective processes are automatically ranked from `bad` to `good`, and a selection of the ones with the least progress is visually presented to the operator, preferably in the order of quality, so that the operator would instantly know which dialog would be the one progressing slowest. This presenting may be done in the form of displaying the verbal items exchanged most recently by the machine and by the user in the form of text, in the user's case as far as recognized. Another way of presenting is by replaying the verbal items in the form as actually exchanged, either at their normal speed, or selectably at a higher than normal speed. This will often reveal the verbal user items that were too difficult for the system to recognize. The operator may select one of the dialogs, such as by cursoring an associated display field. Subsequently, the operator via the lines 46, 48, 50 may present emulated user items to the system. As shown, these are coupled in the form of keyed-in user answers into the associated dialog control blocks 34, 36, 38. Another manner would be that the operator by means of speech presents emulated verbal user items to the associated speech recognizer block 31, 32, 33 in a way not shown for clarity. If this operator presentation helps the progress of the dialog in question, the user will receive a verbal machine item that is better suited to his own needs. In many cases, this will render further operator intervention superfluous. In similar way, block 52 may represent two or more operator stations, so that each of the operators can intervene independently in a respective dialog that is progressing too slowly.

Now, the recognition of the verbal user items, which in this embodiment are generally answers to machine-generated questions, may have various outcomes:

1. No speech received: this is treated as a pause in speech, and the system will wait for some time. If the pause is too long, the last question is repeated;

2. No useful content recognized;

3. No content of an expected category recognized, e.g. a question: "what class of transportation do you want?" is answered by "wednesday"; in these cases the system repeats the last question, preceded by a statement "I did not understand your answer". However, an unexpected answer may be a reasonable one to another question, such as the next one to be presented by the system. In the latter case, the next system question contains the content of the most recent answer as a qualifier, and a positive acknowledgement thereof by the user causes the recognized content to be retained for later use.

4. No reasonable content recognized, e.g. an illogical detour is specified; also in this case the procedure of #3 is followed.

5. A sensible and expectable verbal user item is recognized. Thereupon, the system presents the next question to the user, if applicable.

The verbal machine items, generated upon one of the recognition results listed hereabove, may be of various character. However, generally, they are put into a questionary form, while containing such elements that have, sometimes tentatively, been recognized. The effect of such repeating is that the user person often needs to utter only one, or only a few words. Even if the user supplements this minimally required information with extra speech, the recognition or otherwise of the latter is not crucial for the progress of the dialog. Furthermore, in a train connection enquiry system, each eventual machine-generated outcome is a particular train connection that has only a finite set of parameters, such as:

departure station and time

arrival station and time

intermediate routing and changeovers

class: first, second

character: local, semi-express, express

extras: sleeping car, restaurant car.

This means that the system only needs to have questions answered with respect to each of those parameters and only as far as they are applicable. These limited requirements make the dialog structure relatively simple. Therefore, the dialog control undertakes to successively get answers on all necessary parameter aspects of the eventual outcome, so that the ultimate answer needs to consist of as few train connections as possible, usually no more than two. Specific type of machine questions are:

From (To) what station do you want to travel?

Here the machine undertakes to recognize the destination name, and also to maintain correct linking between the words `from-to` and the two station names. After recognizing one of the combinations identifying starting point or destination, the machine poses a question about the other end of the journey:

From (To) what station do you want to travel to (from) <name of destination (starting point) station>?

After recognizing the two combinations, the system asks required day and time, whilst repeating the two identifiers of the journey. When the system is not quite sure about the answer(s) received, it may repeat:

So you want to travel from <name> to <name>?

In this way the system proceeds to successively identify the various parameters for pinpointing the target connection. If the user gives statements that are outside the question actually answered, but relevant to the connection to be selected, these items are nevertheless retained in the system and repeated in the next question.

Now, in blocks 40-44 a progress score of the various dialogs is maintained. Various policies thereto are feasible. A first one is to accumulate the number of repeat questions presented to the user after answers that did not lead to any progress in the dialog. This can be combined or not with resetting of the accumulated sum in case of a well-recognized and fully functional answer. A second policy is to accumulate the total number of questions presented by the machine. If this number gets too high, it is proof that the dialog proceeds too slowly. Various other tactics are open to the skilled art practitioner.

FIG. 2 shows an exemplary flow chart of a method according to the invention. In the Figure, only a single user process has been shown, inasmuch as a hardware realization pertaining to a plurality of dialogs in parallel has been shown in FIG. 1 already. Furthermore, for reasons of clarity, only an elementary system has been presented. In practice, the structure of the program that generates the dialog is much more complicated. For a more extensive disclosure of such a system, see German Patent Application P 44 32 632.7, corresponding U.S. Ser. No. 08/526,017 to the present assignee, herein incorporated by reference. It is understood that various elements of FIG. 2 can be realized as well in a software context. Now in this Figure, block 60 represents the initializing of the system, or as the case may be, pertaining to only one of the dialogs. Block 62 represents the outputting of an introductory message by the system. Block 64 represents the generating of the first question by the system: "Where do you want to go?". The embodiment relates to the case where the starting point is known, for example, because of the location of the caller. In block 66 the system tries to recognize the next received answer as the name of a destination station. As stated with respect to FIG. 1, various types of unrecognizable or illogical answers may be forthcoming. In any of these cases, the system goes to block 68 and asks the user person to make a new statement indicating the destination. If the problem was unrecognizability of the answer, the statement may have its content adapted to the answer actually received, such as: "<Name> is the station where you are at present; where do you want to go?", in case the user accidentally gave the starting location. A similar phrase applies when the starting location had already been given by the user.

"I did not understand your answer; where do you want to go?", if the recognized message did not contain the name of a relevant station.

"Please repeat your answer", in case the recognition of a name was ambiguous, such as in case where two stations have nearly identical names.

If a correct destination name was recognized, the system goes to block 70 and checks whether the recognized name indicates a unique station, or, given the departure station, only a single station would be relevant. In the positive case, the system goes to block 78. The negative may occur, when a particular city has more than one station. Thereupon, in block 72 the system asks the user to specify which station pertaining to the destination in question. In block 74, the system checks whether a correct indication has been recognized. If no, a similar loop as regarding blocks 66, 68 is executed, which loop has not been shown for clarity. If a correct station name has been recognized, the system goes to block 78. Here, the user is asked to specify the hour of travel, which is treated in similar way as the destination was found in blocks 64-74. In block 80, the user is asked to specify a route. In block 82, the user is asked to specify extras, such as first or second class. In block 84 the system outputs the necessary information, possibly on a display screen or by means of hard copy. In block 86, the session is closed. In block 88, the dialog is terminated. Various additional speech outputs may be given by the system, such as repeating the recognized answer in a textual context, presenting the user with a binary choice, etcetera, but these do not represent elements of the invention proper. The operator may intervene when activation of blocks like block 68 signals that progress is too slow. In that case, the flow diagram of FIG. 2 is either left, when the operator fully takes over, but according to the invention, the operator may also just emulate one of the user items, whereupon the flow chart is traversed in exactly the same way as when a correct answer would have been recognized. The measuring of the progress actually made, may be based on the number of times a repeat block like block 68 is activated. These numbers may be accumulated continually. Various types of non-standard or low progress may cause respective different progress score signallings. Alternatively, a period of fast progress, as measured be securing of successive travel parameters by the system, may actually diminish the score, and thereby change the ranking of the process in question. As stated earlier, the blocks like 78-82 need not be positioned according to a linear sequence, but may be arranged more in parallel, so that their functionalities may be called for in about any stage of the dialog process. Every recognized parameter is then stored as an ultimate parameter of the train connection sought. It should be noted that the present invention does not relate to the structure of the dialog proper, but rather to measures taken when progress is too slow (or too fast).

FIG. 3 shows a functional block diagram of the invention, in particular as regarding the recognition process. Element 94 is a telephone line connected to telephone interface 96. Block 98 represents the speech recognition sub-process, that will output a string of provisionally recognized words. The string may be a multiple parallel one, in that more than one outcome would be feasible, and may have the character of a trellis. Block 100 represents the speech understanding sub-process. Here, the recognized words are evaluated as to their meaning, such as to whether a station name is recognized in conjunction with a directional qualifier `to` or `from`, or a time or time interval indication. Block 102 represents the dialog control sub-process that compares the speech understood by block 100 with expectable information in view of one or more most recent questions presented by the system. On account of this, the next question to be presented to the user is selected, or as the case may be, the final answer. Moreover, the dialog control subsystem maintains a progress score in block 112. Under control of block 102, speech output block 104 generates the appropriate speech item for outputting via interface 96. The loop as shown is realized once for each respective dialog. Mapping on shared or separate hardware is left unconsidered. Item 106 represents an operator microphone. Block 108 is a speech recognizer sub-process; block 110 a speech understanding sub-process. The latter outputs to the dialog control sub-process to emulate a user item for enhancing the progress of the dialog in question. The operator control sub-system through interrogating the progress score storage block 112 in a way not shown may select one or more of currently active dialogs as candidate for operator intervention. Again, the blocks 108, 110 may be mapped on hardware that is shared with further (sub-)processes.

Claims (14)

We claim:
1. A method of carrying out a human-machine dialog whilst executing successive sections of a computer program of a user interactive system in accordance with successive verbal user items presented by a user as reacting to successive verbal system items presented by the system to the user, each such verbal system item corresponding to a respective program section, said method comprising successive steps of:
a. presentation by the system to the user of a verbal system item corresponding to a program section;
b. recognition by the system of a verbal user item presented by the user to the system;
characterized by:
c. evaluation of a result of said recognition, and reverting to part a. whilst on the basis of said recognition, selecting among a plurality of such program sections each associated to a subsequent verbal system item, and on the basis of said evaluation keeping a progress score pertaining to said dialog, and under control of a particular progress score level enabling a system operator to intervene in said dialog in an unobtrusive manner.
2. A method as claimed in claim 1, wherein said enabling allows the system operator to emulate a user person by means of an emulated verbal user item presented to the system.
3. A method as claimed in claim 2, wherein in said system past verbal user items as recognized are retained for reviewing by the system operator.
4. A method as claimed in claim 3, wherein said system past verbal user items as received are retained for second reviewing by the system operator.
5. A method as claimed in claim 2, wherein in said system past verbal user items as received are retained for second reviewing by the system operator.
6. A method as claimed in claim 5, wherein said second reviewing is effected at raised speed as compared to standard speed of human speech.
7. A method as claimed in claim 1, wherein said enabling allows the system operator to emulate a user person by means of a mechanically entered user item presented to the system.
8. A method as claimed in claim 7, wherein in said system past verbal user items as recognized are retained for reviewing by the system operator.
9. A method as claimed in claim 7, wherein in said system past verbal user items as received are retained for second reviewing by the system operator.
10. A method of carrying out a plurality of human-machine dialogs in parallel whilst for each dialog carrying out successive sections of an associated shared computer program of a user interactive system, in accordance with successive verbal user items presented by its respective user as reacting to successive verbal system items presented by the system to the user in question, each such verbal system item corresponding to a respective program section, said method for each dialog comprising successive steps of:
a. presentation by the system to the user of a verbal system item corresponding to a program section;
b. recognition by the system of a verbal user item presented by the user to the system;
c. evaluation of a result of said recognition and reverting to part a. whilst on the basis of said recognition selecting among a plurality of such program sections each associated to a subsequent verbal system item, and on the basis of said evaluation keeping a progress score pertaining to each respective said dialog;
said method comprising comparing and ranking separate progress score levels each pertaining to a respective said dialog, and under control of a particular progress score ranking among said dialogs enabling a system operator to intervene in the associated dialog in an unobtrusive manner.
11. A method as claimed in claim 10, wherein said enabling allows the system operator to emulate a user person by means of an emulated verbal user item presented to the system.
12. A method as claimed in claim 10, wherein said enabling allows the system operator to emulate a user person by means of a mechanically entered user item presented to the system.
13. A computer apparatus for carrying out a human-machine dialog whilst carrying out successive sections of a computer program of a user interactive system in accordance with successive verbal user items presented by a user as reacting to successive verbal system items presented by the system to the user, each such system item corresponding to a respective program section, said apparatus comprising:
a. presentation means for presenting to the user of a verbal system item corresponding to a program section;
b. recognition means for recognizing a verbal user item presented by the user;
characterized by:
c. evaluation means fed by said recognition means for evaluating a result of said recognition, and having activating means for activating said presentation means, reverting to part a. whilst on the basis of said recognition, selecting among a plurality of program sections each associated to a subsequent verbal system item, and having scoring means fed by said evaluation means for on the basis of said evaluation keeping a progress score pertaining to said dialog, and having an enabling output for under control of a particular progress score level outputting an enabling signal for enabling a system operator to intervene in said dialog in an unobtrusive manner.
14. An apparatus as claimed in claim 13, wherein said evaluation means, recognition means and evaluation means are executed in a multiple way so as to allow simultaneous supporting of a plurality of human-machine dialogs in parallel, and said evaluation means furthermore comprise ranking means for ranking multiple progress scores each associated to a respective said dialog, and said enabling output is fed by said ranking means.
US08587190 1995-01-18 1996-01-16 Method and apparatus for providing a human-machine dialog supportable by operator intervention Expired - Lifetime US5745877A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP95200120 1995-01-18
EP95200120 1995-01-18

Publications (1)

Publication Number Publication Date
US5745877A true US5745877A (en) 1998-04-28

Family

ID=8219961

Family Applications (1)

Application Number Title Priority Date Filing Date
US08587190 Expired - Lifetime US5745877A (en) 1995-01-18 1996-01-16 Method and apparatus for providing a human-machine dialog supportable by operator intervention

Country Status (6)

Country Link
US (1) US5745877A (en)
JP (1) JPH09510803A (en)
CN (1) CN1097769C (en)
DE (2) DE69617544T2 (en)
EP (1) EP0752129B1 (en)
WO (1) WO1996022568A1 (en)

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1215655A1 (en) * 2000-12-09 2002-06-19 Hewlett-Packard Company Voice service system and method
US20030097249A1 (en) * 2001-03-14 2003-05-22 Walker Marilyn A. Trainable sentence planning system
EP1317749A1 (en) * 2000-07-31 2003-06-11 Eliza Corporation Method of and system for improving accuracy in a speech recognition system
US20030110037A1 (en) * 2001-03-14 2003-06-12 Walker Marilyn A Automated sentence planning in a task classification system
US20030115062A1 (en) * 2002-10-29 2003-06-19 Walker Marilyn A. Method for automated sentence planning
US6647363B2 (en) 1998-10-09 2003-11-11 Scansoft, Inc. Method and system for automatically verbally responding to user inquiries about information
EP1378886A1 (en) * 2002-07-02 2004-01-07 Ubicall Communications en abrégé "UbiCall" S.A. Speech recognition device
US20040098245A1 (en) * 2001-03-14 2004-05-20 Walker Marilyn A Method for automated sentence planning in a task classification system
US20050109052A1 (en) * 2003-09-30 2005-05-26 Albers Walter F. Systems and methods for conditioning air and transferring heat and mass between airflows
US20060229872A1 (en) * 2005-03-29 2006-10-12 International Business Machines Corporation Methods and apparatus for conveying synthetic speech style from a text-to-speech system
US7143039B1 (en) 2000-08-11 2006-11-28 Tellme Networks, Inc. Providing menu and other services for an information processing system using a telephone or other audio interface
US20070198272A1 (en) * 2006-02-20 2007-08-23 Masaru Horioka Voice response system
US7308408B1 (en) 2000-07-24 2007-12-11 Microsoft Corporation Providing services for an information processing system using an audio interface
US20080154601A1 (en) * 2004-09-29 2008-06-26 Microsoft Corporation Method and system for providing menu and other services for an information processing system using a telephone or other audio interface
US20090119586A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Automatic Generation of Interactive Systems From a Formalized Description Language
US20090119104A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Switching Functionality To Control Real-Time Switching Of Modules Of A Dialog System
US7571226B1 (en) 1999-10-22 2009-08-04 Tellme Networks, Inc. Content personalization over an interface with adaptive voice character
US20090210232A1 (en) * 2008-02-15 2009-08-20 Microsoft Corporation Layered prompting: self-calibrating instructional prompting for verbal interfaces
US7941481B1 (en) 1999-10-22 2011-05-10 Tellme Networks, Inc. Updating an electronic phonebook over electronic communication networks

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997033221A3 (en) * 1996-03-05 1997-11-13 Philips Electronics Nv Transaction system based on a bidirectional speech channel through status graph building and problem detection for thereupon providing feedback to a human user person
DE19942871B4 (en) * 1999-09-08 2013-11-21 Volkswagen Ag Method for operating a voice-controlled command input unit in a motor vehicle
US7143040B2 (en) 2000-07-20 2006-11-28 British Telecommunications Public Limited Company Interactive dialogues
DE60126948D1 (en) * 2000-07-20 2007-04-12 British Telecomm interactive dialogues
GB2372864B (en) * 2001-02-28 2005-09-07 Vox Generation Ltd Spoken language interface
US20100286490A1 (en) * 2006-04-20 2010-11-11 Iq Life, Inc. Interactive patient monitoring system using speech recognition

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4959864A (en) * 1985-02-07 1990-09-25 U.S. Philips Corporation Method and system for providing adaptive interactive command response

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2165969B (en) * 1984-10-19 1988-07-06 British Telecomm Dialogue system
CA2115210C (en) * 1993-04-21 1997-09-23 Joseph C. Andreshak Interactive computer system recognizing spoken commands

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4959864A (en) * 1985-02-07 1990-09-25 U.S. Philips Corporation Method and system for providing adaptive interactive command response

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6647363B2 (en) 1998-10-09 2003-11-11 Scansoft, Inc. Method and system for automatically verbally responding to user inquiries about information
US7571226B1 (en) 1999-10-22 2009-08-04 Tellme Networks, Inc. Content personalization over an interface with adaptive voice character
US7941481B1 (en) 1999-10-22 2011-05-10 Tellme Networks, Inc. Updating an electronic phonebook over electronic communication networks
US7308408B1 (en) 2000-07-24 2007-12-11 Microsoft Corporation Providing services for an information processing system using an audio interface
US8812314B2 (en) 2000-07-31 2014-08-19 Eliza Corporation Method of and system for improving accuracy in a speech recognition system
EP1317749A1 (en) * 2000-07-31 2003-06-11 Eliza Corporation Method of and system for improving accuracy in a speech recognition system
US7624010B1 (en) 2000-07-31 2009-11-24 Eliza Corporation Method of and system for improving accuracy in a speech recognition system
EP1317749A4 (en) * 2000-07-31 2005-12-21 Eliza Corp Method of and system for improving accuracy in a speech recognition system
US20100100378A1 (en) * 2000-07-31 2010-04-22 Eliza Corporation Method of and system for improving accuracy in a speech recognition system
US7552054B1 (en) 2000-08-11 2009-06-23 Tellme Networks, Inc. Providing menu and other services for an information processing system using a telephone or other audio interface
US7143039B1 (en) 2000-08-11 2006-11-28 Tellme Networks, Inc. Providing menu and other services for an information processing system using a telephone or other audio interface
EP1215655A1 (en) * 2000-12-09 2002-06-19 Hewlett-Packard Company Voice service system and method
US20020087323A1 (en) * 2000-12-09 2002-07-04 Andrew Thomas Voice service system and method
US8185401B2 (en) 2001-03-14 2012-05-22 At&T Intellectual Property Ii, L.P. Automated sentence planning in a task classification system
US20040098245A1 (en) * 2001-03-14 2004-05-20 Walker Marilyn A Method for automated sentence planning in a task classification system
US8620669B2 (en) 2001-03-14 2013-12-31 At&T Intellectual Property Ii, L.P. Automated sentence planning in a task classification system
US8019610B2 (en) 2001-03-14 2011-09-13 At&T Intellectual Property Ii, L.P. Automated sentence planning in a task classification system
US8209186B2 (en) 2001-03-14 2012-06-26 At&T Intellectual Property Ii, L.P. Method for automated sentence planning in a task classification system
US7516076B2 (en) * 2001-03-14 2009-04-07 At&T Intellectual Property Ii, L.P. Automated sentence planning in a task classification system
US7949537B2 (en) 2001-03-14 2011-05-24 At&T Intellectual Property Ii, L.P. Method for automated sentence planning in a task classification system
US20030110037A1 (en) * 2001-03-14 2003-06-12 Walker Marilyn A Automated sentence planning in a task classification system
US20100241420A1 (en) * 2001-03-14 2010-09-23 AT&T Intellectual Property II, L.P., via transfer from AT&T Corp. Automated sentence planning in a task classification system
US20030097249A1 (en) * 2001-03-14 2003-05-22 Walker Marilyn A. Trainable sentence planning system
US8180647B2 (en) 2001-03-14 2012-05-15 At&T Intellectual Property Ii, L.P. Automated sentence planning in a task classification system
US7574362B2 (en) * 2001-03-14 2009-08-11 At&T Intellectual Property Ii, L.P. Method for automated sentence planning in a task classification system
US7729918B2 (en) * 2001-03-14 2010-06-01 At&T Intellectual Property Ii, Lp Trainable sentence planning system
US20090222267A1 (en) * 2001-03-14 2009-09-03 At&T Corp. Automated sentence planning in a task classification system
US20110218807A1 (en) * 2001-03-14 2011-09-08 AT&T Intellectual Property ll, LP Method for Automated Sentence Planning in a Task Classification System
EP1378886A1 (en) * 2002-07-02 2004-01-07 Ubicall Communications en abrégé "UbiCall" S.A. Speech recognition device
US20050080627A1 (en) * 2002-07-02 2005-04-14 Ubicall Communications En Abrege "Ubicall" S.A. Speech recognition device
US20030115062A1 (en) * 2002-10-29 2003-06-19 Walker Marilyn A. Method for automated sentence planning
US20050109052A1 (en) * 2003-09-30 2005-05-26 Albers Walter F. Systems and methods for conditioning air and transferring heat and mass between airflows
US20080154601A1 (en) * 2004-09-29 2008-06-26 Microsoft Corporation Method and system for providing menu and other services for an information processing system using a telephone or other audio interface
US20060229872A1 (en) * 2005-03-29 2006-10-12 International Business Machines Corporation Methods and apparatus for conveying synthetic speech style from a text-to-speech system
US7415413B2 (en) * 2005-03-29 2008-08-19 International Business Machines Corporation Methods for conveying synthetic speech style from a text-to-speech system
US20090141871A1 (en) * 2006-02-20 2009-06-04 International Business Machines Corporation Voice response system
US8145494B2 (en) * 2006-02-20 2012-03-27 Nuance Communications, Inc. Voice response system
US8095371B2 (en) * 2006-02-20 2012-01-10 Nuance Communications, Inc. Computer-implemented voice response method using a dialog state diagram to facilitate operator intervention
US20070198272A1 (en) * 2006-02-20 2007-08-23 Masaru Horioka Voice response system
US20090119104A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Switching Functionality To Control Real-Time Switching Of Modules Of A Dialog System
US20090119586A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Automatic Generation of Interactive Systems From a Formalized Description Language
US8001469B2 (en) 2007-11-07 2011-08-16 Robert Bosch Gmbh Automatic generation of interactive systems from a formalized description language
US8155959B2 (en) * 2007-11-07 2012-04-10 Robert Bosch Gmbh Dialog system for human agent to correct abnormal output
US8165884B2 (en) 2008-02-15 2012-04-24 Microsoft Corporation Layered prompting: self-calibrating instructional prompting for verbal interfaces
US20090210232A1 (en) * 2008-02-15 2009-08-20 Microsoft Corporation Layered prompting: self-calibrating instructional prompting for verbal interfaces

Also Published As

Publication number Publication date Type
DE69617544T2 (en) 2002-08-08 grant
DE69617544D1 (en) 2002-01-17 grant
JPH09510803A (en) 1997-10-28 application
CN1097769C (en) 2003-01-01 grant
EP0752129A1 (en) 1997-01-08 application
WO1996022568A1 (en) 1996-07-25 application
EP0752129B1 (en) 2001-12-05 grant
CN1147871A (en) 1997-04-16 application

Similar Documents

Publication Publication Date Title
Arons Hyperspeech: Navigating in speech-only hypermedia
Litman et al. ITSPOKE: An intelligent tutoring spoken dialogue system
US7236932B1 (en) Method of and apparatus for improving productivity of human reviewers of automatically transcribed documents generated by media conversion systems
US7490039B1 (en) Text to speech system and method having interactive spelling capabilities
US7149690B2 (en) Method and apparatus for interactive language instruction
Richards Listening comprehension: Approach, design, procedure
Foster et al. The influence of planning and task type on second language performance
US6192332B1 (en) Adaptive electronic phrase book
US5393236A (en) Interactive speech pronunciation apparatus and method
US5597312A (en) Intelligent tutoring method and system
US20020173955A1 (en) Method of speech recognition by presenting N-best word candidates
US6073102A (en) Speech recognition method
US20060111902A1 (en) System and method for assisting language learning
US6157913A (en) Method and apparatus for estimating fitness to perform tasks based on linguistic and other aspects of spoken responses in constrained interactions
Lauria et al. Training personal robots using natural language instruction
Mostow et al. Giving help and praise in a reading tutor with imperfect listening—Because automated speech recognition means never being able to say you're certain
US20040120476A1 (en) Voice response system
US6139330A (en) Computer-aided learning system and method
US4749353A (en) Talking electronic learning aid for improvement of spelling with operator-controlled word list
Plauche et al. Speech recognition for illiterate access to information and technology
US6321198B1 (en) Apparatus for design and simulation of dialogue
Delgado et al. Spoken, multilingual and multimodal dialogue systems: development and assessment
US4507750A (en) Electronic apparatus from a host language
Kamm User interfaces for voice applications
Gustafson et al. AdApt-a multimodal conversational dialogue system in an apartment domain

Legal Events

Date Code Title Description
AS Assignment

Owner name: U.S. PHILIPS CORPORATION, NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NIJMAN, ALOIJSIUS J.;AUST, HARALD;BESTING, PETER;AND OTHERS;REEL/FRAME:007868/0735;SIGNING DATES FROM 19960216 TO 19960223

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: SCANSOFT, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:U.S. PHILIPS CORPORATION;REEL/FRAME:013828/0931

Effective date: 20030210

AS Assignment

Owner name: SCANSOFT, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:U.S. PHILIPS CORPORATION;REEL/FRAME:013998/0986

Effective date: 20030210

REMI Maintenance fee reminder mailed
AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: MERGER AND CHANGE OF NAME TO NUANCE COMMUNICATIONS, INC.;ASSIGNOR:SCANSOFT, INC.;REEL/FRAME:016914/0975

Effective date: 20051017

FPAY Fee payment

Year of fee payment: 8

SULP Surcharge for late payment

Year of fee payment: 7

AS Assignment

Owner name: USB AG, STAMFORD BRANCH, CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:017435/0199

Effective date: 20060331

Owner name: USB AG, STAMFORD BRANCH,CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:017435/0199

Effective date: 20060331

AS Assignment

Owner name: USB AG. STAMFORD BRANCH, CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:018160/0909

Effective date: 20060331

Owner name: USB AG. STAMFORD BRANCH,CONNECTICUT

Free format text: SECURITY AGREEMENT;ASSIGNOR:NUANCE COMMUNICATIONS, INC.;REEL/FRAME:018160/0909

Effective date: 20060331

FPAY Fee payment

Year of fee payment: 12

AS Assignment

Owner name: SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: MITSUBISH DENKI KABUSHIKI KAISHA, AS GRANTOR, JAPA

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTO

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DEL

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: NUANCE COMMUNICATIONS, INC., AS GRANTOR, MASSACHUS

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: NORTHROP GRUMMAN CORPORATION, A DELAWARE CORPORATI

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: TELELOGUE, INC., A DELAWARE CORPORATION, AS GRANTO

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: SCANSOFT, INC., A DELAWARE CORPORATION, AS GRANTOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: NOKIA CORPORATION, AS GRANTOR, FINLAND

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: HUMAN CAPITAL RESOURCES, INC., A DELAWARE CORPORAT

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPOR

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: INSTITIT KATALIZA IMENI G.K. BORESKOVA SIBIRSKOGO

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: STRYKER LEIBINGER GMBH & CO., KG, AS GRANTOR, GERM

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: DSP, INC., D/B/A DIAMOND EQUIPMENT, A MAINE CORPOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: DICTAPHONE CORPORATION, A DELAWARE CORPORATION, AS

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: SPEECHWORKS INTERNATIONAL, INC., A DELAWARE CORPOR

Free format text: PATENT RELEASE (REEL:018160/FRAME:0909);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0869

Effective date: 20160520

Owner name: ART ADVANCED RECOGNITION TECHNOLOGIES, INC., A DEL

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520

Owner name: NUANCE COMMUNICATIONS, INC., AS GRANTOR, MASSACHUS

Free format text: PATENT RELEASE (REEL:017435/FRAME:0199);ASSIGNOR:MORGAN STANLEY SENIOR FUNDING, INC., AS ADMINISTRATIVE AGENT;REEL/FRAME:038770/0824

Effective date: 20160520