US20030191625A1 - Method and system for creating a named entity language model - Google Patents

Method and system for creating a named entity language model Download PDF

Info

Publication number
US20030191625A1
US20030191625A1 US10402976 US40297603A US2003191625A1 US 20030191625 A1 US20030191625 A1 US 20030191625A1 US 10402976 US10402976 US 10402976 US 40297603 A US40297603 A US 40297603A US 2003191625 A1 US2003191625 A1 US 2003191625A1
Authority
US
Grant status
Application
Patent type
Prior art keywords
named
entity
corpus
training
word
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10402976
Inventor
Allen Gorin
Frederic Bechet
Jeremy Wright
Dilek Hakkani-Tur
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nuance Communications Inc
Original Assignee
AT&T Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20Handling natural language data
    • G06F17/27Automatic analysis, e.g. parsing
    • G06F17/2765Recognition
    • G06F17/2775Phrasal analysis, e.g. finite state techniques, chunking
    • G06F17/278Named entity recognition
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L15/18Speech classification or search using natural language modelling
    • G10L15/183Speech classification or search using natural language modelling using context dependencies, e.g. language models
    • G10L15/19Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
    • G10L15/193Formal grammars, e.g. finite state automata, context free grammars or word networks
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/08Speech classification or search
    • G10L2015/088Word spotting

Abstract

The invention concerns a method and system for creating a named entity language model. The method may include recognizing input communications from a training corpus, parsing the training corpus, tagging the parsed training corpus, aligning the recognized training corpus with the tagged training corpus, and creating a named entity language model from the aligned corpus.

Description

  • [0001]
    This non-provisional application claims the benefit of U.S. Provisional Patent Application No. 60/307,624, filed Apr. 5, 2002, and U.S. Provisional Patent Application No. 60/443,642, filed Jan. 29, 2003, which are both incorporated herein by reference in their entireties. This application is also a continuation-in-part of 1) U.S. Patent Application No. 10/158,082 which claims priority from U.S. Provisional Patent Application No. 60/322,447, filed Sep. 17, 2001, 2) U.S. Patent Application No. 09/690,721 and 3) No. 09/690,903 both filed Oct. 18, 2000, which claim priority from U.S. Provisional Application No. 60/163,838, filed Nov. 5, 1999. U.S. patent application Ser. Nos. 09/690,721, 09/690,903, 10/158,082 and U.S. Provisional Application Nos. 60/163,838 and 60/322,447 are incorporated herein by reference in their entireties.
  • TECHNICAL FIELD
  • [0002]
    The invention relates to automated systems for communication recognition and understanding.
  • BACKGROUND OF THE INVENTION
  • [0003]
    Examples of conventional automated dialogue systems can be found in U.S. Pat. Nos. 5,675,707, 5,860,063, 6,021,384, 6,044,337, 6,173,261 and 6,192,110, which are incorporated herein by reference in their entireties. Interactive spoken dialogue systems are now employed in a wide range of applications, such as directory assistance or customer care on a very large scale. Dealing with a large population of non-expert users results in a great variability in the spontaneous communications being processed. This variability requires a very high robustness from every part of dialogue system.
  • [0004]
    In addition, the large amount of real data collected through these dialogue systems raises many new dialogue system training issues and makes possible the use of even more automatic learning and corpus-based methods at each step of the dialogue process. One of the issues that these developments have raised is the role of the dialogue system in determining, firstly what kind of query the database is going to be asked and secondly with which parameters.
  • SUMMARY OF THE INVENTION
  • [0005]
    The invention concerns a method and system for creating a named entity language model. The method may include recognizing input communications from a training corpus, parsing the training corpus, tagging the parsed training corpus, aligning the recognized training corpus with the tagged training corpus, and creating a named entity language model from the aligned corpus.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • [0006]
    The invention is described in detail with reference to the following drawings wherein like numerals reference like elements, and wherein:
  • [0007]
    [0007]FIG. 1 is a block diagram of an exemplary communication recognition and understanding system;
  • [0008]
    [0008]FIG. 2 is a detailed block diagram of an exemplary named entity training unit;
  • [0009]
    [0009]FIG. 3 is a detailed flowchart illustrating an exemplary named entity training process;
  • [0010]
    [0010]FIG. 4 is a detailed block diagram of an exemplary named entity detection and extraction unit;
  • [0011]
    [0011]FIG. 5 is a detailed block diagram of an exemplary named entity detector;
  • [0012]
    [0012]FIG. 6 is a flowchart illustrating an exemplary named entity detection and extraction process; and
  • [0013]
    [0013]FIG. 7 is a flowchart of an exemplary task classification process.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • [0014]
    A dialogue system can be viewed as an interface between a user and a database. The role of the system is to determine first, what kind of query the database is going to be asked, and second, with which parameters. For example, in the How May I Help You?SM, TM (HMIHY) customer care corpus, if a user wants his or her account balance, the query concerns accessing the account balance field of the database with the customer identification number as the parameter.
  • [0015]
    Such database queries are denoted by task-type (or in this example, call-type) and their parameters are the information items that are contained in the user's request. They are often called “named entities”. The most general definition of a named entity is a sequence of words, symbols, etc. that refers to a unique identifier. For example, a named entity may refer to:
  • [0016]
    proper name identifiers, like organization, person or location names;
  • [0017]
    time identifiers, like dates, time expressions or durations; or
  • [0018]
    quantities and numerical expressions, like monetary values, percentage or phone numbers.
  • [0019]
    In the framework of a dialogue system, the definition of a named entity is often associated to its meaning for the application targeted. For example, in a customer care corpus, most of the relevant time or monetary expressions may be those related to an item on a customer's bill (the date of the bill, a date or an amount of an item or service, etc.).
  • [0020]
    In this respect, there are context-dependent named entities that are named entities whose definition is linked to the dialogue context, and context-independent named entities that are independent from the dialogue application (e.g., a date). One of the aspects of the invention described herein is the detection and extraction of such context-dependent and independent named entities from spontaneous communications in a mixed-initiative dialogue context.
  • [0021]
    Within the dialogue system, the module that is responsible for interacting with the user is called a dialogue manager. Dialogue managers can be classified according to the type of the type of system interaction implemented, including system-initiative, user-initiative or mixed-initiative.
  • [0022]
    A system-initiative dialogue manager handles very constrained dialogues where the user has to answer to direct questions by either one key word or a simple short sentence. The task to fulfill can be rather sophisticated but the user has to be cooperative and patient as the language accepted is not spontaneous and the list of questions asked can be quite long.
  • [0023]
    A user-initiative dialogue manager gives the user the possibility of directing the dialogue. The system waits to know what the user wants before asking a specific question. In this case, spontaneous communications have to be accepted. However, because of the difficulty in processing such spontaneous input, the range of tasks which can be addressed within a single application is limited. Thus, in a customer care system, the first stage of attempting to “understand” a user's query involves classifying the user's intent according to a list of task-types specific to the application.
  • [0024]
    A customer care corpus, for example, may contain dialogue belonging to a third category which is known as mixed-initiative dialogue manager systems. In this case, the top-level of the dialogue implements a user-initiative dialogue by simply asking the user “How may I help you?”, for example. A short dialogue sometimes ensues for clarifying the request, and finally the user is sent to either an automated dialogue system or a human representative depending on the availability of such an automatic process for the request recognized.
  • [0025]
    According to the above-described dialogue manager interaction type implemented, the performance of the named entity extraction task strongly varies. For example, extracting a phone number is much easier in a system-initiative context where the user has to answer to a prompt like “Please give me your home phone number starting with line area code”, than in a user-initiative dialogue where a phone number is not expected and is likely to be embedded into a long spontaneous utterance such as:
  • [0026]
    Okay, my name is XXXX from Sarasota Fla. telephone number area code XXX XXXXXXX and there's a two nine four beside it but I don't what that means anyways on December the thirty first at one thirty five P M I w-there was a call from our number called to area code XXX XXXXXXX.
  • [0027]
    Accordingly, one of the aspects of this invention addresses how to automatically process such spontaneous responses. In particular, this invention concerns a dialogue system that automatically detects and extracts, from a recognized output, the task-type request expressed by a user and its parameters, such as numerical expressions, time expressions or proper names. As noted above, these parameters are called “named entities” and their definitions can be either independent from the context of the application or strongly linked to the application domain. Thus, a method and system that trains named entity language models, and a method and system that detects and extracts such named entities to improve understanding during an automated dialogue session with a user, will be discussed in detail below.
  • [0028]
    [0028]FIG. 1 is an exemplary block diagram of a possible communication recognition and understanding system 100 that utilizes named entity detection and extraction. The exemplary communication recognition and understanding system 100 includes two related subsystems, namely a named entity training subsystem 110 and input communication processing subsystem 120.
  • [0029]
    The named entity training subsystem 110 includes a named entity training unit 130 and a named entity database 140. The named entity training unit 130 generates named entity language models and a text classifier training corpus from a training corpus of transcribed or untranscribed training communications. The generated named entity language models and text classifier training corpus are stored in the named entity database 140 for use by the named entity detection and extraction unit 160.
  • [0030]
    The input communication processing subsystem 120 includes an input communication recognizer 150, a named entity detection and extraction unit 160, a natural language understanding unit 170 and a dialogue manager 180. The input communication recognizer 150 receives a user's task objective request or other communications in the form of verbal and/or non-verbal communications. Non-verbal communications may include tablet strokes, gestures, head movements, hand movements, body movements, etc.). The input communication recognizer 150 may perform the function of recognizing or spotting the existence of one or more words, phones, sub-units, acoustic morphemes, non-acoustic morphemes, morpheme lattices, etc., in the user's input communications using any algorithm known to one of ordinary skill in the art.
  • [0031]
    One such algorithm may involve the input communication recognizer 150 forming a lattice structure to represent a distribution of recognized phone sequences, such as a probability distribution. For example, the input communication recognizer 150 may extract the n-best word strings that may be extracted from the lattice, either by themselves or along with their confidence scores. Such lattice representations are well known those skilled in the art and are further described in detail below. While the invention is described below as being used in a system that forms and uses lattice structures, this is only one possible embodiment and the invention should not be limited as such.
  • [0032]
    The named entity detection and extraction unit 160 detects the named entities present in the lattice that represents the user's input request or other communication. The named entity detection and extraction unit 160 then tags and classifies detected named entities and extracts the named entity values using a process such as discussed in relation to FIGS. 4-6 below. These extracted values are then provided as an input to the natural language understanding unit 170.
  • [0033]
    The natural language understanding unit 170 may apply a confidence function, based on the probabilistic relationship between the recognized communication including the named entity values and selected task objectives. As a result, the natural language understanding unit 170 will pass the information to the dialogue manager 180. The dialogue manager 180 may then make a decision either to implement a particular task objective, or determine that no decision can made based on the information provided, in which case the user may be defaulted to a human or other automated system for assistance. In any case, the dialogue manager 180 will inform the user of the status and/or solicit more information.
  • [0034]
    The dialogue manager 180 may also store the various dialogue iterations during a particular user dialogue session. These previous dialogue iterations may be used by the dialogue manager 180 in conjunction with the current user input to provide an acceptable response, task completion, or task routing objective, for example.
  • [0035]
    [0035]FIG. 2 is a more detailed diagram of the named entity training subsystem 110 shown in FIG. 1. The named entity training unit 130 may include a transcriber 210, a labeler 220, a named entity parser 230, a named entity training tagger 240, a training recognizer 250, and an aligner 260. The named entity language model and text classifier training corpus generated by the named entity training unit 130 are stored in the named entity database 140 for use by the named entity detection and extraction unit 160. The operation of the individual units of the named entity training subsystem 110 will be discussed in detail with respect to FIG. 3 below.
  • [0036]
    [0036]FIG. 3 illustrates an exemplary named entity training process for using the named entity training subsystem 110 shown in FIGS. 1 and 2. Note that while the steps of any of the exemplary flowcharts illustrated herein are shown having steps arranged in a particular order, the order of the steps in the figures may be rearranged to be performed in any order, or simultaneously, for example.
  • [0037]
    The process in FIG. 3 begins at step 3100 and proceeds to step 3200 where the training corpus is input into a task-independent training recognizer 250. The recognition process may involve a phonotactic language model that was trained on the switchboard corpus using a Variable-Length N-gram Stochastic Automaton, for example. This training corpus may be derived from a collection of sentences generated from the recordings of callers responding to a system prompt, for example. In experiments conducted on the system of the invention, 7642 and 1000 sentences in the training and test sets were used, respectively. Sentences are represented at the word level and provided with semantic labels drawn from 46 call-types. This training corpus may be unrelated to the system task. Moreover, off-the-shelf telephony acoustic models may be used.
  • [0038]
    In step 3300, the transcriber 210 also receives raw training communications from the training corpus in conjunction with the training corpus being put through the training recognizer 250. The processes in steps 3200 and 3300 may be performed simultaneously. In the traditional approach, corpora of spontaneous communications dedicated to a specific application are small and obtained by a particular protocol or a directed experiment trial. Because of their small size, these training corpora may be integrally transcribed by humans. In large-scale dialogue systems, the amount of live customer traffic is very large and the cost of transcribing all of the data available would be enormous.
  • [0039]
    In this case, a selective sampling process may be used in order to select the dialogues that are going to be labeled. This process can be done randomly, but by being able to extract information from the recognizer output in an unsupervised way, the selection method can be made more efficient. For example, some rare task types or named entities can be very badly modeled because of the lack of data representing them. By automatically detecting named entity tags, specifically selected dialogues can be represented that are likely to contain them, and accelerate in a very significant way, the coverage of the training corpus.
  • [0040]
    The dual process shown in FIG. 2 is important to improve the quality of named entity detection and extraction. For instance, consider that named entities are usually represented by either handwritten rules or statistical models. Even if statistical models seem to be much more robust toward recognizer errors, in both cases the models are derived from communications transcribed by the transcriber 210 and the errors from the training recognizer 250 are not taken into account explicitly by the models.
  • [0041]
    This strategy certainly emphasizes the precision of the resulting detection, but a great loss in recall can occur by not modeling the training recognizer's 250 behavior. For example, a word can be considered as very salient information for detecting a particular named entity tag. But if this word is, for any reason, very often badly recognized by the training recognizer 250, its salience won't be useful on the output of the system. For this reason, the training recognizer's 250 behavior in the contextual named entity tagging process is explicitly modeled.
  • [0042]
    In step 3400, the transcribed corpus is then labeled by the labeler 220 and parsed by the named entity parser 230. In this manner, the labeler 220 classifies each sentence according to the list of named entity tags contained in it. Then for each sentence, the named entity parser 230 marks all of the words or group of words selected by the labeler 220 for characterizing the sentence according to the named entity tags. In this manner, the named entity parser 230 marks the part-of-speech of each word and performs a syntactic bracketing on the corpus.
  • [0043]
    In step 3500, as a way to make the user's input communication useful to the dialogue manager 180, the named entity training tagger 240 inserts named entity tags on the labeled and parsed training corpus. This process may include using the list of named entity tags included in each of sentence as well as using both statistic and syntactic criteria to present which context is considered salient for identifying named entities.
  • [0044]
    For example, consider the following four exemplary named entity tags:
  • [0045]
    (1) Date: any date expression with at least a day and a month specified;
  • [0046]
    (2) Phone: phone numbers expressed in a 10 or 11-digit string;
  • [0047]
    (3) Item_Amount: money expression referring to a charge written on the customer's bill;
  • [0048]
    (4) Which_Bill: a temporal expression identifying the bill the customer is talking about.
  • [0049]
    The first two tags can be considered as context-independent named entities. For example, “Date” can correspond to any date expression. In addition, nearly all of the 10 or 11-digit strings in the customer care corpus are effectively considered “Phone” numbers.
  • [0050]
    In contrast, the last two tags are context-dependent named entities. “Which_Bill” is not any temporal expression but an expression that allows identification of a customer's bill. “Item_Amount” refers to a numerical money expression that is explicitly written on the bill. According to this definition, the following sentence contains an Item_Amount tag: I don't recognize this 22 dollars call to . . . but this one doesn't: he told me we would get a 50 dollars gift . . . .
  • [0051]
    Thus, each named entity tag can correspond to one or several kinds of values. To the tags Phone, Item_Amount and Date correspond only one type of pattern for their values (respectively: 10-digit string, <num>$<num><cents>and<year>/<month>/<day>). But Which Bill can be either a date (dates of the period of the bill, date of arrival of the bill, date of payment, etc.) or a relative temporal expression like current or previous.
  • [0052]
    For the named entity tagging process, the named entity tagger 240 may use a probabilistic tagging approach. This approach involves is a Hidden Markov Model (HMM) where each word of a sentence is emitted by a state in the model. The hidden state sequence S=(s1. SN) corresponds to the following situations: beginning a named entity, being inside a named entity, ending a named entity, being in the background text.
  • [0053]
    Finding the most probable sequence of state which have produced the known word sequence W=(w1 . . . wN) is equivalent to maximizing the probability: P(S/W). By using Bayes' rule and the assumption that the state at time “f” is only dependent on the state and observation at time t−1, according to equation (1) below: P ( S | W ) arg max S t = 1 N P ( w t | w t - 1 , s t ) P ( s t | s t - 1 , w t - 1 )
    Figure US20030191625A1-20031009-M00001
  • [0054]
    The first term, P(wt/wt−1,st), is implemented as a state-dependent bigram model. For example, if st is the state inside a PHONE, this first term corresponds to the bigram probability Pphone (wt/wt−1) estimated on the corpus CNE. Similarly, the bigram probability for the background text, Pbk (wt/wt−1), is estimated on the corpus CBK.
  • [0055]
    The second term is the state transition probability of going from the state t−1 to the state t. These probabilities are estimated on the training corpus, once the named entity context selection process has been done. The context corresponds to the smallest portion of text, containing the named entity, which allows the labeler 220 to decide that a named entity is occurring.
  • [0056]
    As it has been previously shown, not all the named entities are considered relevant for the natural language understanding unit 170. Therefore, the named entity training tagger 240 must model not only the named entity itself (e.g., 20 dollars and 40 cents) but its whole context of occurrence (e.g., . . . this 20 dollars and 40 cents call . . . ) in order to disambiguate relevant named entities from others. Thus, the relevant context of a named entity tag in a sentence is the concatenation of all the syntactic phrases containing a word marked by the named entity parser 230.
  • [0057]
    After processing the whole training corpus, two corpora are attached to each tag:
  • [0058]
    (1) the corpus CNE containing only named entity contexts (e.g., <PH>my phone area code d3 number d7</PH>); and
  • [0059]
    (2) the corpus CBK which contains the background text without the named entity contexts (e.g., I'm calling about <PH></PH>).
  • [0060]
    In both cases, non-terminal symbols are used for representing numerical values and proper names.
  • [0061]
    As discussed above, in conjunction with the training of the named entity language model, this process takes into account the recognition errors explicitly in that model. In this manner, the whole training corpus is also processed by the training recognizer 250 as discussed in relation to step 3200 above, in order to learn automatically the confusions and the mistakes which are likely to occur in a deployed communication recognition and understanding system 100.
  • [0062]
    Therefore, in the transcribed part of the customer care corpus, each named entity may be represented by three items: tag, context and value.
  • [0063]
    As part of the final process, in step 3600 the aligner 260 aligns the training recognizer's 250 output corpus, at the word level, with the transcribed corpus that has been tagged by the named entity training tagger 240. Then, in step 3700, a named entity language model is created. This named entity language model may be a series of regular grammars coded as Finite-State-Machines (FSMs). The aligner 260 creates named entity language models after the transcribed training corpus is labeled, parsed and tagged by the named entity training tagger 240 in order to extract named entity contexts on the clean text.
  • [0064]
    Only the named entity contexts correctly tagged according to the labels marked by the labeler 220 are kept. Then, all the digits, natural numbers and proper names are replaced by corresponding non-terminal symbols. Finally, all of the patterns representing a given tag are merged in order to obtain one FSM for each tag coding the regular grammar of the patterns found in the corpus.
  • [0065]
    The corpora CNE and CBK are replaced by their corresponding sections in the recognizer output corpus and stored along with the named entity language models in the named entity training database 140. As such, the inconvenience of learning directly a model on a very noisy channel is balanced by structuring the noisy data according to constraints obtained on the clean channel. This leads to an increase in performance.
  • [0066]
    Specifically, the training recognizer 250 can generate, as output, a word-lattice as well as the highest probability hypothesis called the 1-best hypothesis. In customer care applications, the word-error-rate of the 1-best hypothesis is around 27%. However, by having the aligner 260 perform an alignment between the transcribed data and the word lattices produced by the training recognizer 250, the word-error-rate of the aligned corpus drops to around 10%.
  • [0067]
    In step 3800, the recognized training corpus is updated using the aligned corpus and also stored in the named entity database 140 for use in the text classification performed by the named entity detection and extraction unit 160. In this process, the aligner 260 processes the text classifier training corpus using the named entity tagger 240 with no rejection. On one side, all of the named entities that are correctly tagged by the named entity tagger 240 according to the labels given by the labeler 220 are kept. On the other side, all the false positive detections are labeled by aligner 260 with the tag OTHER. Then, the text classifier 520 (introduced below) is trained in order to separate the named entity tags from the OTHER tags using the text classifier training corpus. The process goes to step 3900 and ends.
  • [0068]
    The named entity detection and extraction process will now be discussed in relation to FIGS. 4-7. More particularly, FIGS. 4 and 5 illustrate more detailed exemplary diagrams of portions of input communication processing subsystem 120 and FIGS. 6 and 7 illustrate an exemplary named detection and extraction process and an exemplary task classification process, respectively.
  • [0069]
    [0069]FIG. 4 illustrates an exemplary named entity detection and extraction unit 160. The named entity detection and extraction unit 160 may include a named entity detector 410 and named entity extractor 420.
  • [0070]
    [0070]FIG. 5 is a more detailed block diagram illustrating an exemplary named entity detector 410. The named entity detector 410 may include a named entity tagger 510 and a text classifier 520. The operations of individual units shown in FIGS. 4 and 5 will be discussed in detail in relation to FIGS. 6 and 7 below.
  • [0071]
    [0071]FIG. 6 is an exemplary flowchart illustrating a possible named entity detection and extraction process using the exemplary system described in relation to the figures discussed above.
  • [0072]
    In this regard, the named entity detection and extraction process begins at step 6100 and proceeds to step 6200 where the input communication recognizer 150 recognizes the input communication from the user and produces a lattice.
  • [0073]
    In an automated communication recognition process, the weights or likelihoods of the paths of the lattices are interpreted as negative logarithms of the probabilities. For practical purposes, the pruned network will be considered. In this case, the beam search is restricted in the lattice output, by considering only the paths with probabilities above a certain threshold relative to the best path.
  • [0074]
    Most recognizers can generate, as output, a word-lattice as well as the highest probability hypothesis called 1-best hypothesis. This lattice generation can be made at the same time as the 1-best string estimation with no further computational cost. As discussed above, in the customer care corpus, the word error rate of the 1-best hypothesis is around 27%. However, by performing an alignment between the transcribed data and the word lattices produced by the recognizer, the word error rate of the aligned corpus (called the oracle word error rate) dropped to approximately 10%. This simply means that, although the system has nearly all the information for decoding the corpus, most of the time the correct transcription is not the most probable one according to the recognition probabilistic models.
  • [0075]
    Past studies attempted to take advantage of the oracle accuracy of the word lattice in order to re-score hypotheses. A small improvement in the word error rate is generally obtained by these techniques but the main advantage attributed to them is the calculation of a confidence score, for each word, integrating both an acoustic and linguistic score into a posterior probability. Nevertheless, these techniques, as well as those based on a mufti-recognizer, still output a re-scored 1-best hypothesis, which is very far from the oracle hypothesis that can be found in the word-lattice.
  • [0076]
    One of the main reasons for this difference in performance between 1-best and oracle hypothesis is due to the fact that recognizer probabilistic models maximize globally the probability of finding the correct sentence. By having to perform equally well on every part of a communication input, the recognizer model is not always able to characterize properly some local phenomenon.
  • [0077]
    For example, it has been shown that having a specific model for recognizing numeric language in conversational speech improves significantly the performances. However, such a model can be used only to decode answers to a direct question asking for a numerical value, and in a user-initiative dialogue context, the kind of language the caller is going to use is not known in advance. For these reasons, a word lattice transformed in a chain-like structure called Word Confusion Network (WCN) and coded as a Finite State Machine (FSM), is provided as an input to the NLU unit 170.
  • [0078]
    The training recognizer 250 generally produces word lattices during the recognition process as an intermediate step. Even if they represent a reduced search-space compared to the first one obtained after the acoustic parameterization, they still can contain a huge number of paths, which limits the complexity of the methods that can be applied efficiently to them. For example, the named entity parser 230 statistically parses the input word lattice by first pruning the word lattice in order to keep the 1000 best paths.
  • [0079]
    In addition, Word Confusion Networks (WCN) can be seen as another kind of pruning. The main idea consists in changing the topology of the lattice in order to reflect the time-alignment of the words in the signal. This new topology may be a concatenation of word-sets.
  • [0080]
    During this transformation, the word lattice is pruned and the score attached to each word is calculated to represent the posterior probability of the word (i.e., the sum of the probabilities of all the paths leading to the word) and some new paths can appear by grouping words into sets. An empty transition (called epsilon transition) is also added to each word set in order to complete the probability sum of all the words of the set to 1.
  • [0081]
    The main advantages of this structure are as follows. First, their size as they are about a hundred times smaller than the originally used lattice. Secondly, the posterior probabilities that can be used directly as a confidence score for each word. Finally, the topology of the network itself because by selecting two states, S1 and S2 that correspond to a given time zone on the signal, all the possible paths covering this zone start at S1 and end at S2 are sure to be covered. In the original lattice, the topology does not directly reflect the time alignment of the words, and enumerating all the paths between two states does not guarantee that no other paths exist on the same time interval.
  • [0082]
    In step 6300, the named entity tagger 510 of the named entity detector 410 detects the named entity values and context of occurrence using the named entity language models stored in the named entity database 140.
  • [0083]
    In step 6400, the named entity tagger 510 inserts named entity tags on the detected named entities. The named entity tagger 510 maximizes the probability expressed by the equation (1) above by means of a search algorithm. The named entity tagger 510 may give to each named entity tag a confidence score for being contained in a particular sentence, without extracting a value. These named entity tags by themselves are a useful source of information for several modules of the dialogue system. Their usefulness is discussed in further detail below.
  • [0084]
    Then, in step 6500, in order to be able to tune the precision and the recall of the named entity language models for the recognition and understanding system 120, each named entity detected by the named entity tagger 510 is scored by the text classifier 520. The scores given by the text classifier 520 are used as confidence scores to accept or reject a named entity tag according to a given threshold. As discussed above, the text classifier 520 was trained to separate the named entity tags from the OTHER tags using the text classifier training corpus stored in the named entity database 140.
  • [0085]
    After the named entities are detected, they are extracted using the named entity extractor 420. A 2-step approach is undertaken for extracting named entity values. This approach is taken because it is difficult to predict exactly what the user is going to say after a given prompt so that the named entities are detected on the 1-best hypothesis produced by the recognizer 150.
  • [0086]
    In addition, once detected areas in the input communications which are likely to contain named entities with a high confidence score are identified by the text classifier 520, the named entity extractor 420 extracts the named entity values from the word lattice using the named entity language models stored in the named entity database 140 (specific to each named entity tag). However, only on the areas selected by the named entity tagger 510.
  • [0087]
    Extracting named entity values is crucial in order to complete a user's request because these values represent the parameters of the database queries. In system-initiative dialogue systems, each named entity value is obtained by a separate question and named entities embedded in a sentence are usually ignored. For mixed-initiative dialogue systems it is important for the named entity extractor 420 to extract the named entity values as soon as the caller expresses them, even if the dialogue manager 180 hasn't explicitly asked for them. This point is particularly crucial in order to make the dialogue feel natural to the user. While extracting named entity values embedded in a sentence is much more difficult than processing answers to a direct question, the named entity extractor 420 can use, for this purpose, all the information that has been collected during the previous iterations of the dialogue.
  • [0088]
    For example, in a customer care application, as soon as the customer is identified, all of the information contained in his bill could be used. If a query concerns an unrecognized call on a bill, the phone number to detect is contained in the bill, among all the other calls (N) made during the same period. Extracting a 10-digit string among N (with N of order a few tens) is significantly easier than finding the right string among the 1010 possible digit strings.
  • [0089]
    In step 6600, the named entity extractor 420 performs a composition operation between the FSM associated to a named entity tag and an area of the word-lattice where such a tag has been detected by the named entity tagger 510. Because having a logical temporal alignment between words makes the transition between 1-best hypothesis and word-lattice easier, and because posterior probabilities are powerful confidence measures for scoring words, the word-lattices produced by the input communication recognizer 150 may be first transformed into a chain-like structure (also called “sausages”).
  • [0090]
    Once the named entity extractor 420 composes the FSM with the portion of the chain corresponding to the named entity area detected, in step 6700, the named entity extractor 420 searches for the best path according only to the confidence scores attached to each word by the text classifier 520 in the chain. The FSM is not weighted, as all the patterns extracted by the named entity extractor 420 from the named entity language model are considered valid. Therefore, only the posterior probability of each sequence of word following the patterns is taken into account. According to the kind of named entity tag extracted by the named entity extractor 420, the named entity extractor 420 performs a simple filtering process of the best path in the FSM in order to output values to the natural language understanding unit 170, in step 6800.
  • [0091]
    Traditionally, the natural language understanding unit 170 component of a dialogue system is located between the recognizer 150 component and the dialogue manager 180. The recognizer 150 outputs the best string of words estimated from the speech input according to the acoustic and language models with a confidence score attached to each word. The goal of the natural language understanding unit 170 is then to extract semantic information from this noisy string of words in order to give it to the dialogue manager 180. This architecture relies on the assumption that ultimately the progress made by automated communication recognition technology will allow the recognizer 150 to transcribe almost perfectly the communication input. Accordingly, as discussed above, a natural language understanding unit 170 may be designed and trained on manual transcriptions of human-computer dialogues.
  • [0092]
    This assumption is reasonable when the language accepted by the dialogue application is very constrained, like in system-initiative dialogue systems. However, it becomes unrealistic when dealing with conversational spontaneous communications. This is because of the Out-Of-Vocabulary (OOV) word phenomenon which is inevitable, even with a very large recognition lexicon (occurrences of proper names like customers name, for example) and also because of the ambiguities intrinsic to spontaneous communication. Indeed, transcribing and understanding are two processes that should be done simultaneously as most of the transcription ambiguities of spontaneous speech can only be resolved by some understanding of what is being said.
  • [0093]
    Thus, the usual architecture of dialogue systems may be modified by integrating the transcription process into the natural language understanding unit 170. Instead of producing only one string of words, the recognizer 150 will output a word lattice where the different paths can correspond to different interpretations of the communication input.
  • [0094]
    The process then goes to step 6900 and ends or continues to step 6100, if the named entity process is applied to a task classification system process.
  • [0095]
    The above processes discussed in relation to FIGS. 3 and 6 will be explained in further detail below. This discussion will also cover how the processes are integrated.
  • [0096]
    Earlier methods developed for the named entity detection task were dedicated to process proper name named entities like person, location or organization names, and little attention had been given to the process of numerical named entities. This was mainly due to the importance of proper names in the corpus processed (news corpus) and also because these named entities are much more ambiguous and difficult to detect and recognize than the numerical ones. Often, simple hand-written grammars are sufficient for processing named entities like dates or money amount from written texts.
  • [0097]
    The situation is very different in a dialogue context. Firstly, numerical expressions are crucial as they correspond to the parameters of the queries that are going to be sent to the application database in order to fulfill a specific task. Secondly, the difficulties of retrieving such entities are increased by three different factors: recognition errors, spontaneous speech input and context-dependent named entities.
  • [0098]
    Recognition of communications over communication devices such as the telephone, videophone, interactive Internet, etc., especially with a large population of non-expert users, is a very difficult task. On the customer corpus, the average word error recognition is about 27%. Even if thus result still allows the natural language understanding unit 170 and the dialogue manger 180 to perform efficient task-type classification, there are too many errors for using the same named entity detection and extraction techniques on the recognizer 150 output as those applied on written text.
  • [0099]
    For example, the sentence “and my number is two oh one two six four twenty six ten” might be mis-recognized as “and my number is too oh one to set for twenty six ten” Therefore, instead of having one string of 10 digits for the phone number, there are 2 strings, one of 3 digits and one of 4 digits.
  • [0100]
    Processing spontaneous conversational communications does not affect just the recognizer 150. The variability of the language used by the callers can be much higher that is found in a written corpus. First, because of the dysfluencies occurring in spontaneous communications, like stops edits and repairs. Second, because of the application itself. In other words, most of the people communicating with a machine are not familiar with human-computer communication interaction and conducting a conversation without knowing the level of understanding of the “person” you are communicating with may be disturbing.
  • [0101]
    Here is an example of such an utterance, where standard text parsing techniques would be difficult to apply:
  • [0102]
    I didn't know if I was talking to a machine or I got the impression I was waiting for some further instructions there yes I just the the latest one that I just received here for an overseas call to U K it was for seventy minutes and the amount was X and I I've gone back my bills and the closest I can come is like in May I'd a ten minute and it was X.
  • [0103]
    As discussed above, most of the named entities used in a dialogue context are context-dependent. This means that a certain level of understanding of the whole sentence is necessary in order to classify a given string as a named entity. For example, for the named entity tag Which Bill in the customer care scenario, the first task is to recognize that the caller is talking about his bill. Then, the context which will allow identification of the bill may be detected. Consider the following examples:
  • [0104]
    the statement I received this morning=>tag=Which_bill, value=latest my October 2001 bill=>tag=Which_bill, value=2001/10/??
  • [0105]
    Such named entities can't be easily represented by regular grammars, as the context needed for detecting them can be quite large.
  • [0106]
    When dealing with text input, handwritten rule-based systems have proven to give the best performances on some named entity extraction tasks. But when the input is noisy (lack of punctuation or capitalization, for example) or when the text is generated by the recognizer 150, data-driven approaches seem to out perform rule-based methods. However, by carefully designing rules specific to the recognizer 150 transcripts, good performances can be achieved.
  • [0107]
    This is also the case when the named entities to process are numerical expressions: context-independent named entities, like dates and phone numbers, are generally expressed according to a set of fixed patterns which can be easily modeled by some hand-written rules in a Context-Free Grammar (CFG). These rules can be obtained by a study of a possibly small example corpus that may be manually transcribed. The main advantage of such grammars is their generalization power, regardless of the size of the original corpus they were induced from. For example, as long as a user expresses a date following a pattern recognized by the grammar, all the possible dates will be equally recognized. However, two issues arise when using CFG: one is the difficulty of taking into account recognition errors; the other one is the modeling of context-dependent named entities expressed in a spontaneous speech context. These two issues are discussed further below.
  • [0108]
    A simple insertion or substitution in a named entity expression will lead to a rejection of the expression by the grammar. A named entity detection system based only on CFG applied to the best string hypothesis generated by the recognizer 150 will have a very high false rejection rate because of the numerous insertions, substitutions and deletions occurring in the recognizer 150 hypothesis. Two possible ways to address this problem are as follows:
  • [0109]
    1) Replace the CFG by a stochastic model in order to estimate the probability of a given distortion of the canonical form of a named entity; or
  • [0110]
    2) Apply the CFG, not only to the best string hypothesis of the recognizer 150, but to the whole WCN as mentioned above.
  • [0111]
    The first possibility will be discussed below. For the second possibility, the CFGs are represented as Finite State Machines (FSM) where each path, from a starting state to a final state, corresponds to an expression accepted by the corresponding grammar. Because the handwritten grammars are not stochastic, the corresponding FSMs aren't weighted and all paths are considered equals. With this representation, applying a grammar to a WCN only consists in composing the two FSMs. Because an epsilon (empty transition exists between each state of the WCN, all the words surrounding a named entity expression will be replaced by the epsilon symbol and finding all the possible matches between a grammar and a WCN corresponds to enumerating all the possible paths in the composed FSM.
  • [0112]
    In practice, not all the possible paths need to be extracted, just the N-best ones. Because the WCN is weighted with the confidence score of each word, the best match between a grammar and the network is the expression that contains the words with the highest confidence score as well as the smallest number of epsilon transitions, corresponding to the best coverage on the WCN by the named entity expression. This technique, leads to a decrease in the false rejection rate of the detection process. Unfortunately, a decrease in correct detections is also noticeable as the number of false positive matches in the WCN is very difficult to control by means of only the word confidence scores.
  • [0113]
    Integrating spontaneous speech dysfluencies in regular grammars in order to represent named entities expressed in a user-initiative dialogue context is not an easy task. Moreover, most of the named entities relevant to the dialogue manager 180 are context-dependent in that a certain portion of the context of occurrence has to be modeled with the entity itself by the grammar. With regard to these difficulties, and thanks to the amount of transcribed data contained in the customer care corpus, it seems natural to try to induce grammars directly from the labeled data.
  • [0114]
    Each dialogue of the labeled corpus is split into turns and to each turn is attached a set of fields containing various information like the prompt name, the dialogue context, the exact transcription, the recognizer output, the NLU tags, etc. One of these fields is made of a list of triplets, one for each named entity contained in the sentence, as presented above. The field corresponding to the named entity context is supposed to contain the smallest portion of the sentence, containing the named entity, which characterizes this portion as a named entity. For example, in the sentence: I don't recognize this 22 dollar phone call on my December bill the context attached to the named entity Item_Amount is “this 22 dollar phone call” and the one attached to the named entity Which_Bill is “December bill.”
  • [0115]
    Of course such a definition relies heavily on the knowledge the labeler has of the task, and some lack of consistency can be observed between labelers. However, this corpus constitutes a precious database containing “real” spontaneous speech data with semantic information. Adding a new sample string to a CFG with a start non-terminal S consists in simply adding a new top-level production rule (for S) that covers the sample precisely. Then, a non-terminal is added for each new terminal of the right-hand side of the new rule in order to facilitate the merging process.
  • [0116]
    The key point of all the grammar induction methods is the strategy chosen for merging the different non-terminal: if no merging is performed, the grammar will only model the training examples, if too many non-terminals are merged, the grammar will accept incoherent strings. In the present case, because the word strings representing the named entity contexts are already quite small, and because the induction of a wrong pattern can heavily affect the performance of the system by generating a lot of false positive matches, the merging strategy may be limited to a set of standard non-terminals: digit, natural numbers, and day and month names. The following substitutions are considered:
  • [0117]
    each digit (0 to 9) is replaced by the token $digitA;
  • [0118]
    each natural number (from 10 to 19) is replaced by the token $digitB;
  • [0119]
    each ten number (20, 30, 40, . . . 90) is replaced by the token $digitC;
  • [0120]
    each ordinal number is replaced by the token $ord;
  • [0121]
    each name representing a day is replaced by the token $day;
  • [0122]
    each name representing a month is replaced by the token $month;
  • [0123]
    For example, the following named entity contexts, corresponding to the named entity tag item_Amount: charged for two ninety five charging me a dollar sixty five become: charged for $digitA $digitc $digita charging me a dollar $digitc $digitA.
  • [0124]
    Let's point out here that the symbols $digitA,B,C, $ord, $month and $day are considered as terminal symbols. The same kind of preprocessing operation will be performed on the text-strings that are going to be parsed by the grammars. Despite the size of the corpus available, there is not enough data for learning a reliable probability for a given named entity to be expressed in a given way. Therefore, all rules are considered are equal and the grammars obtained aren't stochastic. Each grammar rule obtained for a given named entity tag is then turned into a simple FSM, and the complete grammar for the tag is the union of all the different FSMs extracted from the corpus.
  • [0125]
    After the training phase, each named entity tag is associated with an induced grammar modeling its different expressions in the training corpus. It is therefore possible to enrich such grammars with handwritten ones as presented above. Because neither kind of the grammars is stochastic, their merging process is straightforward: the merged grammar is simply the union of the different FSMs corresponding to the different grammars. These handwritten grammars can be seen as a back-off strategy for detecting named entities.
  • [0126]
    For example, the named entity tag Which Bill can have either a value corresponding to a date (the bill issued date, for example) or to a relative position (current or previous). But not all dates can be a considered as a Which_Bill, like for example, a date corresponding to a given phone call. In the named entity context-training corpus, all the dates corresponding to a tag Which-Bill are embedded in a string clarifying the nature of the date, like: bill issued on Nov. 12, 2001.
  • [0127]
    Therefore all the strings matching a grammar built from this example are very likely to represent a Which_Bill tag. However, if the named entity is expressed in a different way and not represented in the training corpus, like bill dated Nov. 12, 2001, the grammar will reject the string. This data sparseness problem is inevitable whatever size is the training corpus when the application is dealing with spontaneous speech.
  • [0128]
    Adding handwritten rules is then an efficient way of increasing the recall of the named entity detection process. For example, in the previous example, if a handwritten grammar representing any kind of date is added to the data-induced grammar related to the tag Which_Bill, all the expressions identifying a bill by means of a date will be accepted. The first expression bill issued on Nov. 12, 2001 will still be identified by the pattern found in the training corpus, because it's longer than the simple date and gives a better coverage of the sentence. The second expression bill dated Nov. 12, 2001 will be reduced to the date itself and accepted by the back-off handwritten grammar representing the dates.
  • [0129]
    However, if this technique improves the recall, the precision drops because of all the false positive detections generated by the non context-dependent rules of the hand-written grammars. That's why such a technique has to be used in conjunction with another process, which can give a confidence score for a given context to contain a given tag. This method will be discussed further below.
  • [0130]
    Once a named entity context is detected by a grammar, a named entity value has to be extracted by the named entity extractor 420. As presented above, each named entity tag can be represented by one or several kind of values. The evaluation of a named entity processing method applied to dialogue systems has to be done on the values extracted and not the word-string itself. From the dialogue manager's 180 point of view, the normalized values of the named entities will be the parameters of any database dip, and not the string of words, symbols, or gestures used to express them.
  • [0131]
    For example, the value of the following named entity bill issued on Nov. 12, 2001 is “2001/11/12”. If the same value is extracted from the recognizer 150 output, this will be considered as a success, even if the named entity string estimated is bill of the Nov. 12, 2001 or issue the November 12th of 2001.
  • [0132]
    Evaluating the values instead of the strings is called the evaluation of the understanding accuracy. Extracting a value from a word-string can be straightforward for some simple numerical named entities, like Item_Amount. However, some ambiguities can exist, even for standard named entities like phone numbers. For example, the following number 220 386 1200 can be read as two twenty three eight six twelve hundred and this string can then be turned into these following digit strings: 2203861200 223861200 22038612100 2238612100.
  • [0133]
    In order to produce correct values, a transduction process is implemented that outputs values during the parsing by the parser 230 using the named entity grammars already presented. The result of this transduction on the previous phone string will be: two->2 twenty->20 three->3 eight->8 six->6 twelve->12 hundred->00.
  • [0134]
    For the handwritten grammars, this is done by simply adding to each terminal symbol the format of the output token that has to be generated. For example, the previous transduction is made by the rule:
  • [0135]
    <PHONE>->$digitA/$digit1 $digitC/$digit2 $digitA/$digit1 $digitA/$digit1 $digitA/$digit1 $digitB/$digit2 hundred/00. with $digit1 corresponding to the first digit of the input symbol, $digit2 to the first two digits of the input symbol, and 00 to the digit string 00.
  • [0136]
    The same process is used for data-induced grammars. In this case, word to word, the word context and the value of each sample of the training corpus are aligned first. The symbols that don't produce any output token are transduced into the epsilon symbol, and similarly the output tokens that are not produced by a word from the named entity context are considered emitted by the same epsilon symbol. This alignment is done by means of simple rules that make the correspondence, at the word level, between input symbols and output tokens.
  • [0137]
    Because all the grammars use are coded into FSMs, the extraction process is implemented as a transduction operation between these FSMs and the output of the recognizer 150. On the grammar side, the FSMs are simply transformed into transducers by adding the output tokens attached to each input symbol for each arc of the FSMs. On the recognizer 150 output side, the following process is performed:
  • [0138]
    (1) if the recognizer 150 output is a 1-best word string, it is turned into a sequential FSM, otherwise the word-lattice, or word confusion network, is used directly as an FSM;
  • [0139]
    (2) the FSM obtained is turned into a transducer by duplicating each word attached to each arc as an input and output symbol;
  • [0140]
    (3) each output symbol belonging to one of these non-terminal class: $digitA, $digitB, $digitc, $ord, $month and $day, is replaced by the name of its class.
  • [0141]
    The extraction process is now a byproduct of the detection phase. In this regard, once a string is accepted by a grammar by using a composition operation between their corresponding transducers, at the same time, the matching of the input and output symbols of both transducers will remove any ambiguities for the translation of a word string into a value. To obtain a value, one path is chosen in the composed FSM, all the epsilon output symbols are filtered, and then the other input-output symbols are matched.
  • [0142]
    Assume for example, that FSM1 is the transducer corresponding to the recognizer output and FSM2 is one of the grammars automatically induced from the training data, which represents the transduction between a named entity context like twenty two sixteen charge and the value 22.16. From FSM1, the following values can be extracted: 64.10 64.00 60.10 60.00 60.04 4.10 4.00 10.00 0.64 0.60 0.04 0.10 But after the composition between the two FSMs, the following transduction occurs:
  • [0143]
    sixty->$digit1 four->$digit1 eps->. ten->$digit10 charge->eps
  • [0144]
    Thus, in order to produce a value, the first digit of sixty and the first digit of four are taken, the token is added, the first two digits of ten are taken and the word charge is finally erase. From the twelve possible values previously enumerated, only one match is obtained, which is 64.10.
  • [0145]
    One of the main advantages of this approach is the possibility of generating an n-best solution on the named entity values instead of the named entity strings. Indeed, each path in the composed FSM between the recognizer 150 output transducer and the grammar transducer, once all the epsilon transitions have been removed, corresponds to a different named entity value. By extracting the n-best paths (according to the confidence score attached to each word in the recognizer 150 output the n-best values are automatically obtained according to the different paths, in the grammars and in the FSM produced by the recognizer 150.
  • [0146]
    In contrast, if the n-best generation is done on the word lattice alone, one has to generate a much bigger set of paths in order to obtain different values, as most of the n-best paths will differ only by words that are not used in the named entity value generation process.
  • [0147]
    Even with grammars induced from data, CFGs still remain too strict for dealing efficiently with recognizer errors and spontaneous speech effects. As discussed above, one possibility is to add non-determinism and probabilities to the grammars in order to model and estimate the likelihood of the various distortions that might occur. This approach relies heavily on the amount of data available as stochastic grammars need a lot of examples in order to estimate reliable transition probabilities. Considering that not enough data existed for estimating such grammars, and with regards to the poorer results obtained with this method compared to those obtained with a rule-based system and a Maximum-Entropy tagger, a simpler model based on a tagging approach was implemented.
  • [0148]
    Tagging methods have been widely used in order to associate to each word of a text a morphological tag called Part-Of-Speech (POS). In the framework of named entity detection, there may be one tag for each kind of named entity and a default tag corresponding to the background text, between each named entity expression.
  • [0149]
    Two kinds of models have been proposed. One is based on a state-dependent Language Model approach considering tile transition probabilities between words and tags within a sentence. The other one is based on a Maximum Entropy (ME) model. Both approaches heavily rely on the features selected for estimating the probabilities of the different models.
  • [0150]
    For example, a tagging approach based on a Language Model (LM) very close to the standard language models used during the speech recognition process is chosen. This choice has been made according to two considerations. First, having recognizer 150 transcripts instead of written text automatically limits the number of features that can be used in order to train the models. No capitalization, punctuation or format information is available, and the only parameters that can be chosen are the words from the text stream produced by the recognizer 150. The advantage of having the possibility of mixing a large scale of features as in the maximum entropy approach, this will not apply in this scenario.
  • [0151]
    Second, because of the recognizer errors (between 25% and 30% word error rate), trigger words or fixed patterns of words and/or part-of-speech cannot be implemented. Indeed, even if a word or a short phrase is very relevant for identifying a given named entity on written text input, this word or this phrase can be, for any reason, very often mis-recognized by the recognizer 150 and all the probabilities associated to them are then lost. That is why the only robust information available for tagging a word with a specific tag is simply its surrounding words within the sentence, and in this case, the language model approaches are very efficient and easy to implement.
  • [0152]
    Following the formal presentation of tagging models, named entity detection process can be further described. It is assumed that the language contains a fixed vocabulary w1, w2 . . . , wv, which is the lexicon used by the recognizer 150. It is also assumed that a fixed set of named entity tags t1 t2, . . . , tT and a tag t0 represents the background text. A particular sequence of n words is represented by the symbol w1,n and for each i≧n, wiεw1, w2, . . . ,wV.
  • [0153]
    In a similar way, a sequence of n tags is represented by t1,n and for each i≧n, tiεt0,t2, . . . , tT. The tagging problem can then be formally defined as finding the sequence of tags t1,n, in the following way: τ ( w 1 , n ) = arg max t 1 , n P ( t 1 , n | w 1 , n ) ( 1 )
    Figure US20030191625A1-20031009-M00002
  • [0154]
    Equation 1 can be turned as: τ ( w 1 , n ) = arg max t 1 , n P ( t 1 , n , w 1 , n ) P ( w 1 , n ) ( 2 )
    Figure US20030191625A1-20031009-M00003
  • [0155]
    Because P(w1,n) is constant for all ti,n,w1,n), the final equation is: τ ( w 1 , n ) = arg max t 1 , n P ( t 1 , n , w 1 , n ) ( 3 )
    Figure US20030191625A1-20031009-M00004
  • [0156]
    For calculating P(t1,n,w1,n), P(t1) in equation 3 can be broken out: P ( t 1 , n , w 1 , n ) = i = 1 n P ( t 1 , i - 1 , w 1 , i - 1 ) P ( w i | t 1 , i , w 1 , i - 1 ) ( 4 )
    Figure US20030191625A1-20031009-M00005
  • [0157]
    In order to collect these probabilities, the following Markov assumptions are made:
  • P(t i |t 1,i−1 ,w 1,i−1)=P(t i |t 1−2,i−1 ,w i−2,i−1)  (5)
  • P(w i |t 1,i ,w 1,i−1)=P(w i |t i−2,i ,w i−2,i−1)  (6)
  • [0158]
    It is assumed that the ti tag is only dependent of the two previous words and tags. Similarly, the word wi is dependent on the two previous words and tags as well as the knowledge of its tag. Unlike the part-of-speech tagging method, it is not assumed that the current tag is independent of the previous words. This assumption is usually made because of the data sparseness problem, but in this case, the words to the history can be integrated because the number of tags are limited, and the number of different words that can be part of a named entity expression is also very limited (usually digits, natural numbers, ordinal number, and a few key words like: dollars, cents, month name, . . . ). With these assumptions, the following equation is obtained: τ ( w 1 , n ) = arg max t 1 , n P ( t i | t i - 2 , i - 1 , w i - 2 , i - 1 , ) P ( w i | t i - 2 , i , w i - 2 , i - 1 ) ( 7 )
    Figure US20030191625A1-20031009-M00006
  • [0159]
    In order to estimate the parameters of this model, the training corpus is defined, as presented further below.
  • [0160]
    The probabilities of this tagging model are estimated on a training corpus, which contains human-computer dialogues transcribed by the transcriber 210 (or alternatively, manually transcribed). To each word wi of this corpus is associated a tag ti where ti=t0 if the word doesn't belong to any named entity expression, and ti=tn if the word is part of an expression corresponding to the named entity tag n. This training corpus is built from the HMIHY corpus in the following way. The corpus may contain about 44K dialogues (130K sentences), for example.
  • [0161]
    This corpus is divided into a training corpus containing 35K dialogues (102K sentences) and a test corpus of 9K dialogues (28K sentences). Only about 30% of the dialogues and 15% of the sentences contain named entities. This corpus represents only the top level of the whole dialogue system, corresponding to the task classification routing process. This is why the average number of turns is rather small (around 3 turns per dialogue and the percentage of sentences containing a named entity is also small (the database queries which require a lot of named entity values are made in the legs of the dialogue and not at the top level. Nevertheless, still obtain a 16K sentence training corpus, manually transcribed, where each sentence contains at least one named entity.
  • [0162]
    All these sentences are transcribed by the transcriber 210 and semantically labeled by the labeler 220. The semantic labeling by the labeler 220 consists in giving to each sentence the list of task types that can be associated to it as well as the list of named entities contained. For example, consider the sentence
  • [0163]
    “I have a question about my bill I I don't recognize this 22 dollar call to Atlanta on my July bill.”
  • [0164]
    This sentence can be associated with the two following task types: Billing-Question and Unrecognized-Number and the named entity tags: Item_Amount, Item_Place and Which_Bill.”
  • [0165]
    In addition to these labels, 35% of the sentences containing a named entity tag have also been labeled by the labeler 220 according to the format presented above where in addition to the label itself, the named entity context and value are also extracted by the named entity extractor 420. This subset of the corpus is directly used to train the named entity training tagger 240. In this regard, a tag is added to each word belonging to a named entity context representing the named entity, and similarly the tag to is added to the background text.
  • [0166]
    The last step in the training corpus process is a non-terminal substitution process applied to digit strings, ordinal numbers, and month and day names. Because the goal of the named entity training tagger 240 is not to predict a string of words but to tag an already existing string, the generalization power of the named entity training tagger 240 can be increased by replacing some words by general non-terminal symbols. This is especially important for digit strings, as the length of a string is a very strong indicator of its purpose.
  • [0167]
    For example, 10-digit strings are very likely to represent phone numbers, but if all the digits are represented as single tokens in the training corpus, the 3-gram language model used by the named entity training tagger 240 won't be able to model accurately this phenomenon as the span of such a model is only 3 words. In contrast, by replacing the 10-digit string in the corpus by the symbol $digit10, a 3-gram LM will be able to correctly model the context surrounding these phone numbers. According to that consideration, all the N-digit strings axe replaced by the symbol $digitN, the ordinal numbers are replaced by $ord, the month names by $month and the day names by $day. The parameters of the probabilistic model of the named entity training tagger 240 are then directly estimated from this corpus by means of a simple 3-gram approach with back off for unseen events.
  • [0168]
    Tagging approaches based on language models with back off can be seen as stochastic grammars with no constraints as every path can be processed and receive a score. Therefore, the handling of the possible distortions of the named entity expressions found in the training corpus is automatic and this allows modeling of longer sequences without risking rejecting correct named entities expressed or recognized in a different way.
  • [0169]
    Thus, it is interesting to expand the contexts used to represent the named entities in the training corpus. This allows taking into account more contextual information bringing which brings two main advantages. First, some relevant information for processing ambiguous context-dependent named entities can be captured. Second, by using a larger span, the process is more robust to recognizer errors. Of course, the trade-off of this technique is the risk of data-sparseness that can occur by increasing the variability inside each named entity class. A context-expansion method may be implemented based on a syntactic criterion as follows:
  • [0170]
    (1) the training corpus is first selected and labeled according to the method presented above;
  • [0171]
    (2) then, a part-of-speech-tagging followed by a syntactic bracketing process is performed on each sentence in order to insert boundaries between each noun phrase, verbal phrase, etc;
  • [0172]
    (3) finally, all the words of each phrase that contains at least one word marked with a named entity tag are marked with the same tag.
  • [0173]
    Increasing the robustness of extraction information systems to recognizer errors is one of the current big issues of automated communication processing. As discussed above, the recognizer transcript cannot be expected to be exempt of errors, as the understanding process is linked to the transcription process. Even if statistical models are much more robust to recognizer errors than rule-based systems, the models are usually trained on manually transcribed communications and the recognizer errors are not taken into account explicitly.
  • [0174]
    This strategy certainly emphasizes the precision of the detection, but a great loss in recall can occur by not modeling the recognizer behavior. For example, a word can be considered as very salient information for detecting a particular named entity tag. But if this word is, for any reason, very often badly recognized by the recognizer system, its salience won't be useful on the recognizer output.
  • [0175]
    Some methods increase the robustness of their models to recognizer errors by randomly generating errors in order to noise the training data. But because of a mismatch between the errors generated and the ones occurring in the recognizer output, no improvement was shown using this technique. In this method, the whole training corpus is processed by the recognizer system in order to learn automatically the confusions and the mistakes that are likely to occur in the deployed system. This recognizer output corpus is then aligned by the aligner 260, at the word level, with the transcription corpus. A symbol NULL is added to the recognizer transcript for every deletion and each insertion is attached to the previous word with the symbol +. By thus means, both manual transcriptions and recognizer outputs contain the same number of tokens. The last process consists simply in transferring the tags attached to each word of the manual transcription, as presented above, to the corresponding token in the recognizer output.
  • [0176]
    Such a method balances the inconvenience of learning directly a model on a very noisy channel (recognizer output) by structuring the noisy data according to constraints obtained on the clean channel (manual transcriptions).
  • [0177]
    The named entity tagging process consists in maximizing the probability expressed by equation 7 by means of a search algorithm. The input is the best-hypothesis word string output of the recognizer module, and is pre-processed in order to replace some tokens by non-terminal symbols as discussed above. Word-lattices are not processed in this step because the tagging model is not trained for finding the best sequence of words, but instead for finding the best sequence of tags for a given word string. In the tagged string, each word is associated with a tag, t0 if the word is not part of any named entity, and tn if the word is part of the named entity n. An SGML-like tag <n> is inserted for each transition between a word tagged t0 and a word tagged tn. Similarly, the end of a named entity context is detected by the transition between a word tagged tn and a word tagged t0 and is represented by the tag </n>.
  • [0178]
    In order to be able to tune the precision and the recall of the model for the deployed system, a text classifier 520 scores each named entity context detected by the tagger 510. This text classifier 520 is trained as follows:
  • [0179]
    (1) the recognizer output of the training corpus is processed by the named entity tagger;
  • [0180]
    (2) on one hand, all the contexts detected and correctly tagged according to the labels are kept and marked with the corresponding named entity tag;
  • [0181]
    (3) on the other hand, all the false positive detections are labeled with the tag OTHER;
  • [0182]
    (4) finally the text classifier 520 is trained in order to separate these samples according to their named entity tags as well as the OTHER tag.
  • [0183]
    During the tagging process, the scores given by the text classifier 520 are used as confidence scores to accept or reject a named entity tag according to a given threshold.
  • [0184]
    As discussed above, the robustness issue of CFCs: on one hand CFCs are often too strict when they are applied to the 1-best string produced by the recognizer 150, and on the other hand, applying them to the entire word lattice might generate too many false detections as there is no way of modeling their surrounding contexts of occurrence within the sentence. The tagging approach presented above provides efficient answers to these problems as the whole context of occurrence of a given named entity expression is modeled with a stochastic grammar that handles distortions due to recognizer errors or spontaneous speech effects. But this latter model can be applied only to the 1-best string produced by the recognizer module, which prevents using the whole word lattice for extracting the named entity values.
  • [0185]
    With this in mind a hybrid method may be implemented based on a 2-step process, which tries to take advantage of the two methods previously presented. First, because what the user is going to say after a given prompt cannot be predicted, the named entities on the 1-best hypothesis are detected only by means of the named entity tagger. Second, once areas in the speech input have been detected which are likely to contain named entities with a high confidence score, the named entity values from the word lattice are extracted with the CFGs but only on the areas selected by the named entity tagger. When processing a sentence, the tagger is first used in order to have a general idea of its content. Then, the transcription is refined using the word-lattice with very constrained models (the CFGs) applied locally to the areas detected by the tagger. By doing so the understanding and the transcribing processes are linked and the final transcription output is a product of the natural language understanding unit 170 instead of the recognizer 150. The general architecture of the process may include the following:
  • [0186]
    a data structure using, both for the models and the input data, the FSM format;
  • [0187]
    the preprocessor as well as the CFG grammars being represented as non-weighted transducers;
  • [0188]
    the Language Model used by the tagger being coded as a stochastic FSM;
  • [0189]
    all the steps in the named entity detection and extraction process being defined as fundamental operations between the corresponding transducers, like composition, best-path estimation and sub-graph extraction;
  • [0190]
    the information which goes to the NLU unit for the task classification process is made of the named entity tags detected, with their confidence scores given by the text classifier, as well as the preprocessed recognizer FSM;
  • [0191]
    the dialogue manager receives the named entity values extracted, with two kind of confidence score: one attached to the tag itself and one given to the value (made from the confidence scores of the words composing the value).
  • [0192]
    [0192]FIG. 7 is a flowchart of a possible task classification process using name entities. In associating a task type to each sentence of the customer care corpus, any method may be used as know to those of skill in the art, including classification methods disclosed in U.S. Pat. Nos. 5,675,707, 5,860,063, 6,021,384, 6,044,337, 6,173,261, and 6,192,110. The input of the dialogue manager 180 is a list of salient phrases detected in the sentence. These phrases are automatically acquired on a training corpus.
  • [0193]
    Named entity tags can be seen as another input for the dialogue manager 180 as they are also salient for characterizing task types. This salience can be estimated by calculating the task-type distribution for a given named entity tag. For example, in the customer care corpus, if a sentence contains an Item_Amount tag, the probability for this sentence to represent a request about an explanation on a bill (Expl_Bill is P(Expl_Bill Item_Amount)=0.35. Thus probability is only 0.09 for a task-type representing a question about an unrecognized number.
  • [0194]
    An important task of the dialogue manager 180 is to generate prompts according to the dialogue history in order to clarify the user's request and complete the task. These prompts must reflect the understanding the system has of the ongoing dialogue. Even if this understanding is correct, asking direct questions without putting them in the dialogue context may confuse the user and lead him to reformulate his query. For example, if a user mentions a phone number in a question about an unrecognized call on his bill, even if the value cannot be extracted because of a lack of confidence, acknowledging the fact that the user has already said the number (with a prompt such as “What was that number again?”) will help the user feel that he or she is being understood.
  • [0195]
    In this regard, the process in FIG. 7 begins from step 6900 in FIG. 6 and continues to step 7100. In step 7100, the dialogue manager 180 may perform task classifications based on the detected named entities and/or background text. The dialogue manager 180 may apply a confidence function based on the probabilistic relation between the recognized named entities and selected task objectives, for example. In step 7200, the dialogue manager 180 determines whether a task can be classified based on the extracted named entities.
  • [0196]
    If the task can be classified, in step 7300, the dialogue manager 180 routes the user/customer according to the classified task objective. In step 7700, the task objective is completed by the communication recognition and understanding system 100 or by another system connected directly or indirectly to the communication recognition and understanding system 100. The process then goes to step 7800 and ends.
  • [0197]
    If the task cannot be classified in step 7200 (e.g., a low confidence level has been generated), in step 7400, the dialogue manager 180 conducts dialogue with the user/customer to obtain clarification of the task objective. After dialogue has been conducted with the user/customer, in step 7500, the dialogue manager unit 180 determines whether the task can now be classified based on the additional dialog. If the task can be classified, the process proceeds to step 7300 and the user/customer is routed in accordance with the classified task objective and the process ends at step 7800. However, if task still cannot be classified, in step 7600, the user/customer is routed to a human for assistance and then the process goes to step 7800 and ends.
  • [0198]
    Although the flowchart in FIG. 7 only shows two iterations, multiple attempts to conduct dialogue with the user may be conducted in order to clarify one or more of the task objectives within the spirit and scope of the invention.
  • [0199]
    While the system and method of the invention is sometime illustrated above using words, numbers or phrases, the invention may also symbols, portions of words or sounds called morphemes (or sub-morphemes known as phone-phrases). In particular, morphemes are essentially a cluster of semantically meaningful phone sequences for classifying of utterances. The representations of the utterances at the phone level are obtained as an output of a task-independent phone recognizer. Morphemes may also be formed by the input communication recognizer 150 into a lattice structure to increase coverage, as discussed in further detail above.
  • [0200]
    It is also important to note that the morphemes may be non-acoustic (i.e., made up of non-verbal sub-morphemes such as tablet strokes, gestures, body movements, etc.). Accordingly, the invention should not be limited to just acoustic morphemes and should encompass the utilization of any sub-units of any known or future method of communication for the purposes of recognition and understanding.
  • [0201]
    Furthermore, while the terms “speech”, “phrase” and “utterance”, used throughout the description below, may connote only spoken language, it is important to note in the context of this invention, “speech”, “phrase” and “utterance” may include verbal and/or non-verbal sub-units (or sub-morphemes). Therefore, “speech”, “phrase” and “utterance” may comprise non-verbal sub-units, verbal sub-units or a combination of verbal and non-verbal sub-units within the sprit and scope of this invention.
  • [0202]
    In addition, the nature of the invention described herein is such that the method and system may be used with a variety of languages and dialects. In particular, the method and system may operate on well-known, standard languages, such as English, Spanish or French, but may operate on rare, new and unknown languages and symbols in building the database. Moreover, the invention may operate on a mix of languages, such as communications partly in one language and partly in another (e.g., several English words along with or intermixed with several Spanish words).
  • [0203]
    Note that while the above-described methods of training for and detecting and extracting named entities are shown in the figures as being associated with an input communication processing system or a task classification system, these methods may have numerous other applications. In this regard, the method of training for and detecting and extracting named entities may be applied to a wide variety of automated communication systems, including customer care systems, and should not be limited to such an input communication processing system or task classification system.
  • [0204]
    As shown in FIGS. 1, 2, 4 and 5, the method of this invention may be implemented using a programmed processor. However, method can also be implemented on a general-purpose or a special purpose computer, a programmed microprocessor or microcontroller, peripheral integrated circuit elements, an application-specific integrated circuit (ASIC) or other integrated circuits, hardware/electronic logic circuits, such as a discrete element circuit, a programmable logic device, such as a PLD, PLA, FPGA, or PAL, or the like. In general, any device on which the finite state machine capable of implementing the flowcharts shown in FIGS. 3, 6 and 7 can be used to implement the recognition and understanding system functions of this invention.
  • [0205]
    While the invention has been described with reference to the above embodiments, it is to be understood that these embodiments are purely exemplary in nature. Thus, the invention is not restricted to the particular forms shown in the foregoing embodiments. Various modifications and alterations can be made thereto without departing from the spirit and scope of the invention.

Claims (29)

    What is claimed is:
  1. 1. A method of creating a named entity language model, comprising:
    recognizing input communications from a training corpus;
    parsing the training corpus;
    tagging the parsed training corpus;
    aligning the recognized training corpus with the tagged training corpus; and
    creating a named entity language model from the aligned corpus.
  2. 2. The method of claim 1, further comprising:
    transcribing the training corpus.
  3. 3. The method of claim 2, wherein the transcribing step is performed automatically.
  4. 4. The method of claim 1, wherein the training corpus includes at least one of untranscribed and transcribed speech.
  5. 5. The method of claim 1, wherein the training corpus includes communications from one or more languages.
  6. 6. The method of claim 1, further comprising:
    labeling the training corpus.
  7. 7. The method of claim 1, further comprising:
    storing the named entity language model in a database.
  8. 8. The method of claim 1, wherein the training corpus includes at least one of verbal and non-verbal speech.
  9. 9. The method of claim 8, wherein the non-verbal speech includes the use of at least one of gestures, body movements, head movements, non-responses, text, keyboard entries, keypad entries, mouse clicks, DTMF codes, pointers, stylus, cable set-top box entries, graphical user interface entries and touchscreen entries.
  10. 10. The method of claim 1, wherein the training corpus includes multimodal speech.
  11. 11. The method of claim 1, wherein the tagging step tags the training corpus with named entity tags.
  12. 12. The method of claim 1, wherein the named entity tags are at least one of context-dependent named entity tags and context-independent named entity tags.
  13. 13. The method of claim 1, wherein named entities are represented by at least one of a tag, a context and a value.
  14. 14. The method of claim 1, wherein recognizing step recognizes a lattice from the training corpus.
  15. 15. A system that creates a named entity language model, comprising:
    a recognizer that recognizes input communications from a training corpus;
    a parser that parses the training corpus;
    a tagger that tags the parsed training corpus;
    an aligner that aligns the recognized training corpus with the tagged training corpus, and creates a named entity language model from the aligned corpus.
  16. 16. The system of claim 15, further comprising:
    a transcriber that transcribes the training corpus.
  17. 17. The system of claim 16, wherein the transcriber transcribes automatically.
  18. 18. The system of claim 15, wherein the training corpus includes at least one of untranscribed and transcribed speech.
  19. 19. The system of claim 15, wherein the training corpus includes communications from one or more languages.
  20. 20. The system of claim 15, further comprising:
    a labeler that labels the training corpus.
  21. 21. The system of claim 15, further comprising:
    storing the named entity language model in a database.
  22. 22. The system of claim 15, wherein the training corpus includes at least one of verbal and non-verbal speech.
  23. 23. The system of claim 22, wherein the non-verbal speech includes the use of at least one of gestures, body movements, head movements, non-responses, text, keyboard entries, keypad entries, mouse clicks, DTMF codes, pointers, stylus, cable set-top box entries, graphical user interface entries and touchscreen entries.
  24. 24. The system of claim 15, wherein the training corpus includes multimodal speech.
  25. 25. The system of claim 15, wherein the tagging step tags the training corpus with named entity tags.
  26. 26. The system of claim 15, wherein the named entity tags are at least one of context-dependent named entity tags and context-independent named entity tags.
  27. 27. The system of claim 15, wherein named entities are represented by at least one of a tag, a context and a value.
  28. 28. The system of claim 15, wherein the recognizer recognizes a lattice.
  29. 29. A method of creating a named entity language model, comprising:
    recognizing input communications from a training corpus;
    parsing the training corpus;
    tagging the parsed training corpus;
    aligning the recognized training corpus with the tagged training corpus;
    creating a named entity language model from the aligned corpus; and
    detecting named entities in input communications using the named entity language model.
US10402976 1999-11-05 2003-04-01 Method and system for creating a named entity language model Abandoned US20030191625A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US16383899 true 1999-11-05 1999-11-05
US09690903 US6681206B1 (en) 1999-11-05 2000-10-18 Method for generating morphemes
US09690721 US7085720B1 (en) 1999-11-05 2000-10-18 Method for task classification using morphemes
US32244701 true 2001-09-17 2001-09-17
US10158082 US7286984B1 (en) 1999-11-05 2002-05-31 Method and system for automatically detecting morphemes in a task classification system using lattices
US44364203 true 2003-01-29 2003-01-29
US10402976 US20030191625A1 (en) 1999-11-05 2003-04-01 Method and system for creating a named entity language model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10402976 US20030191625A1 (en) 1999-11-05 2003-04-01 Method and system for creating a named entity language model

Related Parent Applications (3)

Application Number Title Priority Date Filing Date
US09690721 Continuation-In-Part US7085720B1 (en) 1999-11-05 2000-10-18 Method for task classification using morphemes
US09690903 Continuation-In-Part US6681206B1 (en) 1999-11-05 2000-10-18 Method for generating morphemes
US10158082 Continuation-In-Part US7286984B1 (en) 1999-11-05 2002-05-31 Method and system for automatically detecting morphemes in a task classification system using lattices

Publications (1)

Publication Number Publication Date
US20030191625A1 true true US20030191625A1 (en) 2003-10-09

Family

ID=28679197

Family Applications (1)

Application Number Title Priority Date Filing Date
US10402976 Abandoned US20030191625A1 (en) 1999-11-05 2003-04-01 Method and system for creating a named entity language model

Country Status (1)

Country Link
US (1) US20030191625A1 (en)

Cited By (79)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050075878A1 (en) * 2003-10-01 2005-04-07 International Business Machines Corporation Method, system, and apparatus for natural language mixed-initiative dialogue processing
WO2005048240A1 (en) * 2003-11-12 2005-05-26 Philips Intellectual Property & Standards Gmbh Assignment of semantic tags to phrases for grammar generation
US20050119896A1 (en) * 1999-11-12 2005-06-02 Bennett Ian M. Adjustable resource based speech recognition system
WO2005064490A1 (en) * 2003-12-31 2005-07-14 Agency For Science, Technology And Research System for recognising and classifying named entities
US20060015484A1 (en) * 2004-07-15 2006-01-19 Fuliang Weng Method and apparatus for providing proper or partial proper name recognition
US20060100856A1 (en) * 2004-11-09 2006-05-11 Samsung Electronics Co., Ltd. Method and apparatus for updating dictionary
US7050977B1 (en) 1999-11-12 2006-05-23 Phoenix Solutions, Inc. Speech-enabled server for internet website and method
US20070033004A1 (en) * 2005-07-25 2007-02-08 At And T Corp. Methods and systems for natural language understanding using human knowledge and collected data
US20070067320A1 (en) * 2005-09-20 2007-03-22 International Business Machines Corporation Detecting relationships in unstructured text
US20070100814A1 (en) * 2005-10-28 2007-05-03 Samsung Electronics Co., Ltd. Apparatus and method for detecting named entity
US7277850B1 (en) * 2003-04-02 2007-10-02 At&T Corp. System and method of word graph matrix decomposition
US7292976B1 (en) * 2003-05-29 2007-11-06 At&T Corp. Active learning process for spoken dialog systems
US20080097951A1 (en) * 2006-10-18 2008-04-24 Rakesh Gupta Scalable Knowledge Extraction
US20080133220A1 (en) * 2006-12-01 2008-06-05 Microsoft Corporation Leveraging back-off grammars for authoring context-free grammars
US20080147400A1 (en) * 2006-12-19 2008-06-19 Microsoft Corporation Adapting a language model to accommodate inputs not found in a directory assistance listing
US20080187121A1 (en) * 2007-01-29 2008-08-07 Rajeev Agarwal Method and an apparatus to disambiguate requests
US20080215329A1 (en) * 2002-03-27 2008-09-04 International Business Machines Corporation Methods and Apparatus for Generating Dialog State Conditioned Language Models
US20080243481A1 (en) * 2007-03-26 2008-10-02 Thorsten Brants Large Language Models in Machine Translation
US20080312904A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Sub-Model Generation to Improve Classification Accuracy
US20080312906A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Reclassification of Training Data to Improve Classifier Accuracy
US20080312905A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Extracting Tokens in a Natural Language Understanding Application
US20080310718A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Information Extraction in a Natural Language Understanding System
US20090023395A1 (en) * 2007-07-16 2009-01-22 Microsoft Corporation Passive interface and software configuration for portable devices
US20090055184A1 (en) * 2007-08-24 2009-02-26 Nuance Communications, Inc. Creation and Use of Application-Generic Class-Based Statistical Language Models for Automatic Speech Recognition
US20090119586A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Automatic Generation of Interactive Systems From a Formalized Description Language
US20090119104A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Switching Functionality To Control Real-Time Switching Of Modules Of A Dialog System
US20090150152A1 (en) * 2007-11-18 2009-06-11 Nice Systems Method and apparatus for fast search in call-center monitoring
US20090249182A1 (en) * 2008-03-31 2009-10-01 Iti Scotland Limited Named entity recognition methods and apparatus
US20090292528A1 (en) * 2008-05-21 2009-11-26 Denso Corporation Apparatus for providing information for vehicle
US7657424B2 (en) 1999-11-12 2010-02-02 Phoenix Solutions, Inc. System and method for processing sentence based queries
US20100106484A1 (en) * 2008-10-21 2010-04-29 Microsoft Corporation Named entity transliteration using corporate corpra
US7725321B2 (en) 1999-11-12 2010-05-25 Phoenix Solutions, Inc. Speech based query system using semantic decoding
US20100131274A1 (en) * 2008-11-26 2010-05-27 At&T Intellectual Property I, L.P. System and method for dialog modeling
US20100131260A1 (en) * 2008-11-26 2010-05-27 At&T Intellectual Property I, L.P. System and method for enriching spoken language translation with dialog acts
US20100185670A1 (en) * 2009-01-09 2010-07-22 Microsoft Corporation Mining transliterations for out-of-vocabulary query terms
US20110136541A1 (en) * 2007-07-16 2011-06-09 Microsoft Corporation Smart interface system for mobile communications devices
US20110196668A1 (en) * 2010-02-08 2011-08-11 Adacel Systems, Inc. Integrated Language Model, Related Systems and Methods
US8140567B2 (en) 2010-04-13 2012-03-20 Microsoft Corporation Measuring entity extraction complexity
US8145474B1 (en) * 2006-12-22 2012-03-27 Avaya Inc. Computer mediated natural language based communication augmented by arbitrary and flexibly assigned personality classification systems
US20120232904A1 (en) * 2011-03-10 2012-09-13 Samsung Electronics Co., Ltd. Method and apparatus for correcting a word in speech input text
US20120253801A1 (en) * 2011-03-28 2012-10-04 Epic Systems Corporation Automatic determination of and response to a topic of a conversation
US20130030810A1 (en) * 2011-07-28 2013-01-31 Tata Consultancy Services Limited Frugal method and system for creating speech corpus
US20130151501A1 (en) * 2010-11-09 2013-06-13 Tracy Wang Index-side synonym generation
US20140025379A1 (en) * 2012-07-20 2014-01-23 Interactive Intelligence, Inc. Method and System for Real-Time Keyword Spotting for Speech Analytics
US20140040313A1 (en) * 2012-08-02 2014-02-06 Sap Ag System and Method of Record Matching in a Database
US20140058724A1 (en) * 2012-07-20 2014-02-27 Veveo, Inc. Method of and System for Using Conversation State Information in a Conversational Interaction System
US20140180676A1 (en) * 2012-12-21 2014-06-26 Microsoft Corporation Named entity variations for multimodal understanding systems
US20140207457A1 (en) * 2013-01-22 2014-07-24 Interactive Intelligence, Inc. False alarm reduction in speech recognition systems using contextual information
US20140379323A1 (en) * 2013-06-20 2014-12-25 Microsoft Corporation Active learning using different knowledge sources
US8924212B1 (en) * 2005-08-26 2014-12-30 At&T Intellectual Property Ii, L.P. System and method for robust access and entry to large structured data using voice form-filling
US20150161996A1 (en) * 2013-12-10 2015-06-11 Google Inc. Techniques for discriminative dependency parsing
US20150169549A1 (en) * 2013-12-13 2015-06-18 Google Inc. Cross-lingual discriminative learning of sequence models with posterior regularization
US20150279348A1 (en) * 2014-03-25 2015-10-01 Microsoft Corporation Generating natural language outputs
US20150348565A1 (en) * 2014-05-30 2015-12-03 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US20150348543A1 (en) * 2014-06-02 2015-12-03 Robert Bosch Gmbh Speech Recognition of Partial Proper Names by Natural Language Processing
US20150348542A1 (en) * 2012-12-28 2015-12-03 Iflytek Co., Ltd. Speech recognition method and system based on user personalized information
WO2016010245A1 (en) * 2014-07-14 2016-01-21 Samsung Electronics Co., Ltd. Method and system for robust tagging of named entities in the presence of source or translation errors
US20160133251A1 (en) * 2013-05-31 2016-05-12 Longsand Limited Processing of audio data
US9460088B1 (en) * 2013-05-31 2016-10-04 Google Inc. Written-domain language modeling with decomposition
US9465833B2 (en) 2012-07-31 2016-10-11 Veveo, Inc. Disambiguating user intent in conversational interaction system for large corpus information retrieval
US9620117B1 (en) * 2006-06-27 2017-04-11 At&T Intellectual Property Ii, L.P. Learning from interactions for a spoken dialog system
US9619457B1 (en) 2014-06-06 2017-04-11 Google Inc. Techniques for automatically identifying salient entities in documents
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US20170140057A1 (en) * 2012-06-11 2017-05-18 International Business Machines Corporation System and method for automatically detecting and interactively displaying information about entities, activities, and events from multiple-modality natural language sources
US9668024B2 (en) 2014-06-30 2017-05-30 Apple Inc. Intelligent automated assistant for TV user interactions
US20170178624A1 (en) * 2015-12-17 2017-06-22 Audeme, LLC Method of facilitating construction of a voice dialog interface for an electronic system
US9747280B1 (en) * 2013-08-21 2017-08-29 Intelligent Language, LLC Date and time processing
US9854049B2 (en) 2015-01-30 2017-12-26 Rovi Guides, Inc. Systems and methods for resolving ambiguous terms in social chatter based on a user profile
US9852136B2 (en) 2014-12-23 2017-12-26 Rovi Guides, Inc. Systems and methods for determining whether a negation statement applies to a current or past query
US9864767B1 (en) 2012-04-30 2018-01-09 Google Inc. Storing term substitution information in an index
US9928296B2 (en) 2010-12-16 2018-03-27 Microsoft Technology Licensing, Llc Search lexicon expansion
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US9966060B2 (en) 2013-06-07 2018-05-08 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US9986419B2 (en) 2014-09-30 2018-05-29 Apple Inc. Social reminders
US9997156B2 (en) * 2016-12-16 2018-06-12 Audeme, LLC Method of facilitating construction of a voice dialog interface for an electronic system

Citations (67)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6183085B2 (en) *
US477600A (en) * 1892-06-21 Wheeled sounding-toy
US4827521A (en) * 1986-03-27 1989-05-02 International Business Machines Corporation Training of markov models used in a speech recognition system
US4866778A (en) * 1986-08-11 1989-09-12 Dragon Systems, Inc. Interactive speech recognition apparatus
US4882759A (en) * 1986-04-18 1989-11-21 International Business Machines Corporation Synthesizing word baseforms used in speech recognition
US4903778A (en) * 1985-10-28 1990-02-27 Brouwer Turf Equipment Limited Method for rolling sod with grass side out
US5005127A (en) * 1987-10-26 1991-04-02 Sharp Kabushiki Kaisha System including means to translate only selected portions of an input sentence and means to translate selected portions according to distinct rules
US5029214A (en) * 1986-08-11 1991-07-02 Hollander James F Electronic speech control apparatus and methods
US5033088A (en) * 1988-06-06 1991-07-16 Voice Processing Corp. Method and apparatus for effectively receiving voice input to a voice recognition system
US5062047A (en) * 1988-04-30 1991-10-29 Sharp Kabushiki Kaisha Translation method and apparatus using optical character reader
US5099425A (en) * 1988-12-13 1992-03-24 Matsushita Electric Industrial Co., Ltd. Method and apparatus for analyzing the semantics and syntax of a sentence or a phrase
US5210689A (en) * 1990-12-28 1993-05-11 Semantic Compaction Systems System and method for automatically selecting among a plurality of input modes
US5212730A (en) * 1991-07-01 1993-05-18 Texas Instruments Incorporated Voice recognition of proper names using text-derived recognition models
US5297039A (en) * 1991-01-30 1994-03-22 Mitsubishi Denki Kabushiki Kaisha Text search system for locating on the basis of keyword matching and keyword relationship matching
US5323316A (en) * 1991-02-01 1994-06-21 Wang Laboratories, Inc. Morphological analyzer
US5333275A (en) * 1992-06-23 1994-07-26 Wheatley Barbara J System and method for time aligning speech
US5337232A (en) * 1989-03-02 1994-08-09 Nec Corporation Morpheme analysis device
US5357596A (en) * 1991-11-18 1994-10-18 Kabushiki Kaisha Toshiba Speech dialogue system for facilitating improved human-computer interaction
US5375164A (en) * 1992-05-26 1994-12-20 At&T Corp. Multiple language capability in an interactive system
US5384892A (en) * 1992-12-31 1995-01-24 Apple Computer, Inc. Dynamic language model for speech recognition
US5390272A (en) * 1993-08-31 1995-02-14 Amphenol Corporation Fiber optic cable connector with strain relief boot
US5434906A (en) * 1993-09-13 1995-07-18 Robinson; Michael J. Method and apparatus for processing an incoming call in a communication system
US5457768A (en) * 1991-08-13 1995-10-10 Kabushiki Kaisha Toshiba Speech recognition apparatus using syntactic and semantic analysis
US5500920A (en) * 1993-09-23 1996-03-19 Xerox Corporation Semantic co-occurrence filtering for speech recognition and signal transcription applications
US5509104A (en) * 1989-05-17 1996-04-16 At&T Corp. Speech recognition employing key word modeling and non-key word modeling
US5544050A (en) * 1992-09-03 1996-08-06 Hitachi, Ltd. Sign language learning system and method
US5619410A (en) * 1993-03-29 1997-04-08 Nec Corporation Keyword extraction apparatus for Japanese texts
US5642519A (en) * 1994-04-29 1997-06-24 Sun Microsystems, Inc. Speech interpreter with a unified grammer compiler
US5651095A (en) * 1993-10-04 1997-07-22 British Telecommunications Public Limited Company Speech synthesis using word parser with knowledge base having dictionary of morphemes with binding properties and combining rules to identify input word class
US5659731A (en) * 1995-06-19 1997-08-19 Dun & Bradstreet, Inc. Method for rating a match for a given entity found in a list of entities
US5666400A (en) * 1994-07-07 1997-09-09 Bell Atlantic Network Services, Inc. Intelligent recognition
US5675707A (en) * 1995-09-15 1997-10-07 At&T Automated call router system and method
US5719921A (en) * 1996-02-29 1998-02-17 Nynex Science & Technology Methods and apparatus for activating telephone services in response to speech
US5724594A (en) * 1994-02-10 1998-03-03 Microsoft Corporation Method and system for automatically identifying morphological information from a machine-readable dictionary
US5724481A (en) * 1995-03-30 1998-03-03 Lucent Technologies Inc. Method for automatic speech recognition of arbitrary spoken words
US5752230A (en) * 1996-08-20 1998-05-12 Ncr Corporation Method and apparatus for identifying names with a speech recognition program
US5794193A (en) * 1995-09-15 1998-08-11 Lucent Technologies Inc. Automated phrase generation
US5832480A (en) * 1996-07-12 1998-11-03 International Business Machines Corporation Using canonical forms to develop a dictionary of names in a text
US5839106A (en) * 1996-12-17 1998-11-17 Apple Computer, Inc. Large-vocabulary speech recognition using an integrated syntactic and semantic statistical language model
US5860063A (en) * 1997-07-11 1999-01-12 At&T Corp Automated meaningful phrase clustering
US5878390A (en) * 1996-12-20 1999-03-02 Atr Interpreting Telecommunications Research Laboratories Speech recognition apparatus equipped with means for removing erroneous candidate of speech recognition
US5905774A (en) * 1996-11-19 1999-05-18 Stentor Resource Centre, Inc. Method and system of accessing and operating a voice message system
US5918222A (en) * 1995-03-17 1999-06-29 Kabushiki Kaisha Toshiba Information disclosing apparatus and multi-modal information input/output system
US5960447A (en) * 1995-11-13 1999-09-28 Holt; Douglas Word tagging and editing system for speech recognition
US5983180A (en) * 1997-10-23 1999-11-09 Softsound Limited Recognition of sequential data using finite state sequence models organized in a tree structure
US6006186A (en) * 1997-10-16 1999-12-21 Sony Corporation Method and apparatus for a parameter sharing speech recognition system
US6021384A (en) * 1997-10-29 2000-02-01 At&T Corp. Automatic generation of superwords
US6023673A (en) * 1997-06-04 2000-02-08 International Business Machines Corporation Hierarchical labeler in a speech recognition system
US6044337A (en) * 1997-10-29 2000-03-28 At&T Corp Selection of superwords based on criteria relevant to both speech recognition and understanding
US6064957A (en) * 1997-08-15 2000-05-16 General Electric Company Improving speech recognition through text-based linguistic post-processing
US6173261B1 (en) * 1998-09-30 2001-01-09 At&T Corp Grammar fragment acquisition using syntactic and semantic clustering
US6183085B1 (en) * 1998-02-11 2001-02-06 Sheri L. Roggy Apparatus and method for converting a standard non-rotatable diagnostic lens into a rotatable diagnostic lens for evaluation of various portions of the eye
US6183110B1 (en) * 1999-02-09 2001-02-06 Ching Hsi Chang U-shaped trough frame for hanging Christmas light bulb series
US6208964B1 (en) * 1998-08-31 2001-03-27 Nortel Networks Limited Method and apparatus for providing unsupervised adaptation of transcriptions
US6233553B1 (en) * 1998-09-04 2001-05-15 Matsushita Electric Industrial Co., Ltd. Method and system for automatically determining phonetic transcriptions associated with spelled words
US6243683B1 (en) * 1998-12-29 2001-06-05 Intel Corporation Video control of speech recognition
US6308156B1 (en) * 1996-03-14 2001-10-23 G Data Software Gmbh Microsegment-based speech-synthesis process
US6311152B1 (en) * 1999-04-08 2001-10-30 Kent Ridge Digital Labs System for chinese tokenization and named entity recognition
US6317707B1 (en) * 1998-12-07 2001-11-13 At&T Corp. Automatic clustering of tokens from a corpus for grammar acquisition
US6397179B2 (en) * 1997-12-24 2002-05-28 Nortel Networks Limited Search optimization system and method for continuous speech recognition
US6681206B1 (en) * 1999-11-05 2004-01-20 At&T Corporation Method for generating morphemes
US20040199375A1 (en) * 1999-05-28 2004-10-07 Farzad Ehsani Phrase-based dialogue modeling with particular application to creating a recognition grammar for a voice-controlled user interface
US6895377B2 (en) * 2000-03-24 2005-05-17 Eliza Corporation Phonetic data processing system and method
US6941266B1 (en) * 2000-11-15 2005-09-06 At&T Corp. Method and system for predicting problematic dialog situations in a task classification system
US6961954B1 (en) * 1997-10-27 2005-11-01 The Mitre Corporation Automated segmentation, information extraction, summarization, and presentation of broadcast news
US7082578B1 (en) * 1997-08-29 2006-07-25 Xerox Corporation Computer user interface using a physical manipulatory grammar
US7085720B1 (en) * 1999-11-05 2006-08-01 At & T Corp. Method for task classification using morphemes

Patent Citations (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6183110B2 (en) *
US6173261B2 (en) *
US477600A (en) * 1892-06-21 Wheeled sounding-toy
US6183085B2 (en) *
US4903778A (en) * 1985-10-28 1990-02-27 Brouwer Turf Equipment Limited Method for rolling sod with grass side out
US4827521A (en) * 1986-03-27 1989-05-02 International Business Machines Corporation Training of markov models used in a speech recognition system
US4882759A (en) * 1986-04-18 1989-11-21 International Business Machines Corporation Synthesizing word baseforms used in speech recognition
US4866778A (en) * 1986-08-11 1989-09-12 Dragon Systems, Inc. Interactive speech recognition apparatus
US5029214A (en) * 1986-08-11 1991-07-02 Hollander James F Electronic speech control apparatus and methods
US5005127A (en) * 1987-10-26 1991-04-02 Sharp Kabushiki Kaisha System including means to translate only selected portions of an input sentence and means to translate selected portions according to distinct rules
US5062047A (en) * 1988-04-30 1991-10-29 Sharp Kabushiki Kaisha Translation method and apparatus using optical character reader
US5033088A (en) * 1988-06-06 1991-07-16 Voice Processing Corp. Method and apparatus for effectively receiving voice input to a voice recognition system
US5099425A (en) * 1988-12-13 1992-03-24 Matsushita Electric Industrial Co., Ltd. Method and apparatus for analyzing the semantics and syntax of a sentence or a phrase
US5337232A (en) * 1989-03-02 1994-08-09 Nec Corporation Morpheme analysis device
US5509104A (en) * 1989-05-17 1996-04-16 At&T Corp. Speech recognition employing key word modeling and non-key word modeling
US5210689A (en) * 1990-12-28 1993-05-11 Semantic Compaction Systems System and method for automatically selecting among a plurality of input modes
US5297039A (en) * 1991-01-30 1994-03-22 Mitsubishi Denki Kabushiki Kaisha Text search system for locating on the basis of keyword matching and keyword relationship matching
US5323316A (en) * 1991-02-01 1994-06-21 Wang Laboratories, Inc. Morphological analyzer
US5212730A (en) * 1991-07-01 1993-05-18 Texas Instruments Incorporated Voice recognition of proper names using text-derived recognition models
US5457768A (en) * 1991-08-13 1995-10-10 Kabushiki Kaisha Toshiba Speech recognition apparatus using syntactic and semantic analysis
US5357596A (en) * 1991-11-18 1994-10-18 Kabushiki Kaisha Toshiba Speech dialogue system for facilitating improved human-computer interaction
US5375164A (en) * 1992-05-26 1994-12-20 At&T Corp. Multiple language capability in an interactive system
US5333275A (en) * 1992-06-23 1994-07-26 Wheatley Barbara J System and method for time aligning speech
US5544050A (en) * 1992-09-03 1996-08-06 Hitachi, Ltd. Sign language learning system and method
US5384892A (en) * 1992-12-31 1995-01-24 Apple Computer, Inc. Dynamic language model for speech recognition
US5619410A (en) * 1993-03-29 1997-04-08 Nec Corporation Keyword extraction apparatus for Japanese texts
US5390272A (en) * 1993-08-31 1995-02-14 Amphenol Corporation Fiber optic cable connector with strain relief boot
US5434906A (en) * 1993-09-13 1995-07-18 Robinson; Michael J. Method and apparatus for processing an incoming call in a communication system
US5500920A (en) * 1993-09-23 1996-03-19 Xerox Corporation Semantic co-occurrence filtering for speech recognition and signal transcription applications
US5651095A (en) * 1993-10-04 1997-07-22 British Telecommunications Public Limited Company Speech synthesis using word parser with knowledge base having dictionary of morphemes with binding properties and combining rules to identify input word class
US5724594A (en) * 1994-02-10 1998-03-03 Microsoft Corporation Method and system for automatically identifying morphological information from a machine-readable dictionary
US5642519A (en) * 1994-04-29 1997-06-24 Sun Microsystems, Inc. Speech interpreter with a unified grammer compiler
US5666400A (en) * 1994-07-07 1997-09-09 Bell Atlantic Network Services, Inc. Intelligent recognition
US5918222A (en) * 1995-03-17 1999-06-29 Kabushiki Kaisha Toshiba Information disclosing apparatus and multi-modal information input/output system
US5724481A (en) * 1995-03-30 1998-03-03 Lucent Technologies Inc. Method for automatic speech recognition of arbitrary spoken words
US5659731A (en) * 1995-06-19 1997-08-19 Dun & Bradstreet, Inc. Method for rating a match for a given entity found in a list of entities
US5675707A (en) * 1995-09-15 1997-10-07 At&T Automated call router system and method
US5794193A (en) * 1995-09-15 1998-08-11 Lucent Technologies Inc. Automated phrase generation
US5960447A (en) * 1995-11-13 1999-09-28 Holt; Douglas Word tagging and editing system for speech recognition
US5719921A (en) * 1996-02-29 1998-02-17 Nynex Science & Technology Methods and apparatus for activating telephone services in response to speech
US6308156B1 (en) * 1996-03-14 2001-10-23 G Data Software Gmbh Microsegment-based speech-synthesis process
US5832480A (en) * 1996-07-12 1998-11-03 International Business Machines Corporation Using canonical forms to develop a dictionary of names in a text
US5752230A (en) * 1996-08-20 1998-05-12 Ncr Corporation Method and apparatus for identifying names with a speech recognition program
US5905774A (en) * 1996-11-19 1999-05-18 Stentor Resource Centre, Inc. Method and system of accessing and operating a voice message system
US5839106A (en) * 1996-12-17 1998-11-17 Apple Computer, Inc. Large-vocabulary speech recognition using an integrated syntactic and semantic statistical language model
US5878390A (en) * 1996-12-20 1999-03-02 Atr Interpreting Telecommunications Research Laboratories Speech recognition apparatus equipped with means for removing erroneous candidate of speech recognition
US6023673A (en) * 1997-06-04 2000-02-08 International Business Machines Corporation Hierarchical labeler in a speech recognition system
US5860063A (en) * 1997-07-11 1999-01-12 At&T Corp Automated meaningful phrase clustering
US6064957A (en) * 1997-08-15 2000-05-16 General Electric Company Improving speech recognition through text-based linguistic post-processing
US7082578B1 (en) * 1997-08-29 2006-07-25 Xerox Corporation Computer user interface using a physical manipulatory grammar
US6006186A (en) * 1997-10-16 1999-12-21 Sony Corporation Method and apparatus for a parameter sharing speech recognition system
US5983180A (en) * 1997-10-23 1999-11-09 Softsound Limited Recognition of sequential data using finite state sequence models organized in a tree structure
US6961954B1 (en) * 1997-10-27 2005-11-01 The Mitre Corporation Automated segmentation, information extraction, summarization, and presentation of broadcast news
US6044337A (en) * 1997-10-29 2000-03-28 At&T Corp Selection of superwords based on criteria relevant to both speech recognition and understanding
US6021384A (en) * 1997-10-29 2000-02-01 At&T Corp. Automatic generation of superwords
US6397179B2 (en) * 1997-12-24 2002-05-28 Nortel Networks Limited Search optimization system and method for continuous speech recognition
US6183085B1 (en) * 1998-02-11 2001-02-06 Sheri L. Roggy Apparatus and method for converting a standard non-rotatable diagnostic lens into a rotatable diagnostic lens for evaluation of various portions of the eye
US6208964B1 (en) * 1998-08-31 2001-03-27 Nortel Networks Limited Method and apparatus for providing unsupervised adaptation of transcriptions
US6233553B1 (en) * 1998-09-04 2001-05-15 Matsushita Electric Industrial Co., Ltd. Method and system for automatically determining phonetic transcriptions associated with spelled words
US6173261B1 (en) * 1998-09-30 2001-01-09 At&T Corp Grammar fragment acquisition using syntactic and semantic clustering
US6317707B1 (en) * 1998-12-07 2001-11-13 At&T Corp. Automatic clustering of tokens from a corpus for grammar acquisition
US6243683B1 (en) * 1998-12-29 2001-06-05 Intel Corporation Video control of speech recognition
US6183110B1 (en) * 1999-02-09 2001-02-06 Ching Hsi Chang U-shaped trough frame for hanging Christmas light bulb series
US6311152B1 (en) * 1999-04-08 2001-10-30 Kent Ridge Digital Labs System for chinese tokenization and named entity recognition
US20040199375A1 (en) * 1999-05-28 2004-10-07 Farzad Ehsani Phrase-based dialogue modeling with particular application to creating a recognition grammar for a voice-controlled user interface
US7085720B1 (en) * 1999-11-05 2006-08-01 At & T Corp. Method for task classification using morphemes
US6681206B1 (en) * 1999-11-05 2004-01-20 At&T Corporation Method for generating morphemes
US6895377B2 (en) * 2000-03-24 2005-05-17 Eliza Corporation Phonetic data processing system and method
US6941266B1 (en) * 2000-11-15 2005-09-06 At&T Corp. Method and system for predicting problematic dialog situations in a task classification system

Cited By (166)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7657424B2 (en) 1999-11-12 2010-02-02 Phoenix Solutions, Inc. System and method for processing sentence based queries
US9076448B2 (en) 1999-11-12 2015-07-07 Nuance Communications, Inc. Distributed real time speech recognition system
US20050119896A1 (en) * 1999-11-12 2005-06-02 Bennett Ian M. Adjustable resource based speech recognition system
US7831426B2 (en) 1999-11-12 2010-11-09 Phoenix Solutions, Inc. Network based interactive speech recognition system
US7912702B2 (en) 1999-11-12 2011-03-22 Phoenix Solutions, Inc. Statistical language model trained with semantic variants
US9190063B2 (en) 1999-11-12 2015-11-17 Nuance Communications, Inc. Multi-language speech recognition system
US7050977B1 (en) 1999-11-12 2006-05-23 Phoenix Solutions, Inc. Speech-enabled server for internet website and method
US7729904B2 (en) 1999-11-12 2010-06-01 Phoenix Solutions, Inc. Partial speech processing device and method for use in distributed systems
US7702508B2 (en) 1999-11-12 2010-04-20 Phoenix Solutions, Inc. System and method for natural language processing of query answers
US8352277B2 (en) 1999-11-12 2013-01-08 Phoenix Solutions, Inc. Method of interacting through speech with a web-connected server
US7725320B2 (en) 1999-11-12 2010-05-25 Phoenix Solutions, Inc. Internet based speech recognition system with dynamic grammars
US7725307B2 (en) 1999-11-12 2010-05-25 Phoenix Solutions, Inc. Query engine for processing voice based queries including semantic decoding
US7672841B2 (en) 1999-11-12 2010-03-02 Phoenix Solutions, Inc. Method for processing speech data for a distributed recognition system
US8229734B2 (en) 1999-11-12 2012-07-24 Phoenix Solutions, Inc. Semantic decoding of user queries
US7725321B2 (en) 1999-11-12 2010-05-25 Phoenix Solutions, Inc. Speech based query system using semantic decoding
US7698131B2 (en) 1999-11-12 2010-04-13 Phoenix Solutions, Inc. Speech recognition system for client devices having differing computing capabilities
US7647225B2 (en) 1999-11-12 2010-01-12 Phoenix Solutions, Inc. Adjustable resource based speech recognition system
US8762152B2 (en) 1999-11-12 2014-06-24 Nuance Communications, Inc. Speech recognition system interactive agent
US7873519B2 (en) * 1999-11-12 2011-01-18 Phoenix Solutions, Inc. Natural language speech lattice containing semantic variants
US7853449B2 (en) * 2002-03-27 2010-12-14 Nuance Communications, Inc. Methods and apparatus for generating dialog state conditioned language models
US20080215329A1 (en) * 2002-03-27 2008-09-04 International Business Machines Corporation Methods and Apparatus for Generating Dialog State Conditioned Language Models
US7603272B1 (en) * 2003-04-02 2009-10-13 At&T Intellectual Property Ii, L.P. System and method of word graph matrix decomposition
US7277850B1 (en) * 2003-04-02 2007-10-02 At&T Corp. System and method of word graph matrix decomposition
US7292976B1 (en) * 2003-05-29 2007-11-06 At&T Corp. Active learning process for spoken dialog systems
US7562014B1 (en) 2003-05-29 2009-07-14 At&T Intellectual Property Ii, L.P. Active learning process for spoken dialog systems
US7386440B2 (en) * 2003-10-01 2008-06-10 International Business Machines Corporation Method, system, and apparatus for natural language mixed-initiative dialogue processing
US7974835B2 (en) 2003-10-01 2011-07-05 Nuance Communications, Inc. Method, system, and apparatus for natural language mixed-initiative dialogue processing
US20080300865A1 (en) * 2003-10-01 2008-12-04 Internatiional Business Machines Corporation Method, system, and apparatus for natural language mixed-initiative dialogue processing
US20050075878A1 (en) * 2003-10-01 2005-04-07 International Business Machines Corporation Method, system, and apparatus for natural language mixed-initiative dialogue processing
WO2005048240A1 (en) * 2003-11-12 2005-05-26 Philips Intellectual Property & Standards Gmbh Assignment of semantic tags to phrases for grammar generation
WO2005064490A1 (en) * 2003-12-31 2005-07-14 Agency For Science, Technology And Research System for recognising and classifying named entities
US20070067280A1 (en) * 2003-12-31 2007-03-22 Agency For Science, Technology And Research System for recognising and classifying named entities
GB2424977A (en) * 2003-12-31 2006-10-11 Agency Science Tech & Res System For Recognising And Classifying Named Entities
US7865356B2 (en) * 2004-07-15 2011-01-04 Robert Bosch Gmbh Method and apparatus for providing proper or partial proper name recognition
US20060015484A1 (en) * 2004-07-15 2006-01-19 Fuliang Weng Method and apparatus for providing proper or partial proper name recognition
US20060100856A1 (en) * 2004-11-09 2006-05-11 Samsung Electronics Co., Ltd. Method and apparatus for updating dictionary
US8311807B2 (en) * 2004-11-09 2012-11-13 Samsung Electronics Co., Ltd. Periodically extracting and evaluating frequency of occurrence data of unregistered terms in a document for updating a dictionary
US8433558B2 (en) * 2005-07-25 2013-04-30 At&T Intellectual Property Ii, L.P. Methods and systems for natural language understanding using human knowledge and collected data
US9792904B2 (en) 2005-07-25 2017-10-17 Nuance Communications, Inc. Methods and systems for natural language understanding using human knowledge and collected data
US20070033004A1 (en) * 2005-07-25 2007-02-08 At And T Corp. Methods and systems for natural language understanding using human knowledge and collected data
US8798990B2 (en) 2005-07-25 2014-08-05 At&T Intellectual Property Ii, L.P. Methods and systems for natural language understanding using human knowledge and collected data
US9824682B2 (en) 2005-08-26 2017-11-21 Nuance Communications, Inc. System and method for robust access and entry to large structured data using voice form-filling
US8924212B1 (en) * 2005-08-26 2014-12-30 At&T Intellectual Property Ii, L.P. System and method for robust access and entry to large structured data using voice form-filling
US9165554B2 (en) 2005-08-26 2015-10-20 At&T Intellectual Property Ii, L.P. System and method for robust access and entry to large structured data using voice form-filling
US20080177740A1 (en) * 2005-09-20 2008-07-24 International Business Machines Corporation Detecting relationships in unstructured text
US20070067320A1 (en) * 2005-09-20 2007-03-22 International Business Machines Corporation Detecting relationships in unstructured text
US8001144B2 (en) 2005-09-20 2011-08-16 International Business Machines Corporation Detecting relationships in unstructured text
US20070100814A1 (en) * 2005-10-28 2007-05-03 Samsung Electronics Co., Ltd. Apparatus and method for detecting named entity
US8655646B2 (en) * 2005-10-28 2014-02-18 Samsung Electronics Co., Ltd. Apparatus and method for detecting named entity
US9620117B1 (en) * 2006-06-27 2017-04-11 At&T Intellectual Property Ii, L.P. Learning from interactions for a spoken dialog system
US20080097951A1 (en) * 2006-10-18 2008-04-24 Rakesh Gupta Scalable Knowledge Extraction
US8738359B2 (en) * 2006-10-18 2014-05-27 Honda Motor Co., Ltd. Scalable knowledge extraction
US8108205B2 (en) * 2006-12-01 2012-01-31 Microsoft Corporation Leveraging back-off grammars for authoring context-free grammars
US20120095752A1 (en) * 2006-12-01 2012-04-19 Microsoft Corporation Leveraging back-off grammars for authoring context-free grammars
US20080133220A1 (en) * 2006-12-01 2008-06-05 Microsoft Corporation Leveraging back-off grammars for authoring context-free grammars
US8862468B2 (en) * 2006-12-01 2014-10-14 Microsoft Corporation Leveraging back-off grammars for authoring context-free grammars
US7912707B2 (en) * 2006-12-19 2011-03-22 Microsoft Corporation Adapting a language model to accommodate inputs not found in a directory assistance listing
US20110137639A1 (en) * 2006-12-19 2011-06-09 Microsoft Corporation Adapting a language model to accommodate inputs not found in a directory assistance listing
US20080147400A1 (en) * 2006-12-19 2008-06-19 Microsoft Corporation Adapting a language model to accommodate inputs not found in a directory assistance listing
US8285542B2 (en) 2006-12-19 2012-10-09 Microsoft Corporation Adapting a language model to accommodate inputs not found in a directory assistance listing
US8145474B1 (en) * 2006-12-22 2012-03-27 Avaya Inc. Computer mediated natural language based communication augmented by arbitrary and flexibly assigned personality classification systems
US9131050B2 (en) 2007-01-29 2015-09-08 Nuance Communications, Inc. Method and an apparatus to disambiguate requests
US20080187121A1 (en) * 2007-01-29 2008-08-07 Rajeev Agarwal Method and an apparatus to disambiguate requests
US8175248B2 (en) 2007-01-29 2012-05-08 Nuance Communications, Inc. Method and an apparatus to disambiguate requests
WO2008097490A3 (en) * 2007-02-02 2008-10-16 Nuance Communications Inc A method and an apparatus to disambiguate requests
WO2008097490A2 (en) * 2007-02-02 2008-08-14 Nuance Communications, Inc. A method and an apparatus to disambiguate requests
US20080243481A1 (en) * 2007-03-26 2008-10-02 Thorsten Brants Large Language Models in Machine Translation
US8812291B2 (en) * 2007-03-26 2014-08-19 Google Inc. Large language models in machine translation
US20130346059A1 (en) * 2007-03-26 2013-12-26 Google Inc. Large language models in machine translation
US8332207B2 (en) * 2007-03-26 2012-12-11 Google Inc. Large language models in machine translation
US9454525B2 (en) 2007-06-18 2016-09-27 International Business Machines Corporation Information extraction in a natural language understanding system
US20080310718A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Information Extraction in a Natural Language Understanding System
US8521511B2 (en) 2007-06-18 2013-08-27 International Business Machines Corporation Information extraction in a natural language understanding system
US8285539B2 (en) * 2007-06-18 2012-10-09 International Business Machines Corporation Extracting tokens in a natural language understanding application
US9342588B2 (en) * 2007-06-18 2016-05-17 International Business Machines Corporation Reclassification of training data to improve classifier accuracy
US9767092B2 (en) 2007-06-18 2017-09-19 International Business Machines Corporation Information extraction in a natural language understanding system
US20080312904A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Sub-Model Generation to Improve Classification Accuracy
US9058319B2 (en) 2007-06-18 2015-06-16 International Business Machines Corporation Sub-model generation to improve classification accuracy
US20080312906A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Reclassification of Training Data to Improve Classifier Accuracy
US20080312905A1 (en) * 2007-06-18 2008-12-18 International Business Machines Corporation Extracting Tokens in a Natural Language Understanding Application
US8185155B2 (en) 2007-07-16 2012-05-22 Microsoft Corporation Smart interface system for mobile communications devices
US20090023395A1 (en) * 2007-07-16 2009-01-22 Microsoft Corporation Passive interface and software configuration for portable devices
US20110136541A1 (en) * 2007-07-16 2011-06-09 Microsoft Corporation Smart interface system for mobile communications devices
US8165633B2 (en) 2007-07-16 2012-04-24 Microsoft Corporation Passive interface and software configuration for portable devices
US8135578B2 (en) * 2007-08-24 2012-03-13 Nuance Communications, Inc. Creation and use of application-generic class-based statistical language models for automatic speech recognition
US20090055184A1 (en) * 2007-08-24 2009-02-26 Nuance Communications, Inc. Creation and Use of Application-Generic Class-Based Statistical Language Models for Automatic Speech Recognition
US20090119586A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Automatic Generation of Interactive Systems From a Formalized Description Language
US20090119104A1 (en) * 2007-11-07 2009-05-07 Robert Bosch Gmbh Switching Functionality To Control Real-Time Switching Of Modules Of A Dialog System
US8001469B2 (en) 2007-11-07 2011-08-16 Robert Bosch Gmbh Automatic generation of interactive systems from a formalized description language
US8155959B2 (en) * 2007-11-07 2012-04-10 Robert Bosch Gmbh Dialog system for human agent to correct abnormal output
US7788095B2 (en) * 2007-11-18 2010-08-31 Nice Systems, Ltd. Method and apparatus for fast search in call-center monitoring
US20090150152A1 (en) * 2007-11-18 2009-06-11 Nice Systems Method and apparatus for fast search in call-center monitoring
US20090249182A1 (en) * 2008-03-31 2009-10-01 Iti Scotland Limited Named entity recognition methods and apparatus
US9626955B2 (en) 2008-04-05 2017-04-18 Apple Inc. Intelligent text-to-speech conversion
US9865248B2 (en) 2008-04-05 2018-01-09 Apple Inc. Intelligent text-to-speech conversion
US20090292528A1 (en) * 2008-05-21 2009-11-26 Denso Corporation Apparatus for providing information for vehicle
US8185380B2 (en) * 2008-05-21 2012-05-22 Denso Corporation Apparatus for providing information for vehicle
US8560298B2 (en) * 2008-10-21 2013-10-15 Microsoft Corporation Named entity transliteration using comparable CORPRA
US20100106484A1 (en) * 2008-10-21 2010-04-29 Microsoft Corporation Named entity transliteration using corporate corpra
US8374881B2 (en) * 2008-11-26 2013-02-12 At&T Intellectual Property I, L.P. System and method for enriching spoken language translation with dialog acts
US20100131274A1 (en) * 2008-11-26 2010-05-27 At&T Intellectual Property I, L.P. System and method for dialog modeling
US20100131260A1 (en) * 2008-11-26 2010-05-27 At&T Intellectual Property I, L.P. System and method for enriching spoken language translation with dialog acts
US9129601B2 (en) * 2008-11-26 2015-09-08 At&T Intellectual Property I, L.P. System and method for dialog modeling
US9972307B2 (en) * 2008-11-26 2018-05-15 At&T Intellectual Property I, L.P. System and method for dialog modeling
US9501470B2 (en) 2008-11-26 2016-11-22 At&T Intellectual Property I, L.P. System and method for enriching spoken language translation with dialog acts
US20150379984A1 (en) * 2008-11-26 2015-12-31 At&T Intellectual Property I, L.P. System and method for dialog modeling
US20100185670A1 (en) * 2009-01-09 2010-07-22 Microsoft Corporation Mining transliterations for out-of-vocabulary query terms
US8332205B2 (en) 2009-01-09 2012-12-11 Microsoft Corporation Mining transliterations for out-of-vocabulary query terms
US20110196668A1 (en) * 2010-02-08 2011-08-11 Adacel Systems, Inc. Integrated Language Model, Related Systems and Methods
US8515734B2 (en) * 2010-02-08 2013-08-20 Adacel Systems, Inc. Integrated language model, related systems and methods
US9633660B2 (en) 2010-02-25 2017-04-25 Apple Inc. User profiling for voice input processing
US8140567B2 (en) 2010-04-13 2012-03-20 Microsoft Corporation Measuring entity extraction complexity
US20120143869A1 (en) * 2010-04-13 2012-06-07 Microsoft Corporation Measuring entity extraction complexity
US9128933B2 (en) * 2010-04-13 2015-09-08 Microsoft Technology Licensing, Llc Measuring entity extraction complexity
US9286405B2 (en) * 2010-11-09 2016-03-15 Google Inc. Index-side synonym generation
US20130151501A1 (en) * 2010-11-09 2013-06-13 Tracy Wang Index-side synonym generation
US9928296B2 (en) 2010-12-16 2018-03-27 Microsoft Technology Licensing, Llc Search lexicon expansion
US20120232904A1 (en) * 2011-03-10 2012-09-13 Samsung Electronics Co., Ltd. Method and apparatus for correcting a word in speech input text
US9190056B2 (en) * 2011-03-10 2015-11-17 Samsung Electronics Co., Ltd. Method and apparatus for correcting a word in speech input text
US20120253801A1 (en) * 2011-03-28 2012-10-04 Epic Systems Corporation Automatic determination of and response to a topic of a conversation
US8756064B2 (en) * 2011-07-28 2014-06-17 Tata Consultancy Services Limited Method and system for creating frugal speech corpus using internet resources and conventional speech corpus
US20130030810A1 (en) * 2011-07-28 2013-01-31 Tata Consultancy Services Limited Frugal method and system for creating speech corpus
US9864767B1 (en) 2012-04-30 2018-01-09 Google Inc. Storing term substitution information in an index
US9953088B2 (en) 2012-05-14 2018-04-24 Apple Inc. Crowd sourcing information to fulfill user requests
US20170140057A1 (en) * 2012-06-11 2017-05-18 International Business Machines Corporation System and method for automatically detecting and interactively displaying information about entities, activities, and events from multiple-modality natural language sources
US9672815B2 (en) * 2012-07-20 2017-06-06 Interactive Intelligence Group, Inc. Method and system for real-time keyword spotting for speech analytics
US9183183B2 (en) 2012-07-20 2015-11-10 Veveo, Inc. Method of and system for inferring user intent in search input in a conversational interaction system
US9424233B2 (en) 2012-07-20 2016-08-23 Veveo, Inc. Method of and system for inferring user intent in search input in a conversational interaction system
US20140058724A1 (en) * 2012-07-20 2014-02-27 Veveo, Inc. Method of and System for Using Conversation State Information in a Conversational Interaction System
US9477643B2 (en) * 2012-07-20 2016-10-25 Veveo, Inc. Method of and system for using conversation state information in a conversational interaction system
US20140025379A1 (en) * 2012-07-20 2014-01-23 Interactive Intelligence, Inc. Method and System for Real-Time Keyword Spotting for Speech Analytics
US9465833B2 (en) 2012-07-31 2016-10-11 Veveo, Inc. Disambiguating user intent in conversational interaction system for large corpus information retrieval
US20140040313A1 (en) * 2012-08-02 2014-02-06 Sap Ag System and Method of Record Matching in a Database
US9218372B2 (en) * 2012-08-02 2015-12-22 Sap Se System and method of record matching in a database
US9971774B2 (en) 2012-09-19 2018-05-15 Apple Inc. Voice-based media searching
US9916301B2 (en) * 2012-12-21 2018-03-13 Microsoft Technology Licensing, Llc Named entity variations for multimodal understanding systems
US20140180676A1 (en) * 2012-12-21 2014-06-26 Microsoft Corporation Named entity variations for multimodal understanding systems
US20150348542A1 (en) * 2012-12-28 2015-12-03 Iflytek Co., Ltd. Speech recognition method and system based on user personalized information
US9564127B2 (en) * 2012-12-28 2017-02-07 Iflytek Co., Ltd. Speech recognition method and system based on user personalized information
US9646605B2 (en) * 2013-01-22 2017-05-09 Interactive Intelligence Group, Inc. False alarm reduction in speech recognition systems using contextual information
US20140207457A1 (en) * 2013-01-22 2014-07-24 Interactive Intelligence, Inc. False alarm reduction in speech recognition systems using contextual information
US9460088B1 (en) * 2013-05-31 2016-10-04 Google Inc. Written-domain language modeling with decomposition
US20160133251A1 (en) * 2013-05-31 2016-05-12 Longsand Limited Processing of audio data
US9966060B2 (en) 2013-06-07 2018-05-08 Apple Inc. System and method for user-specified pronunciation of words for speech synthesis and recognition
US20140379323A1 (en) * 2013-06-20 2014-12-25 Microsoft Corporation Active learning using different knowledge sources
US9747280B1 (en) * 2013-08-21 2017-08-29 Intelligent Language, LLC Date and time processing
US20150161996A1 (en) * 2013-12-10 2015-06-11 Google Inc. Techniques for discriminative dependency parsing
US9507852B2 (en) * 2013-12-10 2016-11-29 Google Inc. Techniques for discriminative dependency parsing
US20150169549A1 (en) * 2013-12-13 2015-06-18 Google Inc. Cross-lingual discriminative learning of sequence models with posterior regularization
US9779087B2 (en) * 2013-12-13 2017-10-03 Google Inc. Cross-lingual discriminative learning of sequence models with posterior regularization
US20150279348A1 (en) * 2014-03-25 2015-10-01 Microsoft Corporation Generating natural language outputs
US9542928B2 (en) * 2014-03-25 2017-01-10 Microsoft Technology Licensing, Llc Generating natural language outputs
US9734193B2 (en) * 2014-05-30 2017-08-15 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US20150348565A1 (en) * 2014-05-30 2015-12-03 Apple Inc. Determining domain salience ranking from ambiguous words in natural speech
US20150348543A1 (en) * 2014-06-02 2015-12-03 Robert Bosch Gmbh Speech Recognition of Partial Proper Names by Natural Language Processing
US9589563B2 (en) * 2014-06-02 2017-03-07 Robert Bosch Gmbh Speech recognition of partial proper names by natural language processing
US9619457B1 (en) 2014-06-06 2017-04-11 Google Inc. Techniques for automatically identifying salient entities in documents
US9668024B2 (en) 2014-06-30 2017-05-30 Apple Inc. Intelligent automated assistant for TV user interactions
WO2016010245A1 (en) * 2014-07-14 2016-01-21 Samsung Electronics Co., Ltd. Method and system for robust tagging of named entities in the presence of source or translation errors
US9986419B2 (en) 2014-09-30 2018-05-29 Apple Inc. Social reminders
US9852136B2 (en) 2014-12-23 2017-12-26 Rovi Guides, Inc. Systems and methods for determining whether a negation statement applies to a current or past query
US9854049B2 (en) 2015-01-30 2017-12-26 Rovi Guides, Inc. Systems and methods for resolving ambiguous terms in social chatter based on a user profile
US20170178624A1 (en) * 2015-12-17 2017-06-22 Audeme, LLC Method of facilitating construction of a voice dialog interface for an electronic system
US9934775B2 (en) 2016-05-26 2018-04-03 Apple Inc. Unit-selection text-to-speech synthesis based on predicted concatenation parameters
US9972304B2 (en) 2016-06-03 2018-05-15 Apple Inc. Privacy preserving distributed evaluation framework for embedded personalized systems
US9997156B2 (en) * 2016-12-16 2018-06-12 Audeme, LLC Method of facilitating construction of a voice dialog interface for an electronic system

Similar Documents

Publication Publication Date Title
Zue Toward systems that understand spoken language
Zissman Comparison of four approaches to automatic language identification of telephone speech
US6501833B2 (en) Method and apparatus for dynamic adaptation of a large vocabulary speech recognition system and for use of constraints from a database in a large vocabulary speech recognition system
US7092888B1 (en) Unsupervised training in natural language call routing
Shriberg et al. Can prosody aid the automatic classification of dialog acts in conversational speech?
US5712957A (en) Locating and correcting erroneously recognized portions of utterances by rescoring based on two n-best lists
US8321220B1 (en) System and method of semi-supervised learning for spoken language understanding using semantic role labeling
US7286987B2 (en) Multi-phoneme streamer and knowledge representation speech recognition system and method
Tur et al. Spoken language understanding: Systems for extracting semantic information from speech
US7031908B1 (en) Creating a language model for a language processing system
US6067520A (en) System and method of recognizing continuous mandarin speech utilizing chinese hidden markou models
US20020198713A1 (en) Method and apparatus for perfoming spoken language translation
US6618702B1 (en) Method of and device for phone-based speaker recognition
US6738745B1 (en) Methods and apparatus for identifying a non-target language in a speech recognition system
US6243680B1 (en) Method and apparatus for obtaining a transcription of phrases through text and spoken utterances
Lee Voice dictation of mandarin chinese
US20040260543A1 (en) Pattern cross-matching
US5855000A (en) Method and apparatus for correcting and repairing machine-transcribed input using independent or cross-modal secondary input
US20050187768A1 (en) Dynamic N-best algorithm to reduce recognition errors
US6983239B1 (en) Method and apparatus for embedding grammars in a natural language understanding (NLU) statistical parser
US7231019B2 (en) Automatic identification of telephone callers based on voice characteristics
US20020052742A1 (en) Method and apparatus for generating and displaying N-best alternatives in a speech recognition system
US20030149561A1 (en) Spoken dialog system using a best-fit language model and best-fit grammar
US7475015B2 (en) Semantic language modeling and confidence measurement
Hazen et al. Recognition confidence scoring and its use in speech understanding systems

Legal Events

Date Code Title Description
AS Assignment

Owner name: AT&T CORP., NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GORIN, ALLEN LOUIS;BECHET, FREDERIC;WRIGHT, JEREMY HUNTLEY;AND OTHERS;REEL/FRAME:014087/0302;SIGNING DATES FROM 20030327 TO 20030414

AS Assignment

Owner name: NUANCE COMMUNICATIONS, INC., MASSACHUSETTS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AT&T INTELLECTUAL PROPERTY II, L.P.;REEL/FRAME:041512/0608

Effective date: 20161214