Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F40/00—Handling natural language data
  • G06F40/20—Natural language analysis
  • G06F40/253—Grammatical analysis; Style critique
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/08—Speech classification or search
  • G10L15/18—Speech classification or search using natural language modelling
  • G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
  • G10L15/19—Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
  • G10L15/197—Probabilistic grammars, e.g. word n-grams

Definitions

• This invention generally relates to language modeling and, more specifically, to a system and iterative method for lexicon, word segmentation and language model joint optimization.
• a language model measures the likelihood of any given sentence. That is, a language model can take any sequence of items (words, characters, letters, etc.) and estimate the probability of the sequence.
• a common approach to building a prior art language model is to utilize a prefix tree-like data structure to build an N-gram language model from a known training set of a textual corpus.
• the use of a prefix tree data structure enables a higher-level application to quickly traverse the language model, providing the substantially real-time performance characteristics described above.
• the N-gram language model counts the number of occurrences of a particular item (word, character, etc.) in a string (of size N) throughout a text. The counts are used to calculate the probability of the use of the item strings.
• a textual corpus is dissected into a plurality of items (characters, letters, numbers, etc.);
• the items e.g., characters (C)
• W are segmented (e.g., into words (W)) in accordance with a small, pre-defined lexicon and a simple, pre-defined segmentation algorithm, wherein each W is mapped in the tree to one or more C's;
• train a language model on the dissected corpus by counting the occurrence of strings of characters, from which the probability of a sequence of words (W 1 ⁇ W 2 , ...W M ) is predicted from the previous two words:
• the N-gram language model is limited in a number of respects.
• the counting process utilized in constructing the prefix tree is very time consuming.
• only small N-gram models typically bi-gram, or tri-gram
• the memory required to store the prefix tree increases by 2 N .
• the memory required to store the N-gram language model, and the access time required to utilize a large N-gram language model is prohibitively large for N-grams larger than three (i.e., a tri-gram).
• a fixed lexicon limits the ability of the model to select the best words in general or specific to a task. If a word is not in the lexicon, it does not exist as far as the model is concerned. Thus, a small lexicon is not likely to cover the intended linguistic content.
• segmentation algorithms are often ad-hoc and not based on any statistical or semantic principles.
• a simplistic segmentation algorithm typically errors in favor of larger words over smaller words.
• the model is unable to accurately predict smaller words contained within larger lexiconically acceptable strings.
• This invention concerns a system and iterative method for lexicon
• Fig. 1 is a block diagram of a computer system incorporating the teachings of the present invention
• Fig. 2 is a block diagram of an example modeling agent to iteratively develop a lexicon, segmentation and language model, according to one implementation of the present invention
• Fig. 3 is a graphical representation of a DOMM tree according to one aspect of the present invention.
• Fig. 4 is a flow chart of an example method for building a DOMM tree
• Fig. 5 is a flow chart of an example method for lexicon, segmentation and language model joint optimization, according to the teachings of the present invention
• Fig. 6 is a flow chart detailing the method steps for generating an initial lexicon, and iteratively altering a dynamically generated lexicon, segmentation and language model until convergence, according to one implementation of the present invention.
• Fig. 7 is a storage medium with a plurality of executable instructions which, when executed, implement the innovative modeling agent of the present invention, according to an alternate embodiment of the present invention.
• This invention concerns a system and iterative method for lexicon, segmentation and language model joint optimization.
• an innovative language model the Dynamic Order Markov Model
• DOMM Language Model Data Structure
• program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
• program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types.
• program modules may be located in both local and remote memory storage devices. It is noted, however, that modification to the architecture and methods described herein may well be made without deviating from spirit and scope of the present invention.
• Fig. 1 illustrates an example computer system 102 including an innovative language modeling agent 104, to jointly optimize a lexicon, segmentation and language model according to the teachings of the present invention.
• language modeling agent 104 may well be implemented as a function of an application, e.g., word processor, web browser, speech recognition system, etc.
• application e.g., word processor, web browser, speech recognition system, etc.
• innovative modeling agent may well be implemented in hardware, e.g., a programmable logic array (PLA), a special purpose processor, an application specific integrated circuit (ASIC), microcontroller, and the like.
• PLA programmable logic array
• ASIC application specific integrated circuit
• computer 102 is intended to represent any of a class of general or special purpose computing platforms which, when endowed with the innovative language modeling agent (LMA) 104, implement the teachings of the present invention in accordance with the first example implementation introduced above.
• LMA language modeling agent
• computer system 102 may alternatively support a hardware implementation of LMA 104 as well.
• LMA 104 the following description of computer system 102 is intended to be merely illustrative, as computer systems of greater or lesser capability may well be substituted without deviating from the spirit and scope of the present invention.
• computer 102 includes one or more processors or processing units 132, a system memory 134, and a bus 136 that couples various system components including the system memory 134 to processors 132.
• the bus 136 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures.
• the system memory includes read only memory (ROM) 138 and random access memory (RAM) 140.
• ROM read only memory
• RAM random access memory
• a basic input/output system (BIOS) 142 containing the basic routines that help to transfer information between elements within computer 102, such as during start-up, is stored in ROM 138.
• Computer 102 further includes a hard disk drive 144 for reading from and writing to a hard disk, not shown, a magnetic disk drive 146 for reading from and writing to a removable magnetic disk 148, and an optical disk drive 150 for reading from or writing to a removable optical disk 152 such as a CD ROM, DVD ROM or other such optical media.
• the hard disk drive 144, magnetic disk drive 146, and optical disk drive 150 are connected to the bus 136 by a SCSI interface 154 or some other suitable bus interface.
• the drives and their associated computer-readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for computer 102.
• a number of program modules may be stored on the hard disk 144, magnetic disk 148, optical disk 152, ROM 138, or RAM 140, including an operating system 158, one or more application programs 160 including, for example, the innovative LMA 104 incorporating the teachings of the present invention, other program modules 162, and program data 164 (e.g., resultant language model data structures, etc.).
• a user may enter commands and information into computer 102 through input devices such as keyboard 166 and pointing device 168.
• Other input devices may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are connected to the processing unit 132 through an interface 170 that is coupled to bus 136.
• a monitor 172 or other type of display device is also connected to the bus 136 via an interface, such as a video adapter 174.
• personal computers often include other peripheral output devices (not shown) such as speakers and printers.
• computer 102 operates in a networked environment using logical connections to one or more remote computers, such as a remote computer 176.
• the remote computer 176 may be another personal computer, a personal digital assistant, a server, a router or other network device, a network "thin-client" PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer 102, although only a memory storage device 178 has been illustrated in Fig. 1.
• LAN local area network
• WAN wide area network
• Internet Internet Web browser program
• computer 102 When used in a LAN networking environment, computer 102 is connected to the local network 180 through a network interface or adapter 184.
• computer 102 When used in a WAN networking environment, computer 102 typically includes a modem 186 or other means for establishing communications over the wide area network 182, such as the Internet.
• the modem 186 which may be internal or external, is connected to the bus 136 via a input/output (I/O) interface 156.
• I/O interface 156 also supports one or more printers 188.
• program modules depicted relative to the personal computer 102, or portions thereof, may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
• the data processors of computer 102 are programmed by means of instructions stored at different times in the various computer-readable storage media of the computer. Programs and operating systems are typically distributed, for example, on floppy disks or CD-ROMs.
• the invention described herein includes these and other various types of computer-readable storage media when such media contain instructions or programs for implementing the innovative steps described below in conjunction with a microprocessor or other data processor.
• the invention also includes the computer itself when programmed according to the methods and techniques described below.
• certain sub-components of the computer may be programmed to perform the functions and steps described below.
• the invention includes such sub-components when they are programmed as described.
• the invention described herein includes data structures, described below, as embodied on various types of memory media.
• programs and other executable program components such as the operating system are illustrated herein as discrete blocks, although it is recognized that such programs and components reside at various times in different storage components of the computer, and are executed by the data processor(s) of the computer.
• Fig. 2 illustrates a block diagram of an example language modeling agent
• language modeling agent 104 is comprised of one or more controllers 202, innovative analysis engine 204, storage/memory device(s) 206 and, optionally, one or more additional applications (e.g., graphical user interface, prediction application, verification application, estimation application, etc.) 208, each communicatively coupled as shown.
• additional applications e.g., graphical user interface, prediction application, verification application, estimation application, etc.
• LMA 104 may well be implemented as a function of a higher level application, e.g., a word processor, web browser, speech recognition system, or a language conversion system.
• controller(s) 202 of LMA 104 are responsive to one or more instructional commands from a parent application to selectively invoke the features of LMA 104.
• LMA 104 may well be implemented as a stand-alone language modeling tool, providing a user with a user interface (208) to selectively implement the features of LMA 104 discussed below.
• controller(s) 202 of LMA 104 selectively invoke one or more of the functions of analysis engine 204 to optimize a language model from a dynamically generated lexicon and segmentation algorithm.
• controller 202 is intended to represent any of a number of alternate control systems known in the art including, but not limited to, a microprocessor, a programmable logic array (PLA), a micro-machine, an application specific integrated circuit (ASIC) and the like.
• controller 202 is intended to represent a series of executable instructions to implement the control logic described above.
• the innovative analysis engine 204 is comprised a Markov probability calculator 212, a data structure generator 210 including a frequency calculation function 213, a dynamic lexicon generation function 214 and a dynamic segmention function 216, and a data structure memory manager 218.
• controller 202 selectively invokes an instance of the analysis engine 204 to develop, modify and optimize a statistical language model (SLM).
• SLM statistical language model
• analysis engine 204 develops a statistical language model data structure fundamentally based on the Markov transition probabilities between individual items (e.g., characters, letters, numbers, etc.) of a textual corpus (e.g., one or more sets of text).
• analysis engine 204 utilizes as much data (referred to as "context” or "order” as is available to calculate the probability of an item string.
• the language model of the present invention is aptly referred to as a Dynamic Order Markov Model (DOMM).
• DOMM Dynamic Order Markov Model
• controller 202 When invoked by controller 202 to establish a DOMM data structure, analysis engine 204 selectively invokes the data structure generator 210. In response, data structure generator 210 establishes a tree-like data structure comprised of a plurality of nodes (associated with each of the plurality of items) and denoting inter-node dependencies. As described above, the tree-like data structure is referred to herein as a DOMM data structure, or DOMM tree. Controller 202 receives the textual corpus and stores at least a subset of the textual corpus in memory 206 as a dynamic training set 222 from which the language model is to be developed. It will be appreciated that, in alternate embodiments, a predetermined training set may also be used.
• Frequency calculation function 213 identifies a frequency of occurrence for each item (character, letter, number, word, etc.) in the training set subset. Based on inter-node dependencies, data structure generator 210 assigns each item to an appropriate node of the DOMM tree, with an indication of the frequency value ( ) and a compare bit
• the Markov probability calculator 212 calculates the probability of an item (character, letter, number, etc.) from a context (j) of associated items. More specifically, according to the teachings of the present invention, the Markov probability of a particular item (C,) is dependent on as many previous characters as data "allows", in other words:
• the number of characters employed as context (j) by Markov probability calculator 212 is a "dynamic" quantity that is different for each sequence of characters C Volunteer C,_ ⁇ , C 1-2 , C 1-3 , etc.
• the number of characters relied upon for context (j) by Markov probability calculator 212 is dependent, at least in part, on a frequency value for each of the characters, i.e., the rate at which they appear throughout the corpus. More specifically, if in identifying the items of the corpus Markov probability calculator 212 does not identify at least a minimum occurrence frequency for a particular item, it may be "pruned" (i.e., removed) from the tree as being statistically irrelevant. According to one embodiment, the minimum frequency threshold is three (3).
• analysis engine 204 does not rely on a fixed lexicon or a simple segmentation algorithm (both of which tend to be error prone). Rather, analysis engine 204 selectively invokes a dynamic segmentation function 216 to segment items (characters or letters, for example) into strings (e.g., words). More precisely, segmentation function 216 segments the training set 222 into subsets (chunks) and calculates a cohesion score (i.e., a measure of the similarity between items within the subset). The segmentation and cohesion calculation is iteratively performed by segmentation function 216 until the cohesion score for each subset reaches a predetermined threshold.
• a cohesion score i.e., a measure of the similarity between items within the subset
• the lexicon generation function 214 is invoked to dynamically generate and maintain a lexicon 220 in memory 206.
• lexicon generation function 214 analyzes the segmentation results and generates a lexicon from item strings with a Markov transition probability that exceeds a threshold.
• lexicon generation function 214 develops a dynamic lexicon 220 from item strings which exceed a pre-determined Markov transition probability taken from one or more language models developed by analysis engine 204.
• analysis engine 204 dynamically generates a lexicon of statistically significant, statistically accurate item strings from one or more language models developed over a period of time.
• the lexicon 220 comprises a "virtual corpus" that Markov probability calculator 212 relies upon (in addition to the dynamic training set) in developing subsequent language models.
• data structure memory manager 218 When invoked to modify or utilize the DOMM language model data structure, analysis engine 204 selectively invokes an instance of data structure memory manager 218.
• data structure memory manager 218 utilizes system memory as well as extended memory to maintain the DOMM data structure. More specifically, as will be described in greater detail below with reference to Figs. 6 and 7, data structure memory manager 218 employs a WriteNode function and a ReadNode function (not shown) to maintain a subset of the most recently used nodes of the DOMM data structure in a first level cache 224 of a system memory 206, while relegating least recently used nodes to extended memory (e.g., disk files in hard drive 144, or some remote drive), to provide for improved performance characteristics.
• extended memory e.g., disk files in hard drive 144, or some remote drive
• a second level cache of system memory 206 is used to aggregate write commands until a predetermined threshold has been met, at which point data structure memory manager make one aggregate WriteNode command to an appropriate location in memory.
• data structure memory manager 218 may well be combined as a functional element of controller(s) 202 without deviating from the spirit and scope of the present invention.
• Fig. 3 graphically represents a conceptual illustration of an example Dynamic Order Markov Model tree-like data structure 300, according to the teachings of the present invention.
• Fig. 3 presents an example DOMM data structure 300 for a language model developed from the English alphabet, i.e., A, B, C, ...Z.
• the DOMM tree 300 is comprised of one or more root nodes 302 and one or more subordinate nodes 304, each associated with an item (character, letter, number, word, etc.) of a textual corpus, logically coupled to denote dependencies between nodes.
• root nodes 302 are comprised of an item and a frequency value (e.g., a count of how many times the item occurs in the corpus).
• a frequency value e.g., a count of how many times the item occurs in the corpus.
• the subordinate nodes are arranged in binary sub-trees, wherein each node includes a compare bit (b ; ), an item with which the node is associated
• a binary sub-tree is comprised of subordinate nodes 308-318 denoting the relationships between nodes and the frequency with which they occur.
• the complexity of a search of the DOMM tree approximates log(N), where N is the total number of nodes to be searched.
• DOMM tree 300 may exceed the space available in the memory device 206 of LMA 104 and/or the main memory 140 of computer system 102. Accordingly, data structure memory manager 218 facilitates storage of a DOMM tree data structure 300 across main memory (e.g., 140 and/or 206) into an extended memory space, e.g., disk files on a mass storage device such as hard drive 144 of computer system 102.
• main memory e.g., 140 and/or 206
• extended memory space e.g., disk files on a mass storage device such as hard drive 144 of computer system 102.
• Fig. 4 is a flow chart of an example method for building a Dynamic Order
• Markov Model data structure, according to one aspect of the present invention.
• language modeling agent 104 may be invoked directly by a user or a higher-level application.
• controller 202 of LMA 104 selectively invokes an instance of analysis engine 204, and a textual corpus (e.g., one or more documents) is loaded into memory 206 as a dynamic training set 222 and split into subsets (e.g., sentences, lines, etc.), block 402.
• data structure generator 210 assigns each item of the subset to a node in data structure and calculates a frequency value for the item, block 404.
• frequency calculation function 213 is invoked to identify the occurrence frequency of each item within the training set subset.
• data structure generator determines whether additional subsets of the training set remain and, if so, the next subset is read in block 408 and the process continues with block 404.
• data structure generator 210 completely populates the data structure, a subset at a time, before invocation of the frequency calculation function 213.
• frequency calculation function 213 simply counts each item as it is placed into associated nodes of the data structure.
• data structure generator 210 may optionally prune the data structure, block 410.
• a number of mechanisms may be employed to prune the resultant data structure 300.
• Fig. 5 is a flow chart of an example method for lexicon, segmentation and language model joint optimization, according to the teachings of the present invention. As shown, the method begins with block 400 wherein LM 104 is invoked and a prefix tree of at least a subset of the received corpus is built. More specifically, as detailed in Fig. 4, data structure generator 210 of modeling agent 104 analyzes the received corpus and selects at least a subset as a training set, from which a DOMM tree is built.
• a very large lexicon is built form the prefix tree and pre- processed to remove some obvious illogical words. More specifically, lexicon generation function 214 is invoked to build an initial lexicon from the prefix tree. According to one implementation, the initial lexicon is built from the prefix tree using all sub-strings whose length is less than some pre-defined value, say ten (10) items (i.e., the sub-string is ten nodes or less from root to the most subordinate node). Once the initial lexicon is compiled, lexicon generation function 214 prunes the lexicon by removing some obvious illogical words (see, e.g., block 604, below). According to one implementation, lexicon generation function 214 appends a predefined lexicon with the new, initial lexicon generated from at least the training set of the received corpus.
• some pre-defined value say ten (10) items
• At least the training set of the received corpus is segmented, using the initial lexicon. More particularly, dynamic segmentation function 216 is invoked to segment at least the training set of the received corpus to generate an initial segmented corpus.
• dynamic segmentation function 216 utilizes a Maximum Match technique to provide an initial segmented corpus.
• segmentation function 216 starts at the beginning of an item string (or branch of the DOMM tree) and searches lexicon to see if the initial item (li) is a one-item "word”. Segmentation function then combines it with the next item in the string to see if the combination (e.g., Ijl 2 ) is found as a "word” in the lexicon, and so on. According to one implementation, the longest string (I l5 I 2 , ...I N ) of items found in the lexicon is deemed to be the correct segmentation for that string. It is to be appreciated that more complex Maximum Match algorithms may well be utilized by segmentation function 216 without deviating from the scope and spirit of the present invention.
• an iterative process is entered wherein the lexicon, segmentation and language model are jointly optimized, block 506. More specifically, as will be shown in greater detail below, the innovative iterative optimization employs a statistical language modeling approach to dynamically adjust the segmentation and lexicon to provide an optimized language model. That is, unlike prior art language modeling techniques, modeling agent 104 does not rely on a pre-defined static lexicon, or simplistic segmentation algorithm to generate a language model. Rather, modeling agent 104 utilizes the received corpus, or at least a subset thereof (training set), to dynamically generate a lexicon and segmentation to produce an optimized language model. In this regard, language models generated by modeling agent 104 do not suffer from the drawbacks and limitations commonly associated with prior art modeling systems.
• Fig. 6 presents a more detailed flow chart for generating an initial lexicon, and the iterative process of refining the lexicon and segmentation to optimize the language model, according to one implementation of the present invention.
• the method begins with step 400 (Fig. 4) of building a prefix tree from the received corpus.
• the prefix tree may be built using the entire corpus or, alternatively, using a subset entire corpus (referred to as a training corpus).
• the process of generating an initial lexicon begins with block
• lexicon generation function 214 generates an initial lexicon from the prefix tree by identifying substrings (or branches of the prefix tree) with less than a select number of items. According to one implementation, lexicon generation function 214 identifies substrings of ten (10) items or less to comprise the initial lexicon. In block 604, lexicon generation function 214 analyzes the initial lexicon generated in step 602 for obvious illogical substrings, removing these substrings from the initial lexicon. That is, lexicon generation function 214 analyzes the initial lexicon of substrings for illogical, or improbable words and removes these words from the lexicon.
• dynamic segmentation function 216 is invoked to segment at least the training set of the received corpus to generate an segmented corpus.
• the Maximum Match algorithm is used to segment based on the initial lexicon.
• the frequency analysis function 213 is invoked to compute the frequency of the occurrence in the received corpus for each word in the lexicon, sorting the lexicon according to the frequency of occurrence. The word with the lowest frequency is identified and deleted from the lexicon.
• the threshold for this deletion and re-segmentation may be determined according to the size of the corpus.
• a corpus of 600M items may well utilize a frequency threshold of 500 to be included within the lexicon. In this way, we can delete most of the obvious illogical words from the initial lexicon.
• the received corpus is segmented based, at least in part, on the initial lexicon, block 504.
• the initial segmentation of the corpus is performed using a maximum matching process.
• the iterative process of dynamically altering the lexicon and segmentation begins to optimize a statistical language model (SLM) from the received corpus (or training set), block 506.
• SLM statistical language model
• the process begins in block 606, wherein the Markov probability calculator 212 utilizes the initial lexicon and_segmentation to begin language model training using the segmented corpus. That is, given the initial lexicon and an initial segmentation, a statistical language model may be generated therefrom.
• the language model does not yet benefit from a refined lexicon and a statistically based segmentation (which will evolve in the steps to follow), it is nonetheless fundamentally based on the received corpus itself.
• the initial language model does not yet benefit from a refined lexicon and a statistically based segmentation (which will evolve in the steps to follow), it is nonetheless fundamentally based on the received corpus itself.
• the initial language model does not yet benefit from a refined lexicon and a statistically based segmentation (which will evolve in the steps to follow
• the segmented corpus (or training set) is re-segmented using SLM-based segmentation.
• SLM-based segmentation Given a sentence wl, w2, ...wn, there are M possible ways to segment it (where M ⁇ l).
• Dynamic segmentation function 216 computes a probability (p,) of each segmentation (S,) based on an N-gram statistical language model.
• a Viterbi search algorithm is employed to find the most probable segmentation S k , where:
• the lexicon is updated using the re-segmented corpus resulting from the SLM-based segmentation described above.
• modeling agent 104 invokes frequency analysis function 213 to compute the frequency of occurrence in the received corpus for each word in the lexicon, sorting the lexicon according to the frequency of occurrence. The word with the lowest frequency is identified and deleted from the lexicon. All occurrences of the word must then be re-segmented into smaller words, as the uni- count for all those words are re-computed.
• the threshold for this deletion and re- segmentation may be determined according to the size of the corpus. According to one implementation, a corpus of 600M items may well utilize a frequency threshold of 500 to be included within the lexicon.
• the language model is updated to reflect the dynamically generated lexicon and the SLM-based segmentation, and a measure of the language model perplexity (i.e., an inverse probability measure) is computer by Markov probability calculator 212. If the perplexity continues to converge (toward zero (0)), i.e., improve, the process continues with block 608 wherein the lexicon and segmentation are once again modified with the intent of further improving the language model performance (as measured by perplexity). If in block 614 it is determined that the language model has not improved as a result of the recent modifications to the lexicon and segmentation, a further determination of whether the perplexity has reached an acceptable threshold is made, block 616. If so, the process ends.
• a measure of the language model perplexity i.e., an inverse probability measure
• lexicon generation function 214 deletes the word with the smallest frequency of occurrence in the corpus from the lexicon, re-segmenting the word into smaller words, block 618, as the process continues with block 610.
• innovative language modeling agent 104 generates an optimized language model premised on a dynamically generated lexicon and segmentation rules statistically predicated on at least a subset of the received corpus.
• the resultant language model has improved computational and predictive capability when compared to prior art language models.
• FIG. 7 is a block diagram of a storage medium having stored thereon a plurality of instructions including instructions to implement the innovative modeling agent of the present invention, according to yet another embodiment of the present invention.
• Fig. 7 illustrates a storage medium/device 700 having stored thereon a plurality of executable instructions 702 including at least a subset of which that, when executed, implement the innovative modeling agent 104 of the present invention.
• the executable instructions 702 When executed by a processor of a host system, the executable instructions 702 implement the modeling agent to generate a statistical language model representation of a textual corpus for use by any of a host of other applications executing on or otherwise available to the host system.
• storage medium 700 is intended to represent any of a number of storage devices and/or storage media known to those skilled in the art such as, for example, volatile memory devices, non-volatile memory devices, magnetic storage media, optical storage media, and the like.
• the executable instructions are intended to reflect any of a number of software languages known in the art such as, for example, C++, Visual Basic, Hypertext Markup Language (HTML), Java, extensible Markup Language (XML), and the like.
• the storage medium/device 700 need not be co-located with any host system. That is, storage medium/device 700 may well reside within a remote server communicatively coupled to and accessible by an executing system. Accordingly, the software implementation of Fig. 7 is to be regarded as illustrative, as alternate storage media and software embodiments are anticipated within the spirit and scope of the present invention.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Health & Medical Sciences (AREA)
• Artificial Intelligence (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Computational Linguistics (AREA)
• General Health & Medical Sciences (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Machine Translation (AREA)

Priority Applications (2)

Applications Claiming Priority (4)

Publications (2)

Family

ID=26860000

Family Applications (1)

Country Status (5)

Cited By (5)

Families Citing this family (110)

Family Cites Families (83)

• 2000
  • 2000-06-30 US US09/609,202 patent/US6904402B1/en not_active Expired - Lifetime
  • 2000-11-03 JP JP2001539153A patent/JP2003523559A/ja active Pending
  • 2000-11-03 WO PCT/US2000/041870 patent/WO2001037128A2/en not_active Ceased
  • 2000-11-03 AU AU46104/01A patent/AU4610401A/en not_active Abandoned
  • 2000-11-03 CN CNB008152942A patent/CN100430929C/zh not_active Expired - Fee Related
• 2004
  • 2004-05-10 US US10/842,264 patent/US20040210434A1/en not_active Abandoned

Also Published As

Similar Documents

Legal Events