US7734459B2 - Automatic extraction of transfer mappings from bilingual corpora - Google Patents

Automatic extraction of transfer mappings from bilingual corpora Download PDF

Info

Publication number
US7734459B2
US7734459B2 US09/899,554 US89955401A US7734459B2 US 7734459 B2 US7734459 B2 US 7734459B2 US 89955401 A US89955401 A US 89955401A US 7734459 B2 US7734459 B2 US 7734459B2
Authority
US
United States
Prior art keywords
nodes
node
computer
verb
aligned
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US09/899,554
Other languages
English (en)
Other versions
US20030061023A1 (en
Inventor
Arul A. Menezes
Stephen D. Richardson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US09/899,554 priority Critical patent/US7734459B2/en
Application filed by Microsoft Corp filed Critical Microsoft Corp
Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MENEZES, ARUL A., RICHARDSON, STEPHEN D.
Priority to EP02011351.0A priority patent/EP1262879B1/en
Priority to JP2002162077A priority patent/JP4494706B2/ja
Publication of US20030061023A1 publication Critical patent/US20030061023A1/en
Priority to JP2007211127A priority patent/JP2007328815A/ja
Priority to JP2009253218A priority patent/JP5538820B2/ja
Priority to US12/773,328 priority patent/US8275605B2/en
Publication of US7734459B2 publication Critical patent/US7734459B2/en
Application granted granted Critical
Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MICROSOFT CORPORATION
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/42—Data-driven translation
    • G06F40/45—Example-based machine translation; Alignment

Definitions

  • the present invention relates to automated language translation systems. More particularly, the present invention relates to extracting transfer mappings automatically from bilingual corpora, the mappings associating words and/or logical forms of a first language with words and/or logical forms of a second language.
  • Machine translation systems are systems that receive a textual input in one language, translate it to a second language, and provide a textual output in the second language.
  • Many machine translation systems now use a knowledge base having examples or mappings in order to translate from the first language to the second language.
  • the mappings are obtained from training the system, which includes parsing sentences, or portions thereof, in parallel sentence-aligned corpora in order to extract the transfer rules or examples.
  • These systems typically obtain a predicate-argument or dependency structure for source and target sentences, which are then aligned, and from the resulting alignment, lexical and structural translation correspondences are extracted.
  • the transfer mappings represent these correspondences or associations.
  • Logical forms are first obtained from a bilingual training corpus.
  • a logical form is a data structure (parent/child) that describes labeled dependencies among content words in corresponding text such as a sentence.
  • a method of associating the logical forms to obtain transfer mappings includes associating nodes of the logical forms to form tentative lexical correspondences. The tentative correspondences are aggressively pursued in this phase such that there may be more than one association between some of the nodes of the logical forms.
  • the nodes of the logical forms are aligned by eliminating competing tentative correspondences and/or as a function of the structural considerations of the logical forms.
  • a set of rules can be used.
  • the rules are ordered to create the most unambiguous alignment (“best”) first and then use these alignments to disambiguate subsequent alignments.
  • the rules are applied to the nodes initially irrespective of the parent/child structure in order to create the strongest, most unambiguous alignments first. After establishment of the most meaningful alignments first, the rest of the logical forms are then aligned outwards from these points.
  • mappings can be varied in the amount and type of context in order to create competing or overlapping mappings that have common elements.
  • the competing mappings are analyzed in order to choose, for example, the largest mapping or the one with the most context, and/or the use of other criteria.
  • the use of a larger mapping can provide a more fluent translation; however, by having mappings of varying context, general applicability of the system is maintained.
  • FIG. 1 is a block diagram of an illustrative environment in which the present invention may be used.
  • FIG. 2 is a block diagram of a machine translation architecture in accordance with one embodiment of the present invention.
  • FIG. 3A is an example of a logical form produced for a textual input in a source language (in this example, Spanish).
  • FIG. 3B is a linked logical form for the textual input in the source language.
  • FIG. 3C is a target logical form representing a translation of the source language input to a target language output (in this example, English).
  • FIG. 4 is a flow diagram illustrating a method for aligning nodes.
  • FIG. 5A is an example of tentative correspondences formed between logical forms.
  • FIG. 5B is an example of aligned nodes formed between the logical forms of FIG. 5A .
  • FIG. 6 is a flow diagram illustrating application of a set of rules to the method of FIG. 4 .
  • FIG. 7 is a flow diagram illustrating application of an ordered set of rules.
  • FIG. 8 is a set of transfer mappings associated with the example of FIG. 5B .
  • FIG. 1 The following is a brief description of a general purpose computer 120 illustrated in FIG. 1 .
  • the computer 120 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computer 120 be interpreted as having any dependency or requirement relating to any one or combination of modules illustrated therein.
  • the invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer.
  • program modules include routines, programs, objects, modules, data structures, etc. that perform particular tasks or implement particular abstract data types.
  • the invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network.
  • program modules may be located in both local and remote computer storage media including memory storage devices. Tasks performed by the programs and modules are described below and with the aid of figures.
  • processor executable instructions which can be written on any form of a computer readable media.
  • modules of computer 120 may include, but are not limited to, a processing unit 140 , a system memory 150 , and a system bus 141 that couples various system modules or components including the system memory to the processing unit 140 .
  • the system bus 141 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
  • bus architectures include Industry Standard Architecture (ISA) bus, Universal Serial Bus (USB), Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Module Interconnect (PCI) bus also known as Mezzanine bus.
  • Computer 120 typically includes a variety of computer readable mediums.
  • Computer readable mediums can be any available media that can be accessed by computer 120 and includes both volatile and nonvolatile media, removable and non-removable media.
  • Computer readable mediums may comprise computer storage media and communication media.
  • Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules/components or other data.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 120 .
  • Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
  • modulated data signal means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
  • communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, FR, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer readable media.
  • the system memory 150 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 151 and random access memory (RAM) 152 .
  • ROM read only memory
  • RAM random access memory
  • BIOS basic input/output system
  • RAM 152 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 140 .
  • FIG. 1 illustrates operating system 154 , application programs 155 , other program modules 156 , and program data 157 .
  • the computer 120 may also include other removable/non-removable volatile/nonvolatile computer storage media.
  • FIG. 1 illustrates a hard disk drive 161 that reads from or writes to non-removable, nonvolatile magnetic media, a magnetic disk drive 171 that reads from or writes to a removable, nonvolatile magnetic disk 172 , and an optical disk drive 175 that reads from or writes to a removable, nonvolatile optical disk 176 such as a CD ROM or other optical media.
  • removable/non-removable, volatile/nonvolatile computer storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory cards, digital versatile disks, digital video tape, solid state RAM, solid state ROM, and the like.
  • the hard disk drive 161 is typically connected to the system bus 141 through a non-removable memory interface such as interface 160
  • magnetic disk drive 171 and optical disk drive 175 are typically connected to the system bus 141 by a removable memory interface, such as interface 170 .
  • hard disk drive 161 is illustrated as storing operating system 164 , application programs 165 , other program modules 166 , and program data 167 . Note that these modules can either be the same as or different from operating system 154 , application programs 155 , other program modules 156 , and program data 157 . Operating system 164 , application programs 165 , other program modules 166 , and program data 167 are given different numbers here to illustrate that, at a minimum, they are different copies.
  • a user may enter commands and information into the computer 120 through input devices such as a keyboard 182 , a microphone 183 , and a pointing device 181 , such as a mouse, trackball or touch pad.
  • Other input devices may include a joystick, game pad, satellite dish, scanner, or the like.
  • a monitor 184 or other type of display device is also connected to the system bus 141 via an interface, such as a video interface 185 .
  • computers may also include other peripheral output devices such as speakers 187 and printer 186 , which may be connected through an output peripheral interface 188 .
  • the computer 120 may operate in a networked environment using logical connections to one or more remote computers, such as a remote computer 194 .
  • the remote computer 194 may be a personal computer, a hand-held device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 120 .
  • the logical connections depicted in FIG. 1 include a local area network (LAN) 191 and a wide area network (WAN) 193 , but may also include other networks.
  • LAN local area network
  • WAN wide area network
  • Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets and the Internet.
  • the computer 120 When used in a LAN networking environment, the computer 120 is connected to the LAN 191 through a network interface or adapter 190 .
  • the computer 120 When used in a WAN networking environment, the computer 120 typically includes a modem 192 or other means for establishing communications over the WAN 193 , such as the Internet.
  • the modem 192 which may be internal or external, may be connected to the system bus 141 via the user input interface 180 , or other appropriate mechanism.
  • program modules depicted relative to the computer 120 may be stored in the remote memory storage device.
  • FIG. 1 illustrates remote application programs 195 as residing on remote computer 194 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communications link between the computers may be used.
  • the invention is also operational with numerous other general purpose or special purpose computing systems, environments or configurations.
  • Examples of well known computing systems, environments, and/or configurations that may be suitable for use with the invention include, but are not limited to, regular telephones (without any screen) personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
  • Logical forms can normalize certain syntactical alternations, (e.g., active/passive) and resolve both intrasentential anaphora and long distance dependencies.
  • a logical form 252 can be represented as a graph, which helps intuitively in understanding the elements of logical forms.
  • the logical forms may not readily be understood as representing a graph.
  • a logical relation consists of two words joined by a directional relation type, such as:
  • a logical form is a data structure of connected logical relations representing a single textual input, such as a sentence or part thereof.
  • the logical form minimally consists of one logical relation and portrays structural relationships (i.e., syntactic and semantic relationships), particularly argument and/or adjunct relation(s) between important words in an input string.
  • the particular code that builds logical forms from syntactic analyses is shared across the various source and target languages that the machine translation system operates on.
  • the shared architecture greatly simplifies the task of aligning logical form segments from different languages since superficially distinct constructions in two languages frequently collapse onto similar or identical logical form representations. Examples of logical forms in different languages are described in greater detail below with respect to FIGS. 3A-3C .
  • FIG. 2 is a block diagram of an architecture of a machine translation system 200 in accordance with one embodiment of the present invention.
  • System 200 includes parsing components 204 and 206 , statistical word association learning component 208 , logical form alignment component 210 , lexical knowledge base building component 212 , bilingual dictionary 214 , dictionary merging component 216 , transfer mapping database 218 and updated bilingual dictionary 220 .
  • the system 200 utilizes analysis component 222 , matching component 224 , transfer component 226 and/or generation component 228 .
  • a bilingual corpus is used to train the system.
  • the bilingual corpus includes aligned translated sentences (e.g., sentences in a source or target language, such as English, in 1-to-1 correspondence with their human-created translations in the other of the source or target language, such as Spanish).
  • sentences are provided from the aligned bilingual corpus into system 200 as source sentences 230 (the sentences to be translated), and as target sentences 232 (the translation of the source sentences).
  • Parsing components 204 and 206 parse the sentences from the aligned bilingual corpus to produce source logical forms 234 and target logical forms 236 .
  • the words in the sentences are converted to normalized word forms (lemmas) and can be provided to statistical word association learning component 208 . Both single word and multi-word associations are iteratively hypothesized and scored by learning component 208 until a reliable set of each is obtained.
  • Statistical word association learning component 208 outputs learned single word translation pairs 238 as well as multi-word pairs 240 .
  • the multi-word pairs 240 are provided to a dictionary merge component 216 , which is used to add additional entries into bilingual dictionary 214 to form updated bilingual dictionary 220 .
  • the new entries are representative of the multi-word pairs 240 .
  • the single and multi-word pairs 238 , along with source logical forms 234 and target logical forms 236 are provided to logical form alignment component 210 .
  • component 210 first establishes tentative correspondences between nodes in the source and target logical forms 230 and 236 , respectively. This is done using translation pairs from a bilingual lexicon (e.g. bilingual dictionary) 214 , which can be augmented with the single and multi-word translation pairs 238 , 240 from statistical word association learning component 208 .
  • alignment component 210 aligns logical form nodes according to both lexical and structural considerations and creates word and/or logical form transfer mappings 242 . This aspect will be explained in greater detail below.
  • alignment component 210 draws links between logical forms using the bilingual dictionary information 214 and single and multi-word pairs 238 , 240 .
  • the transfer mappings are optionally filtered based on a frequency with which they are found in the source and target logical forms 234 and 236 and are provided to a lexical knowledge base building component 212 .
  • transfer mapping is not seen at least twice in the training data, it is not used to build transfer mapping database 218 , although any other desired frequency can be used as a filter as well. It should also be noted that other filtering techniques can be used as well, other than frequency of appearance. For example, transfer mappings can be filtered based upon whether they are formed from complete parses of the input sentences and based upon whether the logical forms used to create the transfer mappings are completely aligned.
  • Component 212 builds transfer mapping database 218 , which contains transfer mappings that basically link words and/or logical forms in one language, to words and/or logical forms in the second language. With transfer mapping database 218 thus created, system 200 is now configured for runtime translations.
  • a source sentence 250 is provided to analysis component 222 .
  • Analysis component 222 receives source sentence 250 and creates a source logical form 252 based upon the source sentence input.
  • source sentence 250 is a Spanish sentence “Haga click en el boton de opcion” which is translated into English as “Click the option button” or, literally, “Make click in the button of option”.
  • FIG. 3A illustrates the source logical form 252 generated for source sentence 250 by analysis component 222 .
  • the source logical form 252 is provided to matching component 224 .
  • Matching component 224 attempts to match the source logical form 252 to logical forms in the transfer mapping database 218 in order to obtain a linked logical form 254 .
  • Multiple transfer mappings may match portions of source logical form 252 .
  • Matching component 224 searches for the best set of matching transfer mappings in database 218 that have matching lemmas, parts of speech, and other feature information. The set of best matches is found based on a predetermined metric. For example, transfer mappings having larger (more specific) logical forms may illustratively be preferred to transfer mappings having smaller (more general) logical forms.
  • mappings having logical forms of equal size may illustratively prefer higher frequency mappings. Mappings may also match overlapping portions of the source logical form 252 provided that they do not conflict with each other in any way. A set of mappings collectively may be illustratively preferred if they cover more of the input sentence than alternative sets. Other metrics used in matching the input logical form to those found in database 218 are discussed in greater detail below with respect to Table 1.
  • mapping component 224 creates links on nodes in the source logical form 252 to copies of the corresponding target words or logical form segments received by the transfer mappings, to generate linked logical form 254 .
  • FIG. 3B illustrates an example of linked logical form 254 for the present example.
  • Links for multi-word mappings are represented by linking the root nodes (e.g., Hacer and Click) of the corresponding segments, then linking an asterisk to the other source nodes participating in the multi-word mapping (e.g., Usted and Clic). Sublinks between corresponding individual source and target nodes of such a mapping (not shown in FIG. 3B ) may also illustratively be created for use during transfer.
  • Transfer component 226 receives linked logical form 254 from matching component 224 and creates a target logical form 256 that will form the basis of the target translation. This is done by performing a top down traversal of the linked logical form 254 in which the target logical form segments pointed to by links on the source logical form 252 nodes are combined. When combining together logical form segments for possibly complex multi-word mappings, the sublinks set by matching component 224 between individual nodes are used to determine correct attachment points for modifiers, etc. Default attachment points are used if needed.
  • FIG. 3C illustrates a target logical form 256 for the present example. It can be seen that the logical form segments from “click” to “button” and from “button” to “option” were stitched together from linked logical form 254 to obtain target logical form 256 .
  • Generation component 228 is illustratively a rule-based, application-independent generation component that maps from target logical form 256 to the target string (or output target sentence) 258 .
  • Generation component 228 may illustratively have no information regarding the source language of the input logical forms, and works exclusively with information passed to it by transfer component 226 .
  • Generation component 228 also illustratively uses this information in conjunction with a monolingual (e.g., for the target language) dictionary to produce target sentence 258 .
  • a monolingual (e.g., for the target language) dictionary to produce target sentence 258 .
  • One generic generation component 228 is thus sufficient for each language.
  • the present system parses information from various languages into a shared, common, logical form so that logical forms can be matched among different languages.
  • the system can also utilize simple filtering techniques in building the transfer mapping database to handle noisy data input. Therefore, the present system can be automatically trained using a very large number of sentence pairs.
  • the number of sentence pairs is in excess of 10,000.
  • the number of sentence pairs is greater than 50,000 to 100,000, and may be in excess of 180,000, 200,000, 350,000 or even in excess if 500,000 or 600,000 sentence pairs.
  • the number of sentence pairs can vary for different languages, and need not be limited to these numbers.
  • FIG. 4 illustrates a method 300 of associating logical forms of at least sentence fragments from two different languages, wherein the logical forms comprise nodes organized in a parent/child structure.
  • Method 300 includes associating nodes of the logical forms to form tentative correspondences as indicated at block 302 and aligning nodes of the logical forms by eliminating at least one of the tentative correspondences and/or structural considerations as indicated at block 304 .
  • alignment component 210 accesses bilingual dictionary 214 in order to form tentative correspondences, typically lexical correspondences, between the logical forms.
  • Bilingual dictionary 214 can be created by merging data from multiple sources, and can also use inverted target-to-source dictionary entries to improve coverage.
  • bilingual dictionary 214 also represents any other type of resource that can provide correspondences between words.
  • Bilingual dictionary 214 can also be augmented with translation correspondences acquired using statistical techniques.
  • component 208 the statistical techniques are performed by component 208 .
  • the output from component 208 can be used by alignment component 210 , it is not necessary for operation of alignment component 210 .
  • one embodiment of component 208 will be described here, briefly, for the sake of completeness.
  • Component 208 receives a parallel, bilingual training corpus that is parsed into its content words.
  • Word association scores for each pair of content words consisting of a word of language L 1 that occurs in a sentence aligned in the bilingual corpus to a sentence of language L 2 in which the other word occurs.
  • a pair of words is considered “linked” in a pair of aligned sentences if one of the words is the most highly associated, of all the words in its sentence, with the other word.
  • the occurrence of compounds is hypothesized in the training data by identifying maximal, connected sets of linked words in each pair of aligned sentences in the processed and scored training data.
  • the subset of the words in that language is hypothesized as a compound.
  • the original input text is rewritten, replacing the hypothesized compounds by single, fused tokens.
  • the association scores are then recomputed for the compounds (which have been replaced by fused tokens) and any remaining individual words in the input text.
  • the association scores are again recomputed, except that this time, co-occurrences are taken into account in computing the association scores only where there is no equally strong or stronger other association in a particular pair of aligned sentences in the training corpus.
  • Translation pairs can be identified as those word pairs or token pairs that have association scores above a threshold, after the final computation of association scores.
  • component 208 can also assist in identifying translations of “captoids”, by which we mean titles, or other special phrases, all of whose words are capitalized. (Finding translations of captoids presents a special problem in languages like French or Spanish, in which convention dictates that only the first word of such an item is capitalized, so that the extent of the captoid translation is difficult to determine.)
  • compounds are first identified in a source language (such as English). This can be done by finding strings of text where the first word begins with a capital letter, and later tokens in the contiguous string do not begin with a lowercase letter. Next, compounds are hypothesized in the target text by finding words that start with a capital letter and flagging this as the possible start of a corresponding compound.
  • the target text is then scanned from left to right flagging subsequent words that are most strongly related to words in the identified compound in the source text, while allowing up to a predetermined number (e.g., 2) contiguous non-most highly related words, so long as they are followed by a most highly related word.
  • a predetermined number e.g., 2
  • the left to right scan can be continued until more than the predetermined number (e.g., more than 2) contiguous words are found that are not most highly related to words in the identified compound in the source text, or until no more most highly related words are present in the target text, or until punctuation is reached.
  • the predetermined number e.g., more than 2 contiguous words are found that are not most highly related to words in the identified compound in the source text, or until no more most highly related words are present in the target text, or until punctuation is reached.
  • component 208 is optional.
  • step 302 is aggressively pursued with the purpose of attempting to maximize the number of tentative correspondences formed between the logical forms. Accuracy of the tentative correspondences is not the most important criteria in step 302 because step 304 will further analyze the tentative correspondences and remove those that are determined to be incorrect.
  • Bilingual dictionary 214 represents direct translations used for forming tentative correspondences.
  • derivational morphology can also be used.
  • translations of morphological bases and derivations, and base and derived forms of translations can also be used to form tentative correspondences in step 302 .
  • tentative correspondences can also be formed between nodes of the logical forms wherein one of the nodes comprises more lexical elements or words than the other node.
  • one of the nodes can comprise a single word in one of the languages, while the other node comprises at least two words in the other language. Closely related languages such as English, Spanish, etc. also have word similarity (cognates) that can be used with fuzzy logic to ascertain associations. These associations can then be used to form tentative correspondences.
  • logical form 320 was generated for the sentence “En Informa Terms del hipervinculo, haga tun en la direc Terms del hipervinculo”, while logical form 322 was generated for the English translation as “Under Hyperlink Information, click the Hyperlink address.”
  • FIG. 5A further illustrates each of the tentative correspondences 323 identified in step 302 .
  • each of the occurrences of “Hipervinculo” includes two different tentative correspondences with “Hyperlink_Information” and “hyperlink” in the English logical form 322 .
  • step 304 the logical forms are aligned, which can include eliminating one or more of the tentative correspondences formed in step 302 , and/or which can be done as a function of structural considerations of the logical forms.
  • step 304 includes aligning nodes of the logical forms as a function of a set of rules.
  • each of the rules of the set of rules is applied to the logical forms in a selected order.
  • the rules are ordered to create the most unambiguous alignments (“best alignments”) first, and then, if necessary, to disambiguate subsequent node alignments.
  • the order that the rules are applied in is not based upon the structure of the logical forms, i.e., top-down processing or bottom-up processing but rather, to begin with the most linguistically meaningful alignments, wherever they appear in the logical form.
  • this set of rules can be considered to be applied to the nodes of each of the logical forms non-linearly as opposed to linearly based upon the structure of the logical forms.
  • the rules are intended to be language-neutral in order that they can be universally applied to any language.
  • FIG. 6 generally illustrates application of the set of rules to the logical forms as method 328 .
  • each of the nodes of the logical forms is considered to be “unaligned” as opposed to “aligned”.
  • the set of rules is applied to the unaligned nodes irrespective of structure at step 332 to form aligned nodes. Therefore, it is desirable to distinguish between unaligned nodes and aligned nodes.
  • One technique includes assigning all of the nodes initially to the set of unaligned nodes, and removing nodes when they are aligned.
  • the use of sets whether actively formed in different locations of a computer readable medium or virtually formed through the use of Boolean tags associated with the nodes merely provides a convenient way in which to identify unaligned nodes and aligned nodes.
  • step 332 the set of rules is applied to each of the unaligned nodes.
  • FIG. 7 schematically illustrates aspects of step 332 that can be implemented to apply the set of rules.
  • the rules are applied in a specified order.
  • N is a counter that is used to indicate which of the rules is applied.
  • step 334 applies the first rule to each of the unaligned nodes. If a rule fails to be applicable to any of the unaligned nodes, another rule from the set (and in one embodiment, the next successive rule indicative of a linguistically meaningful alignment) is then applied as indicated at steps 336 and 338 .
  • a rule can be applied to a set of nodes of the logical forms, the nodes are identified as being aligned and removed from the set of unaligned nodes, and application of the rules continues.
  • each of the rules of the set of rules is applied again starting with, for example, the first rule as indicated at step 342 .
  • the following is an exemplary set of rules for aligning nodes of the logical forms.
  • the set of nodes presented herein is ordered based on the strongest to weakest linguistically meaningful alignments of the nodes. As appreciated by those skilled in the art, reordering of at least some of the rules presented herein may not significantly alter the quality of alignments of the logical forms.
  • a bi-directionally unique translation exists between a node or set of nodes in one logical form and a node or set of nodes in the other logical form, the two nodes or sets of nodes are aligned to each other.
  • a bi-directionally unique translation exists if a node or a set of nodes of one logical form has a tentative correspondence with a node or a set of nodes in the other logical form, such that every node in the first set of nodes has a tentative correspondence with every node in the second set of nodes, and no other correspondences, and every node in the second set of nodes has a tentative correspondence with every node in the first set of nodes, and no other correspondences.
  • a pair of parent nodes, one from each logical form, having a tentative correspondence to each other, are aligned with each other if each child node of each respective parent node is already aligned to a child of the other parent node.
  • a pair of child nodes, one from each logical form, are aligned with each other if a tentative correspondence exists between them and if a parent node of each respective child node is already aligned to a corresponding parent node of the other child.
  • a pair of nodes, one from each logical form, are aligned to each other if respective parent nodes of the nodes under consideration are aligned with each other and respective child nodes are also aligned with each other.
  • a node that is a verb and an associated child node that is not a verb from one logical form are aligned to a second node that is a verb of the other logical form if the associated child node is already aligned with the second verb node, and either the second verb node has no aligned parent nodes, or the first verb node and the second verb node have child nodes aligned with each other.
  • a pair of nodes, one from each logical form, comprising the same part-of-speech, are aligned to each other, if there are no unaligned sibling nodes, and respective parent nodes are aligned, and linguistic relationships between the set of nodes under consideration and their respective parent nodes are the same.
  • a pair of nodes, one from each logical form, comprising the same part-of-speech, are aligned to each other if respective child nodes are aligned with each other and the linguistic relationship between the set of nodes under consideration and their respective child nodes are the same.
  • an unaligned node of one of the logical forms having immediate neighbor nodes comprising respective parent nodes, if any, all aligned, and respective child nodes, if any, all aligned, and if exactly one of the immediate nodes is a non-compound word aligned to a node of the other logical form comprising a compound word then align the unaligned node with the node comprising the compound word.
  • the immediate neighbor nodes herein comprise adjacent parent and child nodes however the existence of parent and child nodes is not required, but if they are present, they must be aligned.
  • a pair of nodes, one from each logical form, comprising pronouns, are aligned to each other if respective parent nodes are aligned with each other and neither of the nodes under consideration have unaligned siblings.
  • a pair of nodes, one from each logical form, comprising nouns are aligned to each other if respective parent nodes comprising nouns are aligned with each other and neither of the nodes under consideration have unaligned sibling nodes, and wherein a linguistic relationship between each of the nodes under consideration and their respective parent nodes comprises either a modifier relationship or a prepositional relationship.
  • a first verb node of one logical form is aligned to a second verb node of the other logical form if the first verb node has no tentative correspondences and has a single associated child verb node that is already aligned with the second verb node.
  • a first verb node and a single, respective parent node of one logical form is aligned to a second verb node of the other logical form if the first verb node has no tentative correspondences and has a single parent verb node that is already aligned with the second verb node, where the single parent verb node has no unaligned verb child nodes besides the first verb node, and the second verb node has no unaligned verb child nodes.
  • a first node comprising a pronoun of one logical form is aligned to a second node of the other logical form if a parent node of the first node is aligned with the second node and the second node has no unaligned child nodes.
  • a first verb node and a respective parent verb of one logical form is aligned to a second verb node of the other logical form if the first verb node has no tentative correspondences and the parent verb node is aligned with the second verb node and where the relationship between the first verb and the parent verb node comprise a modal relationship.
  • rule 1 Some general classifications of the rules provided above include that one rule (rule 1) is primarily based upon the correspondences established in step 302 , and in the embodiment illustrated, it is considered to be the strongest meaningful alignment since no ambiguity is present.
  • Other rules such as rules 2, 3, 11, 12 and 14 are based on a combination of, or a lack of, tentative correspondences and the structure of the nodes under consideration and previously aligned nodes. The remaining rules rely solely on relationships between nodes under consideration and previously aligned nodes.
  • Other general classifications that can be drawn include that the rules pertain to verbs, nouns and pronouns.
  • rule 1 applies in three places, creating alignment mappings between “direc Terms” and “address”, “usted” and “you”, and “clic” and “click”. These are the initial “best” alignments that provide the anchors from which the method will work outwards to align the rest of the structure.
  • Rule 2 does not apply to any nodes, but Rule 3 applies next to align the instance of “hipervinculo”, that is the child of “direc Terms” to “hyperlink”, which is the child of “address”.
  • the alignment method thus leveraged a previously created alignment (“direc Terms” to “address”) and the structure of the logical form to resolve the ambiguity present at the lexical level.
  • Rule 1 applies (where previously it did not) to create a many-to-one mapping between “Informa Terms” and “hipervinculo” to “Hyperlink_Information”.
  • the uniqueness condition in this rule is now met because the ambiguous alternative was cleared away by the prior application of Rule 3.
  • rule 4 does not apply, but rule 5 applies to rollup “hacer” with its object “clic”, since the latter is already aligned to a verb. This produces the many-to-one alignment of “hacer” and “clic” to “click”.
  • FIG. 8 illustrates some of the transfer mappings obtainable from the example of aligned logical forms in FIG. 5B (other than transfer mapping 353 which is included as an example of a conflicting transfer mapping discussed in the next section).
  • a transfer mapping or simply “mapping” is indicative of associating a word or logical form of a first language with a corresponding word or logical form of a second language.
  • the mappings can be stored on any computer readable medium as explicit pointers linking the words or logical forms of the first language with the corresponding words or logical forms of the second language.
  • the mappings can be stored with the words or logical forms rather than in a separate database.
  • other techniques can be used to associate words or logical forms of the first language with words or logical forms of the second language, and it is this association, that constitutes the mappings regardless of the specific techniques used in order to record this information.
  • Each mapping created during the alignment procedure can be a base structure upon which further mappings with additional context are also created.
  • information can be stored on a computer readable medium to translate text from a first language to a second language, where the information comprises a plurality of mappings.
  • Each mapping is indicative of associating a word or logical form of the first language with a word or logical form of the second language.
  • at least some of the mappings corresponding to logical forms of the first language have varying context with some common elements.
  • at least some of the logical forms of the second language corresponding to the logical forms of the first language may also have varying context with some common elements.
  • at least some of the core mappings obtained from the alignment procedure are used to create other, competing mappings having varying types and amounts of local context.
  • mappings 350 , 352 , and 354 illustrate how an element of a logical form can vary.
  • Mapping 350 comprises the base or core mapping on which further mappings are created.
  • Mapping 352 expands the core mapping 350 to include an additional linguistic element, herein the direct object of the word “click”, while the mapping 354 is expanded from the core mapping 350 such that the additional element comprises an under-specified node (“*”) indicating a part of speech but no specific lemma.
  • mappings 350 , 352 and 354 as well as mappings 356 and 358 , it can be seen that the logical forms of the first language have common elements (parts of speech and/or lemmas), while the logical forms of the second language also have common elements.
  • mappings indicative of logical forms with overlapping context By storing mappings indicative of logical forms with overlapping context, during translation run time, fluency and general applicability of the mappings for translating between the languages is maintained. In particular, by having mappings associating both words and smaller logical forms of the languages, translation from the first language to the second language is possible if the text to be translated was not seen in the training data. However, to the extent that the larger context was present in the training data, this is also reflected in the mappings such that when a mapping of larger context is applicable, a more fluent translation between the first language and the second language can be obtained.
  • mappings for an adjective can be expanded to include the noun it modifies.
  • a mapping for a verb can be expanded to include the object as context.
  • mappings for noun collocations are provided individually as well as a whole.
  • some of the mappings can include under-specified nodes (“*”), wherein the part of speech is indicated but no specific lemma is provided.
  • mappings that can be created may have any number of wild-card or underspecified nodes, which may be underspecified in a number of different ways. For example, they may or may not specify a part-of-speech, and they may specify certain syntactic or semantic features. For example, a pattern may have a wild-card node with the feature “ProperName” or “Location” marked, indicating that the pattern only applies when that node is matched to an input node that has the same feature. These wild-cards allow the system to hypothesize generalized mappings from specific data.
  • additional information can also be stored or used during run time translation.
  • the additional information can be used to choose an appropriate set of mappings and resolve conflicts as to which mappings to use, i.e. (referring to FIG. 2 ) when a source logical form 252 (or part thereof) generated for a source sentence 250 matches more than one source side of the transfer mappings in the transfer mapping database 218 .
  • a subset of these matching transfer mappings is illustratively selected such that all transfer mappings in the subset are compatible with one another (i.e., they are not conflicting) and based on a metric that is a function of how much of the input sentence the transfer mappings in the subset collectively cover, as well as other measures related to individual transfer mappings.
  • Some such measures are set out in Table 1.
  • the transfer mappings in the subset are combined into a transfer logical form from which the output text is generated.
  • the subset of matching transfer mappings can contain overlapping transfer mappings, so long as they are compatible.
  • the following logical form can be generated for the Spanish sentence “Haga project en el direccion de la oficina” which can be translated as “Click the office address”:
  • transfer mapping 353 An example of conflicting, matching transfer mappings is shown as transfer mapping 353 , which conflicts with transfer mapping 352 . Therefore, for example, the logical form:
  • Hacer -- Dobj - click - en - direccion would match all of transfer mappings 350 , 352 , 353 and 354 .
  • transfer mappings 352 and 353 conflict because they are translated differently
  • both cannot be part of the selected subset of matching transfer mappings.
  • one is selected based on a predetermined metric. For example, subset 350 , 352 and 354 can be compared against subset 350 , 353 and 354 to see which covers the most nodes in the input logical form, collectively.
  • both transfer mappings 352 and 353 are the same size (on the source side). Therefore, other information can be used to distinguish between them in selecting the subset of matching transfer mappings.
  • mappings (note these examples are English mapped to Spanish whereas previous examples have been Spanish mapped to English):
  • translating “click” to “select” may be a legitimate variation. However it does present a problem in some cases. For example, notice that the source side of both transfers is identical, so at runtime, if the input logical form matches that source side, we are left with having to choose between the two different target sides, i.e. it must be decided whether to translate the input as “hacer project . . .” or as “elegir . . .”? In the absence of further context (which would likely have manifested itself by causing differing source sides of the transfers) we choose between them based on various frequency and scoring metrics.
  • Table 1 shows examples of the information that can be used to further define the subset of matching transfer mappings (either to choose among conflicting matching transfer mappings or to narrow the subset of compatible, matching transfer mappings).
  • Such information can include how much of the input sentence is covered by the subset of matching transfer mappings (collectively) and the size of the mappings, which can be ascertained from the logical form that is matched in the transfer mapping itself.
  • the size of a logical form includes both the number of specified nodes as well as the number of linguistic relationships between the nodes.
  • the size of the logical form from the source side of mapping 350 equals 2
  • the size of the logical form on the target side equals 1.
  • the logical form on the source side of mapping 354 equals 4
  • the target side of mapping 354 equals 2.
  • the information for choosing the subset of transfer mappings can also include other information related to individual transfer mappings, such as the frequency with which the logical forms in the transfer mapping are seen in the training data.
  • the training data can include “trusted” training data, which can be considered more reliable than other training data.
  • the frequency of the mapping as seen in the trusted training data can be retained in addition, or in the alternative, to storing the frequency of the mapping as seen in all of the training data.
  • Other information that can be helpful in selecting the subset of matching transfer mappings when matching source logical forms to transfer mappings includes the extent of complete alignment of the logical forms in the training data from which the logical forms of a transfer mapping have been obtained.
  • the alignment procedure can fully or completely align the nodes of the larger logical forms, or some nodes can remain unaligned. In the example of FIG. 5B , all the nodes were aligned; however, as indicated above, this may not always be the case.
  • Those mappings associated with fully aligned logical forms may be considered more reliable.
  • information for resolving conflicts or further defining the subset can also indicate the frequency with which the mapping was generated from both fully aligned logical forms as well as partially aligned logical forms.
  • additional information can include the frequency with which the logical forms in the transfer mapping originated from a complete parse of the corresponding training data.
  • the frequency with which the mapping originated from a complete or fitted parse, or in contrast, the frequency that the mapping originated from only a partial parse can be stored for later use in resolving conflicts while matching during translation.
  • Another form of information can include a score or value assigned to the transfer mapping by the alignment procedure used to extract the mapping.
  • the score can be a function of how “strong” (linguistically meaningful) the aligned nodes are (or how confident the alignment component is in the transfer mapping).
  • the score can therefore be a function of when (which iteration) and which rule formed the alignment.
  • the particular function or metric used to calculate the alignment score is not crucial, and any such metric can be used to generate information related to an alignment score that can be used during run time translation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
US09/899,554 2001-06-01 2001-07-05 Automatic extraction of transfer mappings from bilingual corpora Expired - Fee Related US7734459B2 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US09/899,554 US7734459B2 (en) 2001-06-01 2001-07-05 Automatic extraction of transfer mappings from bilingual corpora
EP02011351.0A EP1262879B1 (en) 2001-06-01 2002-05-23 Automatic extraction of transfer mappings from bilingual corpora
JP2002162077A JP4494706B2 (ja) 2001-06-01 2002-06-03 ２カ国語コーパスからの変換マッピングの自動抽出プログラム
JP2007211127A JP2007328815A (ja) 2001-06-01 2007-08-13 ２カ国語コーパスからの変換マッピングの自動抽出プログラム
JP2009253218A JP5538820B2 (ja) 2001-06-01 2009-11-04 ２カ国語コーパスからの変換マッピングの自動抽出プログラム
US12/773,328 US8275605B2 (en) 2001-06-01 2010-05-04 Machine language translation with transfer mappings having varying context

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US29533801P 2001-06-01 2001-06-01
US09/899,554 US7734459B2 (en) 2001-06-01 2001-07-05 Automatic extraction of transfer mappings from bilingual corpora

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/773,328 Division US8275605B2 (en) 2001-06-01 2010-05-04 Machine language translation with transfer mappings having varying context

Publications (2)

Publication Number Publication Date
US20030061023A1 US20030061023A1 (en) 2003-03-27
US7734459B2 true US7734459B2 (en) 2010-06-08

Family

ID=26969061

Family Applications (2)

Application Number Title Priority Date Filing Date
US09/899,554 Expired - Fee Related US7734459B2 (en) 2001-06-01 2001-07-05 Automatic extraction of transfer mappings from bilingual corpora
US12/773,328 Expired - Fee Related US8275605B2 (en) 2001-06-01 2010-05-04 Machine language translation with transfer mappings having varying context

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/773,328 Expired - Fee Related US8275605B2 (en) 2001-06-01 2010-05-04 Machine language translation with transfer mappings having varying context

Country Status (3)

Country Link
US (2) US7734459B2 (ja)
EP (1) EP1262879B1 (ja)
JP (3) JP4494706B2 (ja)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100223049A1 (en) * 2001-06-01 2010-09-02 Microsoft Corporation Machine language translation with transfer mappings having varying context
US20110257962A1 (en) * 2008-10-22 2011-10-20 Kumar Bulusu Gopi System and method for automatically generating sentences of a language
US9916306B2 (en) 2012-10-19 2018-03-13 Sdl Inc. Statistical linguistic analysis of source content
US9954794B2 (en) 2001-01-18 2018-04-24 Sdl Inc. Globalization management system and method therefor
US9984054B2 (en) 2011-08-24 2018-05-29 Sdl Inc. Web interface including the review and manipulation of a web document and utilizing permission based control
US10061749B2 (en) 2011-01-29 2018-08-28 Sdl Netherlands B.V. Systems and methods for contextual vocabularies and customer segmentation
US10140320B2 (en) 2011-02-28 2018-11-27 Sdl Inc. Systems, methods, and media for generating analytical data
US10198438B2 (en) 1999-09-17 2019-02-05 Sdl Inc. E-services translation utilizing machine translation and translation memory
US10216726B2 (en) 2015-07-31 2019-02-26 Samsung Electronics Co., Ltd. Apparatus and method for determining translation word
US10248650B2 (en) 2004-03-05 2019-04-02 Sdl Inc. In-context exact (ICE) matching
US10261994B2 (en) 2012-05-25 2019-04-16 Sdl Inc. Method and system for automatic management of reputation of translators
US10319252B2 (en) 2005-11-09 2019-06-11 Sdl Inc. Language capability assessment and training apparatus and techniques
US10417646B2 (en) 2010-03-09 2019-09-17 Sdl Inc. Predicting the cost associated with translating textual content
US10452740B2 (en) 2012-09-14 2019-10-22 Sdl Netherlands B.V. External content libraries
US10572928B2 (en) 2012-05-11 2020-02-25 Fredhopper B.V. Method and system for recommending products based on a ranking cocktail
US10580015B2 (en) 2011-02-25 2020-03-03 Sdl Netherlands B.V. Systems, methods, and media for executing and optimizing online marketing initiatives
US10614167B2 (en) 2015-10-30 2020-04-07 Sdl Plc Translation review workflow systems and methods
US10635863B2 (en) 2017-10-30 2020-04-28 Sdl Inc. Fragment recall and adaptive automated translation
US10657540B2 (en) 2011-01-29 2020-05-19 Sdl Netherlands B.V. Systems, methods, and media for web content management
US10817676B2 (en) 2017-12-27 2020-10-27 Sdl Inc. Intelligent routing services and systems
US11256867B2 (en) 2018-10-09 2022-02-22 Sdl Inc. Systems and methods of machine learning for digital assets and message creation
US11308528B2 (en) 2012-09-14 2022-04-19 Sdl Netherlands B.V. Blueprinting of multimedia assets
US11386186B2 (en) 2012-09-14 2022-07-12 Sdl Netherlands B.V. External content library connector systems and methods

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7024666B1 (en) * 2002-01-28 2006-04-04 Roy-G-Biv Corporation Motion control systems and methods
US6076051A (en) 1997-03-07 2000-06-13 Microsoft Corporation Information retrieval utilizing semantic representation of text
US20010032278A1 (en) 1997-10-07 2001-10-18 Brown Stephen J. Remote generation and distribution of command programs for programmable devices
US7050964B2 (en) 2001-06-01 2006-05-23 Microsoft Corporation Scaleable machine translation system
US7191115B2 (en) 2001-06-20 2007-03-13 Microsoft Corporation Statistical method and apparatus for learning translation relationships among words
US20030074188A1 (en) * 2001-10-12 2003-04-17 Tohgo Murata Method and apparatus for language instruction
EP1306775A1 (en) * 2001-10-29 2003-05-02 BRITISH TELECOMMUNICATIONS public limited company Machine translation
JP3973549B2 (ja) * 2002-12-19 2007-09-12 沖電気工業株式会社 対訳依存構造対応付け装置、方法及びプログラム、並びに、対訳依存構造対応付けプログラムを記録した記録媒体
US7356457B2 (en) * 2003-02-28 2008-04-08 Microsoft Corporation Machine translation using learned word associations without referring to a multi-lingual human authored dictionary of content words
US8108534B2 (en) * 2003-05-08 2012-01-31 Jda Software Group, Inc. Data integration system with programmatic source and target interfaces
AU2004202391A1 (en) * 2003-06-20 2005-01-13 Microsoft Corporation Adaptive machine translation
US8027349B2 (en) 2003-09-25 2011-09-27 Roy-G-Biv Corporation Database event driven motion systems
US20060064503A1 (en) * 2003-09-25 2006-03-23 Brown David W Data routing systems and methods
US7689412B2 (en) * 2003-12-05 2010-03-30 Microsoft Corporation Synonymous collocation extraction using translation information
KR100561227B1 (ko) 2003-12-19 2006-03-15 한국전자통신연구원 한-중 기계번역시스템의 용언구 패턴 자동 확장 및 검증장치 및 방법
JP3790825B2 (ja) * 2004-01-30 2006-06-28 独立行政法人情報通信研究機構 他言語のテキスト生成装置
US7593843B2 (en) * 2004-03-30 2009-09-22 Microsoft Corporation Statistical language model for logical form using transfer mappings
JP4404211B2 (ja) 2005-03-14 2010-01-27 富士ゼロックス株式会社 マルチリンガル翻訳メモリ、翻訳方法および翻訳プログラム
US20060282255A1 (en) * 2005-06-14 2006-12-14 Microsoft Corporation Collocation translation from monolingual and available bilingual corpora
US7680647B2 (en) * 2005-06-21 2010-03-16 Microsoft Corporation Association-based bilingual word alignment
US20070016397A1 (en) * 2005-07-18 2007-01-18 Microsoft Corporation Collocation translation using monolingual corpora
US7957953B2 (en) * 2005-10-03 2011-06-07 Microsoft Corporation Weighted linear bilingual word alignment model
US20070083357A1 (en) * 2005-10-03 2007-04-12 Moore Robert C Weighted linear model
US7725306B2 (en) * 2006-06-28 2010-05-25 Microsoft Corporation Efficient phrase pair extraction from bilingual word alignments
US7805289B2 (en) * 2006-07-10 2010-09-28 Microsoft Corporation Aligning hierarchal and sequential document trees to identify parallel data
JP5342760B2 (ja) * 2007-09-03 2013-11-13 株式会社東芝 訳語学習のためのデータを作成する装置、方法、およびプログラム
AU2008333773B2 (en) 2007-12-05 2013-09-26 Facebook, Inc. Community translation on a social network
CN101996166B (zh) * 2009-08-14 2015-08-05 张龙哺 双语句对模式化记录方法以及翻译方法和翻译系统
CN101908042B (zh) * 2010-08-09 2016-04-13 中国科学院自动化研究所 一种双语联合语义角色的标注方法
KR101356417B1 (ko) * 2010-11-05 2014-01-28 고려대학교 산학협력단 병렬 말뭉치를 이용한 동사구 번역 패턴 구축 장치 및 그 방법
KR20120089502A (ko) * 2010-12-09 2012-08-13 한국전자통신연구원 번역지식 서버 생성 방법 및 그 장치
US20120158398A1 (en) * 2010-12-17 2012-06-21 John Denero Combining Model-Based Aligner Using Dual Decomposition
US8903707B2 (en) 2012-01-12 2014-12-02 International Business Machines Corporation Predicting pronouns of dropped pronoun style languages for natural language translation
KR101694286B1 (ko) * 2012-05-02 2017-01-09 한국전자통신연구원 양방향 자동 통역 및 번역 서비스 제공 장치 및 그 방법
US9519639B2 (en) 2012-06-08 2016-12-13 Facebook, Inc. Community translation of user-generated content
US9779087B2 (en) * 2013-12-13 2017-10-03 Google Inc. Cross-lingual discriminative learning of sequence models with posterior regularization
JP2019082860A (ja) * 2017-10-30 2019-05-30 富士通株式会社 生成プログラム、生成方法及び生成装置
US11276391B2 (en) * 2020-02-06 2022-03-15 International Business Machines Corporation Generation of matched corpus for language model training
CN111476739B (zh) * 2020-04-17 2023-04-18 宁波大学科学技术学院 水下图像增强方法、系统及存储介质

Citations (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0304191A2 (en) 1987-08-14 1989-02-22 International Business Machines Corporation Text search system
US4839853A (en) 1988-09-15 1989-06-13 Bell Communications Research, Inc. Computer information retrieval using latent semantic structure
EP0386825A1 (en) 1989-03-10 1990-09-12 Bso/Buro Voor Systeemontwikkeling B.V. Method for determining the semantic relatedness of lexical items in a text
US5237502A (en) 1990-09-04 1993-08-17 International Business Machines Corporation Method and apparatus for paraphrasing information contained in logical forms
US5278980A (en) 1991-08-16 1994-01-11 Xerox Corporation Iterative technique for phrase query formation and an information retrieval system employing same
US5325298A (en) 1990-11-07 1994-06-28 Hnc, Inc. Methods for generating or revising context vectors for a plurality of word stems
US5377103A (en) 1992-05-15 1994-12-27 International Business Machines Corporation Constrained natural language interface for a computer that employs a browse function
US5424947A (en) * 1990-06-15 1995-06-13 International Business Machines Corporation Natural language analyzing apparatus and method, and construction of a knowledge base for natural language analysis
US5477450A (en) * 1993-02-23 1995-12-19 International Business Machines Corporation Machine translation method and apparatus
EP0687987A1 (en) 1994-06-16 1995-12-20 Xerox Corporation A method and apparatus for retrieving relevant documents from a corpus of documents
US5592661A (en) 1992-07-16 1997-01-07 International Business Machines Corporation Detection of independent changes via change identifiers in a versioned database management system
US5619709A (en) 1993-09-20 1997-04-08 Hnc, Inc. System and method of context vector generation and retrieval
US5675745A (en) 1995-02-13 1997-10-07 Fujitsu Limited Constructing method of organization activity database, analysis sheet used therein, and organization activity management system
EP0805404A1 (en) 1996-05-02 1997-11-05 Microsoft Corporation Method and system for lexical processing of uppercase and unaccented text
US5724594A (en) 1994-02-10 1998-03-03 Microsoft Corporation Method and system for automatically identifying morphological information from a machine-readable dictionary
US5794050A (en) 1995-01-04 1998-08-11 Intelligent Text Processing, Inc. Natural language understanding system
US5799308A (en) 1993-10-04 1998-08-25 Dixon; Robert Method and apparatus for data storage and retrieval
US5873056A (en) 1993-10-12 1999-02-16 The Syracuse University Natural language processing system for semantic vector representation which accounts for lexical ambiguity
US5884247A (en) 1996-10-31 1999-03-16 Dialect Corporation Method and apparatus for automated language translation
US5930746A (en) * 1996-03-20 1999-07-27 The Government Of Singapore Parsing and translating natural language sentences automatically
US5933833A (en) 1995-10-03 1999-08-03 International Business Machines Corp. Data table structure and calculation method for mathematical calculations of irregular cells
US6006221A (en) 1995-08-16 1999-12-21 Syracuse University Multilingual document retrieval system and method using semantic vector matching
US6076051A (en) 1997-03-07 2000-06-13 Microsoft Corporation Information retrieval utilizing semantic representation of text
US6098033A (en) 1997-07-31 2000-08-01 Microsoft Corporation Determining similarity between words
US6243669B1 (en) * 1999-01-29 2001-06-05 Sony Corporation Method and apparatus for providing syntactic analysis and data structure for translation knowledge in example-based language translation
US20020040292A1 (en) * 2000-05-11 2002-04-04 Daniel Marcu Machine translation techniques
EP1262879A1 (en) 2001-06-01 2002-12-04 Microsoft Corporation Automatic extraction of transfer mappings from bilingual corpora
EP1262880A2 (en) 2001-06-01 2002-12-04 Microsoft Corporation Scaleable machine translation system

Patent Citations (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0304191A2 (en) 1987-08-14 1989-02-22 International Business Machines Corporation Text search system
US4839853A (en) 1988-09-15 1989-06-13 Bell Communications Research, Inc. Computer information retrieval using latent semantic structure
EP0386825A1 (en) 1989-03-10 1990-09-12 Bso/Buro Voor Systeemontwikkeling B.V. Method for determining the semantic relatedness of lexical items in a text
US5424947A (en) * 1990-06-15 1995-06-13 International Business Machines Corporation Natural language analyzing apparatus and method, and construction of a knowledge base for natural language analysis
US5237502A (en) 1990-09-04 1993-08-17 International Business Machines Corporation Method and apparatus for paraphrasing information contained in logical forms
US5325298A (en) 1990-11-07 1994-06-28 Hnc, Inc. Methods for generating or revising context vectors for a plurality of word stems
US5278980A (en) 1991-08-16 1994-01-11 Xerox Corporation Iterative technique for phrase query formation and an information retrieval system employing same
US5377103A (en) 1992-05-15 1994-12-27 International Business Machines Corporation Constrained natural language interface for a computer that employs a browse function
US5592661A (en) 1992-07-16 1997-01-07 International Business Machines Corporation Detection of independent changes via change identifiers in a versioned database management system
US5477450A (en) * 1993-02-23 1995-12-19 International Business Machines Corporation Machine translation method and apparatus
US5619709A (en) 1993-09-20 1997-04-08 Hnc, Inc. System and method of context vector generation and retrieval
US5794178A (en) 1993-09-20 1998-08-11 Hnc Software, Inc. Visualization of information using graphical representations of context vector based relationships and attributes
US5799308A (en) 1993-10-04 1998-08-25 Dixon; Robert Method and apparatus for data storage and retrieval
US5873056A (en) 1993-10-12 1999-02-16 The Syracuse University Natural language processing system for semantic vector representation which accounts for lexical ambiguity
US5724594A (en) 1994-02-10 1998-03-03 Microsoft Corporation Method and system for automatically identifying morphological information from a machine-readable dictionary
EP0687987A1 (en) 1994-06-16 1995-12-20 Xerox Corporation A method and apparatus for retrieving relevant documents from a corpus of documents
US5675819A (en) 1994-06-16 1997-10-07 Xerox Corporation Document information retrieval using global word co-occurrence patterns
US5794050A (en) 1995-01-04 1998-08-11 Intelligent Text Processing, Inc. Natural language understanding system
US5675745A (en) 1995-02-13 1997-10-07 Fujitsu Limited Constructing method of organization activity database, analysis sheet used therein, and organization activity management system
US6006221A (en) 1995-08-16 1999-12-21 Syracuse University Multilingual document retrieval system and method using semantic vector matching
US5933833A (en) 1995-10-03 1999-08-03 International Business Machines Corp. Data table structure and calculation method for mathematical calculations of irregular cells
US5930746A (en) * 1996-03-20 1999-07-27 The Government Of Singapore Parsing and translating natural language sentences automatically
EP0805404A1 (en) 1996-05-02 1997-11-05 Microsoft Corporation Method and system for lexical processing of uppercase and unaccented text
US5884247A (en) 1996-10-31 1999-03-16 Dialect Corporation Method and apparatus for automated language translation
US6076051A (en) 1997-03-07 2000-06-13 Microsoft Corporation Information retrieval utilizing semantic representation of text
US6871174B1 (en) 1997-03-07 2005-03-22 Microsoft Corporation System and method for matching a textual input to a lexical knowledge base and for utilizing results of that match
US6098033A (en) 1997-07-31 2000-08-01 Microsoft Corporation Determining similarity between words
US6243669B1 (en) * 1999-01-29 2001-06-05 Sony Corporation Method and apparatus for providing syntactic analysis and data structure for translation knowledge in example-based language translation
US20020040292A1 (en) * 2000-05-11 2002-04-04 Daniel Marcu Machine translation techniques
EP1262879A1 (en) 2001-06-01 2002-12-04 Microsoft Corporation Automatic extraction of transfer mappings from bilingual corpora
EP1262880A2 (en) 2001-06-01 2002-12-04 Microsoft Corporation Scaleable machine translation system

Non-Patent Citations (36)

* Cited by examiner, † Cited by third party
Title
Agirre, Eneko, and German Rigau, "Word Sense Disambiguation Using Conceptual Density," in Proceedings of COLING 96, 1996, pp. 16-23.
Braden-Harder, Lisa, Natural Language Processing: the PLNLP Approach, Kluwer Academic Publishers, Boston, MA, 1993, Chap. 19, "Sense Disambiguation Using Online Dictionaries," pp. 247-261.
Bro'na Collins et al. Adaption-Guided Retrieval in EBMT: A Case-Based Approach to Machine Translation--1996 XP002209833 p. 91-104.
Brown et al., "Class-Based n-gram Models of Natural Language," Computational Linguistics 18(4):467-479, Dec. 1992.
Collins, B. et al. "Adaption- guided Retrieval in EBMT: a case-based approach to machine translation". Ewcbr-96 Switzerland pp. 14-16. 1996.
Dagan et al., "Contextual Word Similarity and Esimation From Sparse Data," in Proceedings of the 31st Annual Meeting of the Assoc. For Computational Linguistics, Columbus, OH, Jun. 2-26, 1993, pp. 164-171.
Dagan et al., "Similarity-Based Estimation of Word Coocurrence Probabilities," in Proceedings of the 32nd Annual Meeting of the ACL, 1994, pp. 272-278. cited by other.
Elsbury, J.M., Personal Authentication Devices-Data Security Applications p. 471-478 1989.
Experiments in Automatic Phase Indexing for Document Retrieval: A Comparison of Syntactic and Nonsyntactic Methods, by Joel L. Fagan Degree Date: 1988.
Furuse, Osamu, and Hitoshi Iida, "An Example-Based Method for Transfer-Driven Machine Translation,"In Proc. Of the 4th International Conference on Theoretical and Methodological Issues in Machine Translation, Montreal, Quebec, Canada, 1992, pp. 139-150.
Grishman, Ralph, and John Sterling, "Generalizing Automatically Generated Seiectional patterns," in Proceedings of the 15th International Conference on Computational Linguistics, Kyoto, Japan, Aug. 5-9, 1994, pp. 742-747.
HabichIer, Jurgen: "Experts (EDI/XML Procurement Enabling Real Trade Standards)" May 1999. pp. 1-13.
Hearst, Marti A., and Gregory Grefenstette, Refining Automatically-Discovered Lexical Relations: Combining Weak Techniques for Stronger Results,: in Papers From the 1992 AAAI Workshop, Menlo Park, CA 1992, pp. 64-72.
Hindle, Donald, "Noun Classification From Predicate-Argument Structures," in Proceedings of the 28th Annual Meetings of the ACL, Pittsburg, PA. Jun. 6-9, 1990, pp. 268-275.
James Allen, "Natural Language Understanding" the Benjamin/Cumming Publishing Company, Inc. Semantic Forms and Logic, Chapter 8. pp. 227-260, 1995.
Kozima, Hideki, and Teiji Furugori, "Similarity Between Words Computed by Spreading Activation on an English Dictionary," in Proceedings of the 6th Conference of the European Chapter of the ACL, Utrecht, Germany, 1993, pp. 232-240.
Meyers et al. "Alignment of Shared Forests for Bilingual Corpora", Online! XP-002210231, 1966.
Meyers, Adam et al., "Alignment of Shared Forests for Bilingual Corpora". p. 1-6 (1996).
Michael, Carl: "Inducing Translation Templates for Example-Based Machine Translation". Proceedings of Mt-Summit VII. 1999. URL:http://www.iai.uni-sb.de/docs/edgar.pdf paragraph 03.2-last paragraph. cited by other. Habichler, Jurgen: "Experts (EDI/XML Procurement Enabling Real Trade Standards)" May 1999. pp. 1-13.
Oz, Z et al. "Ordering translation templates by assigning confidence factors". Machine Translation and the Information Soup. 1998 Berlin AMTA pp. 51-61.
Patent Abstracts of Japan, Japanese Patent Application Laid-open No. 2003-030185, published Jan. 31, 2003.
Pereira et al., "Distributional Clustering of English Words," in Proceedings of the 31st Annual Meeting of Assoc. For Computational Linguistics, Columbus, OH, Jun. 22-26, 1993, pp. 183-190.
Resnik, Philip, "Disambiguating Noun Groupings With Respect to WordNet Sense," in Proceedings of the 3rd Workshop on Very Large Corpora, Boston, MA., Jun. 30, 1995, pp. 1-16. cited by other.
Sadler, Victor, Working With Analogical Semantics: Disambiguation Techniques in DLT, Foris Publications, Dordrecht, Holland, 1989, entire book. cited by other.
Salton, Gerard, and Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill Publishing Co., New York, NY 1983, entire book.
Salton, Gerard: "Automatic Information Organization and Retrieval", McGraw Hill Book Company, pp. 168-178, 1968. cited by other.
Sato, Satoshi, "Example-Based Machine Translation," in Proceedings of the International Workshop on Fundamental Research for the Future Generation of Natural Language Processing, Kyoto, Japan, Sep. 1991, pp. 1-16.
Somers "Review Article: Example-based Machine Translation", XP-001092717, vol. 14, 1999, pp. 113-157.
Somers, Harold "Review Article: Example-based Machine Translation" 2001 Kluwer Academic Pub. Printed in the Netherlands p. 113-157.
Sumita, Eiichiro, and Hotioshi Iida, "Experiments and Prospects of Example-Based Machine Translation," in Proceedings of the 29th Annual Meetings of the ACL, 1991, pp. 185-192.
Tsutsumi, Taijiro, Natural Language Processing: the PLNLP Approach, Kluwer Academic Publishers, Boston, MA, 1993, Chap. 20, "Word-Sense Disambiguation by Examples," pp. 263-272.
Uramoto, Naohiko, "A Best-Match Algorithm for Broad-Coverage Example-Based Disambiguation," in Proceedings of the 15th International Conference on Computational Linguistics, Kyoto, Japan, Aug. 5-9, 1994, pp. 717-721.
Utsuro et al., "Thesaurus-Based Efficient Example Retrieval by Generating Retrieval Queries From Similarities," in Proceedings of the 15th International Conference on Computational Linguistics, Kyoto, Japan, Aug. 509, 1994, pp. 1044-1048.
Van Zuijlen, Job. M., "Probabilistic Methods in Dependency Grammar Parsing": International Parsing Workshop. 10375:142-141, 1989.
Wilks et al., "Providing Machine Tractable Dictonary Tools," Machine Translations 5:99-154, 1990.
Yarowsky, David, "Word-Sense Disambiguation Using Statistical Models of Roget's Categories Trained on Large Corpora," in Proceedings in the 15th Int'l. Conference on Computational Linguistics, Nantes, France, Aug. 23-28, 1992, pp. 454-460.

Cited By (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10198438B2 (en) 1999-09-17 2019-02-05 Sdl Inc. E-services translation utilizing machine translation and translation memory
US10216731B2 (en) 1999-09-17 2019-02-26 Sdl Inc. E-services translation utilizing machine translation and translation memory
US9954794B2 (en) 2001-01-18 2018-04-24 Sdl Inc. Globalization management system and method therefor
US8275605B2 (en) 2001-06-01 2012-09-25 Microsoft Corporation Machine language translation with transfer mappings having varying context
US20100223049A1 (en) * 2001-06-01 2010-09-02 Microsoft Corporation Machine language translation with transfer mappings having varying context
US10248650B2 (en) 2004-03-05 2019-04-02 Sdl Inc. In-context exact (ICE) matching
US10319252B2 (en) 2005-11-09 2019-06-11 Sdl Inc. Language capability assessment and training apparatus and techniques
US20110257962A1 (en) * 2008-10-22 2011-10-20 Kumar Bulusu Gopi System and method for automatically generating sentences of a language
US8849650B2 (en) * 2008-10-22 2014-09-30 Sankhya Technologies Private Limited System and method for automatically generating sentences of a language
US10984429B2 (en) 2010-03-09 2021-04-20 Sdl Inc. Systems and methods for translating textual content
US10417646B2 (en) 2010-03-09 2019-09-17 Sdl Inc. Predicting the cost associated with translating textual content
US10521492B2 (en) 2011-01-29 2019-12-31 Sdl Netherlands B.V. Systems and methods that utilize contextual vocabularies and customer segmentation to deliver web content
US10657540B2 (en) 2011-01-29 2020-05-19 Sdl Netherlands B.V. Systems, methods, and media for web content management
US11694215B2 (en) 2011-01-29 2023-07-04 Sdl Netherlands B.V. Systems and methods for managing web content
US10061749B2 (en) 2011-01-29 2018-08-28 Sdl Netherlands B.V. Systems and methods for contextual vocabularies and customer segmentation
US11301874B2 (en) 2011-01-29 2022-04-12 Sdl Netherlands B.V. Systems and methods for managing web content and facilitating data exchange
US11044949B2 (en) 2011-01-29 2021-06-29 Sdl Netherlands B.V. Systems and methods for dynamic delivery of web content
US10990644B2 (en) 2011-01-29 2021-04-27 Sdl Netherlands B.V. Systems and methods for contextual vocabularies and customer segmentation
US10580015B2 (en) 2011-02-25 2020-03-03 Sdl Netherlands B.V. Systems, methods, and media for executing and optimizing online marketing initiatives
US10140320B2 (en) 2011-02-28 2018-11-27 Sdl Inc. Systems, methods, and media for generating analytical data
US11366792B2 (en) 2011-02-28 2022-06-21 Sdl Inc. Systems, methods, and media for generating analytical data
US11263390B2 (en) 2011-08-24 2022-03-01 Sdl Inc. Systems and methods for informational document review, display and validation
US9984054B2 (en) 2011-08-24 2018-05-29 Sdl Inc. Web interface including the review and manipulation of a web document and utilizing permission based control
US10572928B2 (en) 2012-05-11 2020-02-25 Fredhopper B.V. Method and system for recommending products based on a ranking cocktail
US10402498B2 (en) 2012-05-25 2019-09-03 Sdl Inc. Method and system for automatic management of reputation of translators
US10261994B2 (en) 2012-05-25 2019-04-16 Sdl Inc. Method and system for automatic management of reputation of translators
US10452740B2 (en) 2012-09-14 2019-10-22 Sdl Netherlands B.V. External content libraries
US11386186B2 (en) 2012-09-14 2022-07-12 Sdl Netherlands B.V. External content library connector systems and methods
US11308528B2 (en) 2012-09-14 2022-04-19 Sdl Netherlands B.V. Blueprinting of multimedia assets
US9916306B2 (en) 2012-10-19 2018-03-13 Sdl Inc. Statistical linguistic analysis of source content
US10216726B2 (en) 2015-07-31 2019-02-26 Samsung Electronics Co., Ltd. Apparatus and method for determining translation word
US11080493B2 (en) 2015-10-30 2021-08-03 Sdl Limited Translation review workflow systems and methods
US10614167B2 (en) 2015-10-30 2020-04-07 Sdl Plc Translation review workflow systems and methods
US11321540B2 (en) 2017-10-30 2022-05-03 Sdl Inc. Systems and methods of adaptive automated translation utilizing fine-grained alignment
US10635863B2 (en) 2017-10-30 2020-04-28 Sdl Inc. Fragment recall and adaptive automated translation
US11475227B2 (en) 2017-12-27 2022-10-18 Sdl Inc. Intelligent routing services and systems
US10817676B2 (en) 2017-12-27 2020-10-27 Sdl Inc. Intelligent routing services and systems
US11256867B2 (en) 2018-10-09 2022-02-22 Sdl Inc. Systems and methods of machine learning for digital assets and message creation

Also Published As

Publication number Publication date
EP1262879A1 (en) 2002-12-04
JP2003030185A (ja) 2003-01-31
US20030061023A1 (en) 2003-03-27
JP5538820B2 (ja) 2014-07-02
JP2010027088A (ja) 2010-02-04
JP2007328815A (ja) 2007-12-20
US8275605B2 (en) 2012-09-25
EP1262879B1 (en) 2018-07-25
US20100223049A1 (en) 2010-09-02
JP4494706B2 (ja) 2010-06-30

Similar Documents

Publication Publication Date Title
US7734459B2 (en) Automatic extraction of transfer mappings from bilingual corpora
US7206735B2 (en) Scaleable machine translation
US7356457B2 (en) Machine translation using learned word associations without referring to a multi-lingual human authored dictionary of content words
EP1422634B1 (en) Statistical method and apparatus for statistical learning of translation relationships among phrases
EP1308851B1 (en) Method of calculating translation relationships among words of different languages
US5895446A (en) Pattern-based translation method and system
US7319949B2 (en) Unilingual translator
US20120239378A1 (en) Methods and Systems for Alignment of Parallel Text Corpora
JP2008262587A (ja) 用例ベースの機械翻訳システム
Dang Investigations into the role of lexical semantics in word sense disambiguation
Menezes et al. Syntactic models for structural word insertion and deletion during translation
Horvat Hierarchical statistical semantic translation and realization
KR950013129B1 (ko) 기계번역장치 및 방법
Ambati et al. Cross lingual syntax projection for resource-poor languages
Slayden et al. Large-scale Thai statistical machine translation
JPH09311858A (ja) 文章の照応関係解析装置
JP2004163991A (ja) ソースシーケンスから情報を復元するための方法およびプログラム

Legal Events

Date Code Title Description
AS Assignment

Owner name: MICROSOFT CORPORATION, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MENEZES, ARUL A.;RICHARDSON, STEPHEN D.;REEL/FRAME:012450/0376

Effective date: 20011011

Owner name: MICROSOFT CORPORATION,WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MENEZES, ARUL A.;RICHARDSON, STEPHEN D.;REEL/FRAME:012450/0376

Effective date: 20011011

STCF Information on status: patent grant

Free format text: PATENTED CASE

FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034541/0001

Effective date: 20141014

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552)

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20220608