WO2004038606A1 - Scalable neural network-based language identification from written text - Google Patents
Scalable neural network-based language identification from written text Download PDFInfo
- Publication number
- WO2004038606A1 WO2004038606A1 PCT/IB2003/002894 IB0302894W WO2004038606A1 WO 2004038606 A1 WO2004038606 A1 WO 2004038606A1 IB 0302894 W IB0302894 W IB 0302894W WO 2004038606 A1 WO2004038606 A1 WO 2004038606A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- alphabet characters
- string
- language
- alphabet
- languages
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/263—Language identification
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
- Document Processing Apparatus (AREA)
Abstract
Description
Claims
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2003253112A AU2003253112A1 (en) | 2002-10-22 | 2003-07-21 | Scalable neural network-based language identification from written text |
BR0314865-3A BR0314865A (en) | 2002-10-22 | 2003-07-21 | Method and system for identifying the language of a series of alphabet characters from a plurality of languages based on an automatic language identification system and electronic device |
CN038244195A CN1688999B (en) | 2002-10-22 | 2003-07-21 | Scalable neural network-based language identification from written text |
EP03809382A EP1554670A4 (en) | 2002-10-22 | 2003-07-21 | Scalable neural network-based language identification from written text |
JP2004546223A JP2006504173A (en) | 2002-10-22 | 2003-07-21 | Scalable neural network based language identification from document text |
CA002500467A CA2500467A1 (en) | 2002-10-22 | 2003-07-21 | Scalable neural network-based language identification from written text |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/279,747 | 2002-10-22 | ||
US10/279,747 US20040078191A1 (en) | 2002-10-22 | 2002-10-22 | Scalable neural network-based language identification from written text |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2004038606A1 true WO2004038606A1 (en) | 2004-05-06 |
Family
ID=32093450
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/IB2003/002894 WO2004038606A1 (en) | 2002-10-22 | 2003-07-21 | Scalable neural network-based language identification from written text |
Country Status (9)
Country | Link |
---|---|
US (1) | US20040078191A1 (en) |
EP (1) | EP1554670A4 (en) |
JP (2) | JP2006504173A (en) |
KR (1) | KR100714769B1 (en) |
CN (1) | CN1688999B (en) |
AU (1) | AU2003253112A1 (en) |
BR (1) | BR0314865A (en) |
CA (1) | CA2500467A1 (en) |
WO (1) | WO2004038606A1 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012174736A1 (en) * | 2011-06-24 | 2012-12-27 | Google Inc. | Detecting source languages of search queries |
Families Citing this family (54)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10334400A1 (en) * | 2003-07-28 | 2005-02-24 | Siemens Ag | Method for speech recognition and communication device |
US7395319B2 (en) * | 2003-12-31 | 2008-07-01 | Checkfree Corporation | System using contact list to identify network address for accessing electronic commerce application |
US7640159B2 (en) * | 2004-07-22 | 2009-12-29 | Nuance Communications, Inc. | System and method of speech recognition for non-native speakers of a language |
DE102004042907A1 (en) * | 2004-09-01 | 2006-03-02 | Deutsche Telekom Ag | Online multimedia crossword puzzle |
US7840399B2 (en) * | 2005-04-07 | 2010-11-23 | Nokia Corporation | Method, device, and computer program product for multi-lingual speech recognition |
US7548849B2 (en) * | 2005-04-29 | 2009-06-16 | Research In Motion Limited | Method for generating text that meets specified characteristics in a handheld electronic device and a handheld electronic device incorporating the same |
US7552045B2 (en) * | 2006-12-18 | 2009-06-23 | Nokia Corporation | Method, apparatus and computer program product for providing flexible text based language identification |
US8949266B2 (en) | 2007-03-07 | 2015-02-03 | Vlingo Corporation | Multiple web-based content category searching in mobile search application |
US8886540B2 (en) * | 2007-03-07 | 2014-11-11 | Vlingo Corporation | Using speech recognition results based on an unstructured language model in a mobile communication facility application |
US20110054895A1 (en) * | 2007-03-07 | 2011-03-03 | Phillips Michael S | Utilizing user transmitted text to improve language model in mobile dictation application |
US10056077B2 (en) * | 2007-03-07 | 2018-08-21 | Nuance Communications, Inc. | Using speech recognition results based on an unstructured language model with a music system |
US20110054899A1 (en) * | 2007-03-07 | 2011-03-03 | Phillips Michael S | Command and control utilizing content information in a mobile voice-to-speech application |
US8949130B2 (en) * | 2007-03-07 | 2015-02-03 | Vlingo Corporation | Internal and external speech recognition use with a mobile communication facility |
US20090030685A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using speech recognition results based on an unstructured language model with a navigation system |
US8838457B2 (en) * | 2007-03-07 | 2014-09-16 | Vlingo Corporation | Using results of unstructured language model based speech recognition to control a system-level function of a mobile communications facility |
US20080221901A1 (en) * | 2007-03-07 | 2008-09-11 | Joseph Cerra | Mobile general search environment speech processing facility |
US20110054897A1 (en) * | 2007-03-07 | 2011-03-03 | Phillips Michael S | Transmitting signal quality information in mobile dictation application |
US20080221884A1 (en) * | 2007-03-07 | 2008-09-11 | Cerra Joseph P | Mobile environment speech processing facility |
US20090030697A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using contextual information for delivering results generated from a speech recognition facility using an unstructured language model |
US20090030691A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Using an unstructured language model associated with an application of a mobile communication facility |
US20090030688A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Tagging speech recognition results based on an unstructured language model for use in a mobile communication facility application |
US20110060587A1 (en) * | 2007-03-07 | 2011-03-10 | Phillips Michael S | Command and control utilizing ancillary information in a mobile voice-to-speech application |
US8635243B2 (en) * | 2007-03-07 | 2014-01-21 | Research In Motion Limited | Sending a communications header with voice recording to send metadata for use in speech recognition, formatting, and search mobile search application |
US20090030687A1 (en) * | 2007-03-07 | 2009-01-29 | Cerra Joseph P | Adapting an unstructured language model speech recognition system based on usage |
US20110054896A1 (en) * | 2007-03-07 | 2011-03-03 | Phillips Michael S | Sending a communications header with voice recording to send metadata for use in speech recognition and formatting in mobile dictation application |
US20110054898A1 (en) * | 2007-03-07 | 2011-03-03 | Phillips Michael S | Multiple web-based content search user interface in mobile search application |
US8886545B2 (en) | 2007-03-07 | 2014-11-11 | Vlingo Corporation | Dealing with switch latency in speech recognition |
JP5246751B2 (en) * | 2008-03-31 | 2013-07-24 | 独立行政法人理化学研究所 | Information processing apparatus, information processing method, and program |
US8266514B2 (en) | 2008-06-26 | 2012-09-11 | Microsoft Corporation | Map service |
US8073680B2 (en) * | 2008-06-26 | 2011-12-06 | Microsoft Corporation | Language detection service |
US8019596B2 (en) * | 2008-06-26 | 2011-09-13 | Microsoft Corporation | Linguistic service platform |
US8107671B2 (en) * | 2008-06-26 | 2012-01-31 | Microsoft Corporation | Script detection service |
US8311824B2 (en) * | 2008-10-27 | 2012-11-13 | Nice-Systems Ltd | Methods and apparatus for language identification |
US8224641B2 (en) * | 2008-11-19 | 2012-07-17 | Stratify, Inc. | Language identification for documents containing multiple languages |
US8224642B2 (en) * | 2008-11-20 | 2012-07-17 | Stratify, Inc. | Automated identification of documents as not belonging to any language |
US8868431B2 (en) | 2010-02-05 | 2014-10-21 | Mitsubishi Electric Corporation | Recognition dictionary creation device and voice recognition device |
JP5259020B2 (en) * | 2010-10-01 | 2013-08-07 | 三菱電機株式会社 | Voice recognition device |
GB201216640D0 (en) * | 2012-09-18 | 2012-10-31 | Touchtype Ltd | Formatting module, system and method for formatting an electronic character sequence |
CN103578471B (en) * | 2013-10-18 | 2017-03-01 | 威盛电子股份有限公司 | Speech identifying method and its electronic installation |
US9195656B2 (en) * | 2013-12-30 | 2015-11-24 | Google Inc. | Multilingual prosody generation |
US20160035344A1 (en) * | 2014-08-04 | 2016-02-04 | Google Inc. | Identifying the language of a spoken utterance |
US9812128B2 (en) * | 2014-10-09 | 2017-11-07 | Google Inc. | Device leadership negotiation among voice interface devices |
US9858484B2 (en) * | 2014-12-30 | 2018-01-02 | Facebook, Inc. | Systems and methods for determining video feature descriptors based on convolutional neural networks |
US10417555B2 (en) | 2015-05-29 | 2019-09-17 | Samsung Electronics Co., Ltd. | Data-optimized neural network traversal |
US10474753B2 (en) * | 2016-09-07 | 2019-11-12 | Apple Inc. | Language identification using recurrent neural networks |
US10282415B2 (en) | 2016-11-29 | 2019-05-07 | Ebay Inc. | Language identification for text strings |
CN108288078B (en) * | 2017-12-07 | 2020-09-29 | 腾讯科技(深圳)有限公司 | Method, device and medium for recognizing characters in image |
CN108197087B (en) * | 2018-01-18 | 2021-11-16 | 奇安信科技集团股份有限公司 | Character code recognition method and device |
KR102123910B1 (en) * | 2018-04-12 | 2020-06-18 | 주식회사 푸른기술 | Serial number rcognition Apparatus and method for paper money using machine learning |
EP3564949A1 (en) * | 2018-04-23 | 2019-11-06 | Spotify AB | Activation trigger processing |
JP2020056972A (en) * | 2018-10-04 | 2020-04-09 | 富士通株式会社 | Language identification program, language identification method and language identification device |
WO2020226948A1 (en) * | 2019-05-03 | 2020-11-12 | Google Llc | Phoneme-based contextualization for cross-lingual speech recognition in end-to-end models |
US11720752B2 (en) * | 2020-07-07 | 2023-08-08 | Sap Se | Machine learning enabled text analysis with multi-language support |
US20220198155A1 (en) * | 2020-12-18 | 2022-06-23 | Capital One Services, Llc | Systems and methods for translating transaction descriptions |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5548507A (en) * | 1994-03-14 | 1996-08-20 | International Business Machines Corporation | Language identification process using coded language words |
US6016471A (en) * | 1998-04-29 | 2000-01-18 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus using decision trees to generate and score multiple pronunciations for a spelled word |
US6157905A (en) * | 1997-12-11 | 2000-12-05 | Microsoft Corporation | Identifying language and character set of data representing text |
US6167369A (en) * | 1998-12-23 | 2000-12-26 | Xerox Company | Automatic language identification using both N-gram and word information |
US6415250B1 (en) * | 1997-06-18 | 2002-07-02 | Novell, Inc. | System and method for identifying language using morphologically-based techniques |
Family Cites Families (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5062143A (en) * | 1990-02-23 | 1991-10-29 | Harris Corporation | Trigram-based method of language identification |
IL109268A (en) * | 1994-04-10 | 1999-01-26 | Advanced Recognition Tech | Pattern recognition method and system |
US6615168B1 (en) * | 1996-07-26 | 2003-09-02 | Sun Microsystems, Inc. | Multilingual agent for use in computer systems |
US6009382A (en) * | 1996-08-19 | 1999-12-28 | International Business Machines Corporation | Word storage table for natural language determination |
US6216102B1 (en) * | 1996-08-19 | 2001-04-10 | International Business Machines Corporation | Natural language determination using partial words |
CA2242065C (en) * | 1997-07-03 | 2004-12-14 | Henry C.A. Hyde-Thomson | Unified messaging system with automatic language identification for text-to-speech conversion |
JPH1139306A (en) * | 1997-07-16 | 1999-02-12 | Sony Corp | Processing system for multi-language information and its method |
US6047251A (en) * | 1997-09-15 | 2000-04-04 | Caere Corporation | Automatic language identification system for multilingual optical character recognition |
EP1016077B1 (en) * | 1997-09-17 | 2001-05-16 | Siemens Aktiengesellschaft | Method for determining the probability of the occurrence of a sequence of at least two words in a speech recognition process |
KR100509797B1 (en) * | 1998-04-29 | 2005-08-23 | 마쯔시다덴기산교 가부시키가이샤 | Method and apparatus using decision trees to generate and score multiple pronunciations for a spelled word |
JP2000148754A (en) * | 1998-11-13 | 2000-05-30 | Omron Corp | Multilingual system, multilingual processing method, and medium storing program for multilingual processing |
JP2000250905A (en) * | 1999-02-25 | 2000-09-14 | Fujitsu Ltd | Language processor and its program storage medium |
US6182148B1 (en) * | 1999-03-18 | 2001-01-30 | Walid, Inc. | Method and system for internationalizing domain names |
DE19963812A1 (en) * | 1999-12-30 | 2001-07-05 | Nokia Mobile Phones Ltd | Method for recognizing a language and for controlling a speech synthesis unit and communication device |
CN1144173C (en) * | 2000-08-16 | 2004-03-31 | 财团法人工业技术研究院 | Probability-guide fault-tolerant method for understanding natural languages |
US7277732B2 (en) * | 2000-10-13 | 2007-10-02 | Microsoft Corporation | Language input system for mobile devices |
FI20010644A (en) * | 2001-03-28 | 2002-09-29 | Nokia Corp | Specify the language of the character sequence |
US7191116B2 (en) * | 2001-06-19 | 2007-03-13 | Oracle International Corporation | Methods and systems for determining a language of a document |
-
2002
- 2002-10-22 US US10/279,747 patent/US20040078191A1/en not_active Abandoned
-
2003
- 2003-07-21 JP JP2004546223A patent/JP2006504173A/en not_active Withdrawn
- 2003-07-21 CN CN038244195A patent/CN1688999B/en not_active Expired - Fee Related
- 2003-07-21 BR BR0314865-3A patent/BR0314865A/en not_active IP Right Cessation
- 2003-07-21 KR KR1020057006862A patent/KR100714769B1/en not_active IP Right Cessation
- 2003-07-21 CA CA002500467A patent/CA2500467A1/en not_active Abandoned
- 2003-07-21 AU AU2003253112A patent/AU2003253112A1/en not_active Abandoned
- 2003-07-21 EP EP03809382A patent/EP1554670A4/en not_active Withdrawn
- 2003-07-21 WO PCT/IB2003/002894 patent/WO2004038606A1/en active Application Filing
-
2008
- 2008-09-18 JP JP2008239389A patent/JP2009037633A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5548507A (en) * | 1994-03-14 | 1996-08-20 | International Business Machines Corporation | Language identification process using coded language words |
US6415250B1 (en) * | 1997-06-18 | 2002-07-02 | Novell, Inc. | System and method for identifying language using morphologically-based techniques |
US6157905A (en) * | 1997-12-11 | 2000-12-05 | Microsoft Corporation | Identifying language and character set of data representing text |
US6016471A (en) * | 1998-04-29 | 2000-01-18 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus using decision trees to generate and score multiple pronunciations for a spelled word |
US6167369A (en) * | 1998-12-23 | 2000-12-26 | Xerox Company | Automatic language identification using both N-gram and word information |
Non-Patent Citations (1)
Title |
---|
See also references of EP1554670A4 * |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2012174736A1 (en) * | 2011-06-24 | 2012-12-27 | Google Inc. | Detecting source languages of search queries |
Also Published As
Publication number | Publication date |
---|---|
KR20050070073A (en) | 2005-07-05 |
JP2006504173A (en) | 2006-02-02 |
AU2003253112A1 (en) | 2004-05-13 |
CN1688999A (en) | 2005-10-26 |
BR0314865A (en) | 2005-08-02 |
US20040078191A1 (en) | 2004-04-22 |
JP2009037633A (en) | 2009-02-19 |
CA2500467A1 (en) | 2004-05-06 |
KR100714769B1 (en) | 2007-05-04 |
CN1688999B (en) | 2010-04-28 |
EP1554670A4 (en) | 2008-09-10 |
EP1554670A1 (en) | 2005-07-20 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP1554670A1 (en) | Scalable neural network-based language identification from written text | |
US10176804B2 (en) | Analyzing textual data | |
CN106598939B (en) | A kind of text error correction method and device, server, storage medium | |
US20180293228A1 (en) | Device and method for converting dialect into standard language | |
Antony et al. | Parts of speech tagging for Indian languages: a literature survey | |
CN105404621B (en) | A kind of method and system that Chinese character is read for blind person | |
US11961010B2 (en) | Method and apparatus for performing entity linking | |
CN113591483A (en) | Document-level event argument extraction method based on sequence labeling | |
Etaiwi et al. | Statistical Arabic name entity recognition approaches: A survey | |
Dien et al. | A maximum entropy approach for Vietnamese word segmentation | |
CN111401012A (en) | Text error correction method, electronic device and computer readable storage medium | |
Oliva et al. | A SMS normalization system integrating multiple grammatical resources | |
Tian et al. | Scalable neural network based language identification from written text | |
WO2022060439A1 (en) | Language autodetection from non-character sub-token signals | |
Munkhjargal et al. | Named entity recognition for Mongolian language | |
CN111767733A (en) | Document security classification discrimination method based on statistical word segmentation | |
JP2010277036A (en) | Speech data retrieval device | |
CN109344388A (en) | A kind of comment spam recognition methods, device and computer readable storage medium | |
Chowdhury et al. | Bangla grapheme to phoneme conversion using conditional random fields | |
Celikkaya et al. | A mobile assistant for Turkish | |
Gutkin et al. | Extensions to Brahmic script processing within the Nisaba library: new scripts, languages and utilities | |
Xydas et al. | Text normalization for the pronunciation of non-standard words in an inflected language | |
CN112560493B (en) | Named entity error correction method, named entity error correction device, named entity error correction computer equipment and named entity error correction storage medium | |
Narupiyakul et al. | Thai syllable-based information extraction using hidden Markov models | |
Singh et al. | Study of cognates among south asian languages for the purpose of building lexical resources |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
DFPE | Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2003809382 Country of ref document: EP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2500467 Country of ref document: CA |
|
WWE | Wipo information: entry into national phase |
Ref document number: 20038244195 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 1020057006862 Country of ref document: KR |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2004546223 Country of ref document: JP |
|
WWP | Wipo information: published in national office |
Ref document number: 1020057006862 Country of ref document: KR |
|
WWP | Wipo information: published in national office |
Ref document number: 2003809382 Country of ref document: EP |