JP2005092883A5 - - Google Patents

Download PDF

Info

Publication number
JP2005092883A5
JP2005092883A5 JP2004269036A JP2004269036A JP2005092883A5 JP 2005092883 A5 JP2005092883 A5 JP 2005092883A5 JP 2004269036 A JP2004269036 A JP 2004269036A JP 2004269036 A JP2004269036 A JP 2004269036A JP 2005092883 A5 JP2005092883 A5 JP 2005092883A5
Authority
JP
Japan
Prior art keywords
corpus
language model
text
output
word
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2004269036A
Other languages
English (en)
Japanese (ja)
Other versions
JP2005092883A (ja
Filing date
Publication date
Priority claimed from US10/662,602 external-priority patent/US20050071148A1/en
Application filed filed Critical
Publication of JP2005092883A publication Critical patent/JP2005092883A/ja
Publication of JP2005092883A5 publication Critical patent/JP2005092883A5/ja
Pending legal-status Critical Current

Links

JP2004269036A 2003-09-15 2004-09-15 中国語の単語分割 Pending JP2005092883A (ja)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/662,602 US20050071148A1 (en) 2003-09-15 2003-09-15 Chinese word segmentation

Publications (2)

Publication Number Publication Date
JP2005092883A JP2005092883A (ja) 2005-04-07
JP2005092883A5 true JP2005092883A5 (enExample) 2007-11-01

Family

ID=34136808

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2004269036A Pending JP2005092883A (ja) 2003-09-15 2004-09-15 中国語の単語分割

Country Status (5)

Country Link
US (1) US20050071148A1 (enExample)
EP (1) EP1515240A3 (enExample)
JP (1) JP2005092883A (enExample)
KR (1) KR20050027931A (enExample)
CN (1) CN1661592A (enExample)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4652737B2 (ja) * 2004-07-14 2011-03-16 インターナショナル・ビジネス・マシーンズ・コーポレーション 単語境界確率推定装置及び方法、確率的言語モデル構築装置及び方法、仮名漢字変換装置及び方法、並びに、未知語モデルの構築方法、
US20070005345A1 (en) * 2005-07-01 2007-01-04 Microsoft Corporation Generating Chinese language couplets
JP2007024960A (ja) * 2005-07-12 2007-02-01 Internatl Business Mach Corp <Ibm> システム、プログラムおよび制御方法
JP3986531B2 (ja) * 2005-09-21 2007-10-03 沖電気工業株式会社 形態素解析装置及び形態素解析プログラム
US20070078644A1 (en) * 2005-09-30 2007-04-05 Microsoft Corporation Detecting segmentation errors in an annotated corpus
US8539349B1 (en) 2006-10-31 2013-09-17 Hewlett-Packard Development Company, L.P. Methods and systems for splitting a chinese character sequence into word segments
US8000955B2 (en) * 2006-12-20 2011-08-16 Microsoft Corporation Generating Chinese language banners
US20080294982A1 (en) * 2007-05-21 2008-11-27 Microsoft Corporation Providing relevant text auto-completions
JP5241828B2 (ja) * 2007-06-14 2013-07-17 グーグル・インコーポレーテッド 辞書の単語及び熟語の判定
WO2008151466A1 (en) 2007-06-14 2008-12-18 Google Inc. Dictionary word and phrase determination
JP5256654B2 (ja) * 2007-06-29 2013-08-07 富士通株式会社 文章分割プログラム、文章分割装置および文章分割方法
US7962507B2 (en) 2007-11-19 2011-06-14 Microsoft Corporation Web content mining of pair-based data
US9411800B2 (en) * 2008-06-27 2016-08-09 Microsoft Technology Licensing, Llc Adaptive generation of out-of-dictionary personalized long words
US8229965B2 (en) * 2009-06-30 2012-07-24 Mitsubishi Electric Research Laboratories, Inc. System and method for maximizing edit distances between particles
CN102043791B (zh) * 2009-10-10 2014-04-30 深圳市世纪光速信息技术有限公司 分词评价方法及装置
EP2488963A1 (en) * 2009-10-15 2012-08-22 Rogers Communications Inc. System and method for phrase identification
CN103631802B (zh) * 2012-08-24 2015-05-20 腾讯科技（深圳）有限公司 歌曲信息检索方法、装置及相应的服务器
WO2014071330A2 (en) * 2012-11-02 2014-05-08 Fido Labs Inc. Natural language processing system and method
JP5770753B2 (ja) * 2013-01-15 2015-08-26 グーグル・インコーポレーテッド Ｃｊｋ名前検出
CN104424177B (zh) * 2013-08-26 2017-09-15 高德软件有限公司 一种抽取核心词的方法及装置
US10607604B2 (en) * 2017-10-27 2020-03-31 International Business Machines Corporation Method for re-aligning corpus and improving the consistency
US10515148B2 (en) * 2017-12-15 2019-12-24 King Fahd University Of Petroleum And Minerals Arabic spell checking error model
CN109190034B (zh) * 2018-08-23 2019-12-13 北京百度网讯科技有限公司 用于获取信息的方法及装置
CN110444210B (zh) * 2018-10-25 2022-02-08 腾讯科技（深圳）有限公司 一种语音识别的方法、唤醒词检测的方法及装置
KR102871441B1 (ko) * 2019-06-18 2025-10-15 엘지전자 주식회사 음성 정보 기반 언어 모델링 시스템 및 방법
CN117422061B (zh) * 2023-12-19 2024-03-08 中南大学 一种文本词项多重分割结果合并标注方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5806021A (en) * 1995-10-30 1998-09-08 International Business Machines Corporation Automatic segmentation of continuous text using statistical approaches
US5963893A (en) * 1996-06-28 1999-10-05 Microsoft Corporation Identification of words in Japanese text by a computer system
US6968308B1 (en) * 1999-11-17 2005-11-22 Microsoft Corporation Method for segmenting non-segmented text using syntactic parse
US7017114B2 (en) * 2000-09-20 2006-03-21 International Business Machines Corporation Automatic correlation method for generating summaries for text documents
TWI256562B (en) * 2002-05-03 2006-06-11 Ind Tech Res Inst Method for named-entity recognition and verification

Similar Documents

Publication Publication Date Title
JP2005092883A5 (enExample)
Hu et al. Parabank: Monolingual bitext generation and sentential paraphrasing via lexically-constrained neural machine translation
Poon et al. Unsupervised morphological segmentation with log-linear models
Sawalha et al. SALMA: standard Arabic language morphological analysis
JP2002215617A (ja) 品詞タグ付けをする方法
KR101869362B1 (ko) 함의 문장 생성 기술을 활용한 문장 표절 판단 장치, 이를 구현하기 위한 프로그램 및 기록 매체
Mohammed Using machine learning to build POS tagger for under-resourced language: the case of Somali
Freihat et al. A single-model approach for Arabic segmentation, POS tagging, and named entity recognition
US8041559B2 (en) System and method for disambiguating non diacritized arabic words in a text
CN111597302B (zh) 文本事件的获取方法、装置、电子设备及存储介质
Tlili-Guiassa Hybrid method for tagging Arabic text
Huang et al. Words without boundaries: Computational approaches to Chinese word segmentation
Hidalgo-Ternero et al. Bridging the “gApp”: improving neural machine translation systems for multiword expression detection
Sethi et al. Automated title generation in English language using NLP
KR101757222B1 (ko) 한글 문장에 대한 의역 문장 생성 방법
Arivazhagan et al. Labeling the semantic roles of commas
Tongtep et al. Multi-stage automatic NE and pos annotation using pattern-based and statistical-based techniques for thai corpus construction
Attia et al. GWU-HASP-2015@ QALB-2015 shared task: priming spelling candidates with probability
Moeller et al. Integrating automated segmentation and glossing into documentary and descriptive linguistics
Sharma et al. Improving existing punjabi grammar checker
Raza et al. Automatic diacritization for urdu
Yohan et al. Automatic named entity identification and classification using heuristic based approach for telugu
Munir et al. Dependency parsing using the URDU. KON-TB treebank
Sutheebanjard et al. Thai personal named entity extraction without using word segmentation or POS tagging
Mirzanezhad et al. Using morphological analyzer to statistical POS Tagging on Persian Text