JPH0519184B2 - - Google Patents

Info

Publication number
JPH0519184B2
JPH0519184B2 JP63056507A JP5650788A JPH0519184B2 JP H0519184 B2 JPH0519184 B2 JP H0519184B2 JP 63056507 A JP63056507 A JP 63056507A JP 5650788 A JP5650788 A JP 5650788A JP H0519184 B2 JPH0519184 B2 JP H0519184B2
Authority
JP
Japan
Prior art keywords
dictionary
word
division
words
rules
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
JP63056507A
Other languages
English (en)
Japanese (ja)
Other versions
JPH01234975A (ja
Inventor
Masayuki Morohashi
Shigeki Umeda
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to JP63056507A priority Critical patent/JPH01234975A/ja
Priority to US07/321,812 priority patent/US5029084A/en
Publication of JPH01234975A publication Critical patent/JPH01234975A/ja
Publication of JPH0519184B2 publication Critical patent/JPH0519184B2/ja
Granted legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/53—Processing of non-Latin text

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
  • Document Processing Apparatus (AREA)
JP63056507A 1988-03-11 1988-03-11 日本語文章分割装置 Granted JPH01234975A (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP63056507A JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置
US07/321,812 US5029084A (en) 1988-03-11 1989-03-10 Japanese language sentence dividing method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP63056507A JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置

Publications (2)

Publication Number Publication Date
JPH01234975A JPH01234975A (ja) 1989-09-20
JPH0519184B2 true JPH0519184B2 (online.php) 1993-03-16

Family

ID=13029039

Family Applications (1)

Application Number Title Priority Date Filing Date
JP63056507A Granted JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置

Country Status (2)

Country Link
US (1) US5029084A (online.php)
JP (1) JPH01234975A (online.php)

Families Citing this family (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5111465A (en) * 1989-06-30 1992-05-05 Digital Equipment Corporation Data integrity features for a sort accelerator
DE4135261C1 (online.php) * 1991-10-25 1993-03-18 International Business Machines Corp., Armonk, N.Y., Us
JPH05216389A (ja) * 1992-02-03 1993-08-27 Matsushita Electric Ind Co Ltd 学習装置
US5867812A (en) * 1992-08-14 1999-02-02 Fujitsu Limited Registration apparatus for compound-word dictionary
JP2965455B2 (ja) * 1994-02-15 1999-10-18 富士ゼロックス株式会社 言語情報提供装置
JPH08329105A (ja) * 1995-05-31 1996-12-13 Canon Inc 文書処理方法及びその装置
US5806021A (en) * 1995-10-30 1998-09-08 International Business Machines Corporation Automatic segmentation of continuous text using statistical approaches
JP2973944B2 (ja) * 1996-06-26 1999-11-08 富士ゼロックス株式会社 文書処理装置および文書処理方法
US6035268A (en) * 1996-08-22 2000-03-07 Lernout & Hauspie Speech Products N.V. Method and apparatus for breaking words in a stream of text
US6351726B1 (en) * 1996-12-02 2002-02-26 Microsoft Corporation Method and system for unambiguously inputting multi-byte characters into a computer from a braille input device
JPH1153384A (ja) 1997-08-05 1999-02-26 Mitsubishi Electric Corp キーワード抽出装置及びキーワード抽出方法並びにキーワード抽出プログラムを格納したコンピュータ読み取り可能な記録媒体
JP3272288B2 (ja) * 1997-12-24 2002-04-08 日本アイ・ビー・エム株式会社 機械翻訳装置および機械翻訳方法
US6694055B2 (en) 1998-07-15 2004-02-17 Microsoft Corporation Proper name identification in chinese
US6185524B1 (en) 1998-12-31 2001-02-06 Lernout & Hauspie Speech Products N.V. Method and apparatus for automatic identification of word boundaries in continuous text and computation of word boundary scores
JP3539479B2 (ja) * 1999-03-11 2004-07-07 シャープ株式会社 翻訳装置及び翻訳方法並びに翻訳プログラムを記録した記録媒体
US6968308B1 (en) * 1999-11-17 2005-11-22 Microsoft Corporation Method for segmenting non-segmented text using syntactic parse
US6678409B1 (en) * 2000-01-14 2004-01-13 Microsoft Corporation Parameterized word segmentation of unsegmented text
SE519636C2 (sv) * 2000-07-06 2003-03-25 Hapax Information Systems Ab Förfarande och anordning för analys av sammansatta ord
US7158930B2 (en) * 2002-08-15 2007-01-02 Microsoft Corporation Method and apparatus for expanding dictionaries during parsing
US8818793B1 (en) 2002-12-24 2014-08-26 At&T Intellectual Property Ii, L.P. System and method of extracting clauses for spoken language understanding
US8849648B1 (en) * 2002-12-24 2014-09-30 At&T Intellectual Property Ii, L.P. System and method of extracting clauses for spoken language understanding
US7941311B2 (en) * 2003-10-22 2011-05-10 Microsoft Corporation System and method for linguistic collation
US7424421B2 (en) * 2004-03-03 2008-09-09 Microsoft Corporation Word collection method and system for use in word-breaking
US7783476B2 (en) * 2004-05-05 2010-08-24 Microsoft Corporation Word extraction method and system for use in word-breaking using statistical information
JP4018668B2 (ja) * 2004-05-28 2007-12-05 株式会社東芝 日中機械翻訳装置、日中機械翻訳方法および日中機械翻訳プログラム
CN101124579A (zh) * 2005-02-24 2008-02-13 富士施乐株式会社 单词翻译装置、翻译方法以及翻译程序
JP4769031B2 (ja) * 2005-06-24 2011-09-07 マイクロソフト コーポレーション 言語モデルを作成する方法、かな漢字変換方法、その装置、コンピュータプログラムおよびコンピュータ読み取り可能な記憶媒体
JP4236057B2 (ja) * 2006-03-24 2009-03-11 インターナショナル・ビジネス・マシーンズ・コーポレーション 新たな複合語を抽出するシステム
CN101512518B (zh) * 2006-09-07 2015-06-24 日本电气株式会社 自然语言处理系统和词典登录系统
JP4446313B2 (ja) * 2006-12-15 2010-04-07 インターナショナル・ビジネス・マシーンズ・コーポレーション 音声処理用の辞書に登録するべき新規語句を検索する技術
EP2416256A4 (en) * 2009-03-30 2017-09-20 Nec Corporation Language analysis device, method, and program
CN102479191B (zh) * 2010-11-22 2014-03-26 阿里巴巴集团控股有限公司 提供多粒度分词结果的方法及其装置
CN103425691B (zh) 2012-05-22 2016-12-14 阿里巴巴集团控股有限公司 一种搜索方法和系统
WO2018146864A1 (ja) * 2017-02-07 2018-08-16 パナソニックＩｐマネジメント株式会社 翻訳装置および翻訳方法
CN111310452B (zh) * 2018-12-12 2024-06-18 北京汇钧科技有限公司 一种分词方法和装置
CN114386407B (zh) * 2021-12-23 2023-04-11 北京金堤科技有限公司 文本的分词方法及装置
KR102669428B1 (ko) * 2023-06-08 2024-05-27 주식회사 무하유 일본어 문서의 표절 여부를 판단하기 위한 장치 및 방법

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5840684A (ja) * 1981-09-04 1983-03-09 Hitachi Ltd 自然言語間の自動翻訳方式
JPS58201175A (ja) * 1982-05-20 1983-11-22 Kokusai Denshin Denwa Co Ltd <Kdd> 機械翻訳方式
JPS62163173A (ja) * 1986-01-14 1987-07-18 Toshiba Corp 機械翻訳方法
US4864502A (en) * 1987-10-07 1989-09-05 Houghton Mifflin Company Sentence analyzer

Also Published As

Publication number Publication date
JPH01234975A (ja) 1989-09-20
US5029084A (en) 1991-07-02

Similar Documents

Publication Publication Date Title
US5029084A (en) Japanese language sentence dividing method and apparatus
Nie et al. On the use of words and n-grams for Chinese information retrieval
US4868750A (en) Collocational grammar system
US5752051A (en) Language-independent method of generating index terms
US7818165B2 (en) Method and system for language identification
US5794177A (en) Method and apparatus for morphological analysis and generation of natural language text
US6035268A (en) Method and apparatus for breaking words in a stream of text
JPH10326275A (ja) 形態素解析方法および装置、並びに日本語形態素解析方法および装置
Tufiş et al. DIAC+: A professional diacritics recovering system
Loftsson et al. Tagging a morphologically complex language using an averaged perceptron tagger: The case of Icelandic
Vasiu et al. Enhancing tokenization by embedding Romanian language specific morphology
Ahmed et al. Gold dataset for the evaluation of Bangla stemmer
Tambouratzis Automatic corpora-based stemming in Greek
Šantić et al. Automatic diacritics restoration in Croatian texts
Saito et al. Multi-language named-entity recognition system based on HMM
Croft et al. Applications of multilingual text retrieval
Kranig Evaluation of language identification methods
Yeshambel et al. Evaluation of corpora, resources and tools for Amharic information retrieval
JP4088171B2 (ja) テキスト解析装置、方法、プログラム及びそのプログラムを記録した記録媒体
Al-Taani et al. Searching concepts and keywords in the Holy Quran
Suriyachay et al. Enhancement of character-level representation in Bi-LSTM model for Thai NER
Nejja et al. Context's impact on the automatic spelling correction
JP2004206659A (ja) 読み情報決定方法及び装置及びプログラム
Zitouni et al. Corpus-based Arabic stemming using N-grams
Al-Shammari A novel algorithm for normalizing noisy Arabic text