JPH01234975A - 日本語文章分割装置 - Google Patents

日本語文章分割装置

Info

Publication number
JPH01234975A
JPH01234975A JP63056507A JP5650788A JPH01234975A JP H01234975 A JPH01234975 A JP H01234975A JP 63056507 A JP63056507 A JP 63056507A JP 5650788 A JP5650788 A JP 5650788A JP H01234975 A JPH01234975 A JP H01234975A
Authority
JP
Japan
Prior art keywords
word
division
dictionary
words
character
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP63056507A
Other languages
English (en)
Japanese (ja)
Other versions
JPH0519184B2 (enExample
Inventor
Masayuki Morohashi
諸橋　正幸
Shigeki Umeda
梅田　茂樹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to JP63056507A priority Critical patent/JPH01234975A/ja
Priority to US07/321,812 priority patent/US5029084A/en
Publication of JPH01234975A publication Critical patent/JPH01234975A/ja
Publication of JPH0519184B2 publication Critical patent/JPH0519184B2/ja
Granted legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/40—Processing or translation of natural language
    • G06F40/53—Processing of non-Latin text

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Machine Translation (AREA)
  • Document Processing Apparatus (AREA)
JP63056507A 1988-03-11 1988-03-11 日本語文章分割装置 Granted JPH01234975A (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP63056507A JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置
US07/321,812 US5029084A (en) 1988-03-11 1989-03-10 Japanese language sentence dividing method and apparatus

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP63056507A JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置

Publications (2)

Publication Number Publication Date
JPH01234975A true JPH01234975A (ja) 1989-09-20
JPH0519184B2 JPH0519184B2 (enExample) 1993-03-16

Family

ID=13029039

Family Applications (1)

Application Number Title Priority Date Filing Date
JP63056507A Granted JPH01234975A (ja) 1988-03-11 1988-03-11 日本語文章分割装置

Country Status (2)

Country Link
US (1) US5029084A (enExample)
JP (1) JPH01234975A (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024177013A (ja) * 2023-06-08 2024-12-19 ムハユ インコーポレーテッド 日本語文書の剽窃有無を判断するための装置及び方法

Families Citing this family (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5111465A (en) * 1989-06-30 1992-05-05 Digital Equipment Corporation Data integrity features for a sort accelerator
DE4135261C1 (enExample) * 1991-10-25 1993-03-18 International Business Machines Corp., Armonk, N.Y., Us
JPH05216389A (ja) * 1992-02-03 1993-08-27 Matsushita Electric Ind Co Ltd 学習装置
US5867812A (en) * 1992-08-14 1999-02-02 Fujitsu Limited Registration apparatus for compound-word dictionary
JP2965455B2 (ja) * 1994-02-15 1999-10-18 富士ゼロックス株式会社 言語情報提供装置
JPH08329105A (ja) * 1995-05-31 1996-12-13 Canon Inc 文書処理方法及びその装置
US5806021A (en) * 1995-10-30 1998-09-08 International Business Machines Corporation Automatic segmentation of continuous text using statistical approaches
JP2973944B2 (ja) * 1996-06-26 1999-11-08 富士ゼロックス株式会社 文書処理装置および文書処理方法
US6035268A (en) * 1996-08-22 2000-03-07 Lernout & Hauspie Speech Products N.V. Method and apparatus for breaking words in a stream of text
US6351726B1 (en) * 1996-12-02 2002-02-26 Microsoft Corporation Method and system for unambiguously inputting multi-byte characters into a computer from a braille input device
JPH1153384A (ja) 1997-08-05 1999-02-26 Mitsubishi Electric Corp キーワード抽出装置及びキーワード抽出方法並びにキーワード抽出プログラムを格納したコンピュータ読み取り可能な記録媒体
JP3272288B2 (ja) * 1997-12-24 2002-04-08 日本アイ・ビー・エム株式会社 機械翻訳装置および機械翻訳方法
US6694055B2 (en) 1998-07-15 2004-02-17 Microsoft Corporation Proper name identification in chinese
US6185524B1 (en) 1998-12-31 2001-02-06 Lernout & Hauspie Speech Products N.V. Method and apparatus for automatic identification of word boundaries in continuous text and computation of word boundary scores
JP3539479B2 (ja) * 1999-03-11 2004-07-07 シャープ株式会社 翻訳装置及び翻訳方法並びに翻訳プログラムを記録した記録媒体
US6968308B1 (en) * 1999-11-17 2005-11-22 Microsoft Corporation Method for segmenting non-segmented text using syntactic parse
US6678409B1 (en) * 2000-01-14 2004-01-13 Microsoft Corporation Parameterized word segmentation of unsegmented text
SE519636C2 (sv) * 2000-07-06 2003-03-25 Hapax Information Systems Ab Förfarande och anordning för analys av sammansatta ord
US7158930B2 (en) * 2002-08-15 2007-01-02 Microsoft Corporation Method and apparatus for expanding dictionaries during parsing
US8818793B1 (en) 2002-12-24 2014-08-26 At&T Intellectual Property Ii, L.P. System and method of extracting clauses for spoken language understanding
US8849648B1 (en) * 2002-12-24 2014-09-30 At&T Intellectual Property Ii, L.P. System and method of extracting clauses for spoken language understanding
US7941311B2 (en) * 2003-10-22 2011-05-10 Microsoft Corporation System and method for linguistic collation
US7424421B2 (en) * 2004-03-03 2008-09-09 Microsoft Corporation Word collection method and system for use in word-breaking
US7783476B2 (en) * 2004-05-05 2010-08-24 Microsoft Corporation Word extraction method and system for use in word-breaking using statistical information
JP4018668B2 (ja) * 2004-05-28 2007-12-05 株式会社東芝 日中機械翻訳装置、日中機械翻訳方法および日中機械翻訳プログラム
CN101124579A (zh) * 2005-02-24 2008-02-13 富士施乐株式会社 单词翻译装置、翻译方法以及翻译程序
JP4769031B2 (ja) * 2005-06-24 2011-09-07 マイクロソフト コーポレーション 言語モデルを作成する方法、かな漢字変換方法、その装置、コンピュータプログラムおよびコンピュータ読み取り可能な記憶媒体
JP4236057B2 (ja) * 2006-03-24 2009-03-11 インターナショナル・ビジネス・マシーンズ・コーポレーション 新たな複合語を抽出するシステム
CN101512518B (zh) * 2006-09-07 2015-06-24 日本电气株式会社 自然语言处理系统和词典登录系统
JP4446313B2 (ja) * 2006-12-15 2010-04-07 インターナショナル・ビジネス・マシーンズ・コーポレーション 音声処理用の辞書に登録するべき新規語句を検索する技術
EP2416256A4 (en) * 2009-03-30 2017-09-20 Nec Corporation Language analysis device, method, and program
CN102479191B (zh) * 2010-11-22 2014-03-26 阿里巴巴集团控股有限公司 提供多粒度分词结果的方法及其装置
CN103425691B (zh) 2012-05-22 2016-12-14 阿里巴巴集团控股有限公司 一种搜索方法和系统
WO2018146864A1 (ja) * 2017-02-07 2018-08-16 パナソニックＩｐマネジメント株式会社 翻訳装置および翻訳方法
CN111310452B (zh) * 2018-12-12 2024-06-18 北京汇钧科技有限公司 一种分词方法和装置
CN114386407B (zh) * 2021-12-23 2023-04-11 北京金堤科技有限公司 文本的分词方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5840684A (ja) * 1981-09-04 1983-03-09 Hitachi Ltd 自然言語間の自動翻訳方式
JPS58201175A (ja) * 1982-05-20 1983-11-22 Kokusai Denshin Denwa Co Ltd <Kdd> 機械翻訳方式
JPS62163173A (ja) * 1986-01-14 1987-07-18 Toshiba Corp 機械翻訳方法
US4864502A (en) * 1987-10-07 1989-09-05 Houghton Mifflin Company Sentence analyzer

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024177013A (ja) * 2023-06-08 2024-12-19 ムハユ インコーポレーテッド 日本語文書の剽窃有無を判断するための装置及び方法

Also Published As

Publication number Publication date
US5029084A (en) 1991-07-02
JPH0519184B2 (enExample) 1993-03-16

Similar Documents

Publication Publication Date Title
US5029084A (en) Japanese language sentence dividing method and apparatus
US7818165B2 (en) Method and system for language identification
EP0971294A2 (en) Method and apparatus for automated search and retrieval processing
US20100161639A1 (en) Complex Queries for Corpus Indexing and Search
US20100161314A1 (en) Region-Matching Transducers for Text-Characterization
Tufiş et al. DIAC+: A professional diacritics recovering system
Jarrar et al. Alma: Fast lemmatizer and pos tagger for arabic
Pla et al. Improving part-of-speech tagging using lexicalized HMMs
Sarabi et al. ParsiPardaz: Persian language processing toolkit
Li et al. Chinese word segmentation
Nandathilaka et al. A rule-based lemmatizing approach for sinhala language
Kann et al. Implementation aspects and applications of a spelling correction algorithm
Goweder et al. Identifying broken plurals in unvowelised arabic tex
Khorjuvenkar et al. Parts of speech tagging for Konkani language
Kranig Evaluation of language identification methods
Bosch et al. Memory-based morphological analysis and part-of-speech tagging of Arabic
Nejja et al. Context's impact on the automatic spelling correction
JP4088171B2 (ja) テキスト解析装置、方法、プログラム及びそのプログラムを記録した記録媒体
EP0992008A1 (en) FSTs APPROXIMATING HIDDEN MARKOV MODELS AND TEXT TAGGING USING SAME
Van Delden et al. Supervised and unsupervised automatic spelling correction algorithms
Suriyachay et al. Enhancement of character-level representation in Bi-LSTM model for Thai NER
Bolshakova Recognition of author’s scientific and technical terms
Talmon et al. Morphological Tagging of the Qur'an
JP3348872B2 (ja) 日本語形態素解析装置
JP3508312B2 (ja) キーワード抽出装置