JPH04188364A - Device for extracting intrinsic wording of japanese sentence - Google Patents

Abstract

PURPOSE:To extract an intrinsic wording mixing HIRAGANA(cursive form of Japanese syllabary) by extracting the peculiar words having 'the continuous form of a verb + inflecting termination' as a constitutive element as intrinsic wording candidates and narrowing down these candidates in two stages. CONSTITUTION:The HIRAGANA to be the continuous form termination of the verb is defined as the other character type to be distinguished from the other HIRAGANA, the character string of an objective document is developed to the column of character type codes and from the column, the intrinsic wordings having 'the continuous form of the verb + the inflecting termination' as the constitutive element are extracted as the intrinsic wording candidates by a HIRAGANA mixed intrinsic wording candidate extraction part 4. Next, those candidates are narrowed down in the first stage according to a narrow-down rule concerning the length of the character string or the combination of the character types at a HIRAGANA mixed intrinsic wording candidate narrow-down part A 6 and in the second stage to delete words already registered in a general word dictionary from the candidates at a HIRAGANA mixed intrinsic wording candidate narrow-down part B 8 by referring to the general word dictionary 9 concerning the intrinsic wording candidates narrowed down in the first stage. Thus, the intrinsic wording mixing HIRAGANA can be automatically extracted.

Description

[Detailed Description of the Invention] [Field of Industrial Application] The present invention automatically extracts proper nouns such as product names, company names, and one person's names used in two Japanese documents from two Japanese documents; This relates to a device for extracting unique Japanese words that automatically extracts new words and words that are considered to be used only in the document (including proper nouns, these are called proper words), even if they are combinations of common words. be.

[Conventional technology]

Techniques for extracting unique terms include Japanese Patent Application No. 1983-238385 and Japanese Patent Application No. 1978-1978 filed by the same applicant.
There is No. 3. In this method, the characters used in 3 Japanese languages are classified into approximately 10 types of characters, including 1 kanji numerals, ``kanji,''``hiragana,'' and ``katakana.''

Words sandwiched between “delimiter ” and “delimiter” or words sandwiched between “hiragana” and “hiragana” are extracted as unique term candidates, and narrowed down by applying candidate narrowing down rules regarding combinations of single character types. and narrow down the search based on whether it is registered in the general word dictionary.
The remaining words were extracted as unique terms.

[Problem to be solved by the invention]

With conventional technology, it was possible to extract unique terms consisting of kanji, kataka, 9 alphabets, etc., but it is difficult to extract unique terms that are mixed with hiragana, such as "handwritten characters" and "built-in functions." The disadvantage was that it could not be extracted.

The present invention aims to overcome the above-mentioned problems by utilizing the structural features of proper terms containing hiragana and to automatically extract proper terms containing hiragana.

[Means to solve the problem] The present invention is based on one of the unique terms included in Japanese documents.

3 Japanese Patent Application No. 197843 from the target Japanese documents.

Kanji extracted using the method disclosed in Japanese Patent Application No. 1983-238385.

Targets character strings that are composed of katakana, alphabetic characters, or a combination thereof, and are divided into text before and after words that are recognized as unique terms.

Taking advantage of the fact that many unique edicts containing hiragana are composed of other nouns added to the conjunctive ending of a verb, hiragana that can be used as the conjunctive ending of a single verb are distinguished from other hiragana and given in a different character type. Then, expand the character string of the target document into a string of character type codes, extract unique terms whose constituent elements are ° “conjugated form of verb + conjugated ending” as unique term candidates, and select these candidates. The first stage of narrowing down is based on the narrowing down rules regarding the length of one character string and the combination of character types, and the unique term candidates narrowed down in the first stage are referred to the common language dictionary to find words that have already been registered in the common language dictionary. The second stage of narrowing down is performed to delete the words that are listed from the candidates.

[For production]

Unique terms whose constituent elements are "conjugated form of verb + conjugated ending" are extracted as unique term candidates, and these candidates are narrowed down in the above two steps to extract unique terms containing hiragana as well.

〔Example〕

11] is a basic configuration circle showing an embodiment of the present invention.

1 is a Japanese document, 2 is a list of unique terms)・'1. J, 1.
1 represents the unique term list "2". 12 is the main body of the Japanese sentence specific term extraction device; 9 character string separation unit 3.
Character type code table 5. Extracting unit for unique term candidates with flat names mixed in. Hiragana mixed solid term candidate narrowing down section A6. Narrowing down rule table 7, hiragana-mixed unique term candidate narrowing down section B8. General language dictionary 9. It is composed of a unique term output section 10.

The Japanese document 1 is an input document to the Japanese language specific term extraction device main body 12.

The unique term list rlJ2 is a list of unique terms that do not include hiragana and are extracted in advance from the Japanese document 1 by the conventional Japanese document unique term extraction device.

The character string separation unit 3 inputs the Japanese document 1, searches the unique term list "1" 2, deletes the unique term if it exists in the human text, separates the character strings before and after the unique term, and separates each Create this list as an independent separated string.

The hiragana-mixed unique term candidate extraction unit 4 inputs the separated character string list created by the 9-character string 6-separation unit 3, converts this into a code string using the character type code table 5, and then converts this code string into a code string. As input, candidates for unique terms mixed with hiragana are extracted, and a list of candidates for unique words mixed with hiragana is created.

The character type code table 5 is a table that separates characters used in a Japanese document into a plurality of character types and shows character type codes corresponding to each of the character types.

The hiragana-mixed unique term candidate narrowing down unit A6 is.

The list of hiragana-mixed proper term candidates created by the hiragana-mixed proper term candidate extraction unit 4 is input, and the rules of the narrowing-down rule table 7 are applied to convert those that meet one selection condition into the original character string.

The narrowing down rule table 7 is a table in which rules for narrowing down candidates are written based on information regarding the arrangement of one character string length, three character type codes, and the like.

The hiragana-mixed unique term candidate narrowing down unit B8 is.

The common word dictionary 9 narrows down the hiragana-mixed unique term candidates narrowed down by the hiragana-mixed unique term candidate narrowing down unit A6 and converted into one-dimensional character strings.

General language dictionary 9 is the font of the Japanese word ``boat fishing''.

This is a dictionary of common words that describes readings, parts of speech, etc.

The unique term output unit 10 accumulates the results narrowed down by the hiragana-mixed unique term candidate narrowing down unit B8, and outputs the unique term list "2" to the output device.

Unique term list) r2J11 is a list of outputted unique terms.

FIG. 2 shows a schematic flow diagram of an example of the operation of the Japanese language specific term extraction device. The operation of FIG. 1 will be explained below with reference to these figures.

Step 31-1: Input any Japanese document.

Step 5L-1 In order to process the first sentence included in a human-generated Japanese document (hereinafter referred to as input document), n=1 is set.

Step St-, 3: Repeat the process up to step 5l-7 until all input documents are processed.

Step 5l-1 The character string separation unit 3 sequentially searches the list of unique terms I-r I J for the purpose of searching for a unique term included in the n-th sentence.

Stencil 31-5: Furthermore, the character string separation unit 3 deletes a character string that matches the unique term searched in step 51-4 from the nth sentence if it exists in the nth sentence. Furthermore, separate the strings before and after that,
Each is an independent separated character string.

Step 5l-6: Let n=n+1 to process the next sentence.

Step 5l-7: Processing is divided depending on whether processing has been completed for all sentences in the input document. If all the processing has not been completed, the process returns to step 51-3, and if it has been completed, the process proceeds to step 51-8.

Step 5l-8: Create a list of separated character strings separated and accumulated through the above processing.

The list of separated character strings created in Step 5l-1 and Step 51-8 is input.

Step 52-2: Input separated character string list (hereinafter referred to as 1-manual separated character string)
In order to process the first character string included in , i=1.

Step 32-3: Step 52-7 until all input separation strings are processed.
Repeat the process up to.

Step 52-4: The hiragana-mixed unique term candidate extraction unit 4 converts each character of the i-th character string into multiple types of codes based on the character types (for example, the 7 characters shown in FIG.
type code).

Generate a code string for the i-th character string.

Step 31-5: Further, in the hiragana-mixed proper term candidate extraction unit 4.

The i-th character string is separated by character type J or character type F. Alternatively, all combinations of the enclosed character strings A, B, C, D, and E that include the character code B are extracted as hiragana-mixed unique term candidates.

Step 52-6: Let i=i+1 to process the next string.

Step 52-7: Processing is divided depending on whether processing has been completed for all character strings in the input character string.

If all processing has not been completed, step 52-
Return to step 3, and if completed, proceed to step 32-8.

Step 82 - A list of unique term candidates containing hiragana characters extracted through the above process is created.

Step 53-1: The list of hiragana-mixed unique term candidates created in step 52-8 is input.

Step 53-2: Set j=1 to process the first unique term candidate character string included in the input hiragana-mixed unique term candidate list.

Step 53-3: Repeat the processing up to step 53-7 until all input unique term candidate character strings are processed.

Step 53-4: In the hiragana-mixed unique term candidate narrowing down section A.

A selection process is performed by applying the rules of narrowing rule table 1 to the term candidate character string for the first term, and it is determined whether or not it is a unique term candidate.

Step 33-5: Furthermore, in the hiragana-mixed unique term candidate narrowing down section A, processing is divided depending on whether or not the unique term candidate is selected as a unique term candidate by applying the narrowing down rule.

If selected, proceed to step 53-6.

If not selected, the process advances to step 53-8.

Step 53-6: In the hiragana-mixed unique term candidate narrowing down section B.

The character type code ′ of the unique term candidate character string selected by the hiragana-mixed unique term candidate narrowing down unit A is replaced with the original character.

Step 53-7: Furthermore, the hiragana-mixed unique term candidate narrowing down unit B narrows down the list of candidates, with reference to the common word dictionary 9, to determine whether or not the unique term candidates replaced with the original characters in step 53-6 are common words.

Step 53-8: In order to process the next unique term candidate, let j=j+1.

Step 53-9: Processing is divided depending on whether processing has been completed for all unique term candidates in the input unique term candidate list.

If all processing has not been completed, step 53-
Return to 3, V! :If completed, step 5ilo
Proceed to.

The unique term candidates selected in step S3-10 and step 53-8 are output to the unique term output unit 10 as a final result.

Step S3-11: Create a unique term list "2" from the final resultant unique terms.

Next, an outline of the operation will be explained using a specific example of an input document. The input sentence example shown in FIG. 3 will be described as Japanese document 1.

First, the 3-character string separator 3 examines the unique term lists "1" and 2, which are lists of unique terms that do not include hiragana that have already been extracted as shown in FIG.

If there is a unique term in the list in the input sentence example.

Delete it as shown in FIG. Furthermore, as shown in FIG. 6, a two-separate character string list is created by making the sequence of characters before and after the deleted unique term into independent separate character strings.

Next, in the unique term candidate extraction unit 4 that includes one flat name.

Each character string in the separated character string list created by the character string separator 3 is processed in order. First, for the first separated character string "On the other hand, each", use the character type code table 5 shown in Fig. 7, which classifies Japanese characters into 7 types, and use the coded separated character string shown in Fig. 8. the first coded string r in the list
Convert to AAFAJ. The second separated character string ``The movement of is an appropriate m number'' can be determined by similar operations such as rJABJ FAAJ
It is converted to AJJ. In the same manner, the coded separated character string list shown in FIG. 8 is obtained.

Next, this coded character string is extracted using two character type codes F and J. Alternatively, all combinations of the enclosed character type codes A, B, C, D, and E that include B are treated as hiragana-mixed unique term candidates.

First, regarding the first coded character string rAAFAJ.

Coded character string rAAJ with character type code F as the boundary.

"A" is separated, but since these do not include the character type code B, they are not candidates.

Next, 5 second code string rJABJFAAJAJ J
For J, rABJ, rAAJ, and rAJ are extracted in the same way, and among these, rAB that includes the character type code B is set as a unique term candidate containing hiragana.

By repeating the same process, a list of coded character strings of unique term candidates mixed with hiragana as shown in FIG. 9 is obtained.

115=Next, in the hiragana-mixed proper term candidate narrowing down section A6,
A narrowing down rule is applied to each element of the hiragana mixed unique term candidate encoded character string list created by the hiragana mixed unique term candidate extraction unit 4, and a narrowing process is performed.

First, the first rule in the narrowing-down rule table shown in FIG. 10 is applied to the first coded character string rABJ, and it is eliminated from the candidates. Then the second coded string r
The fourth rule is applied to AABJ and it is removed from the list of candidates.

In the same manner, a rule application result as shown in FIG. 11 is obtained. As a result, "motion frame" and "movement amount"
are extracted as unique term candidates mixed with hiragana.

Next, in the hiragana-mixed unique term candidate narrowing down section B8,
The common word dictionary 9 is checked for the unique term candidates created by the hiragana-mixed unique term candidate narrowing down section A, and if the unique term candidates are in the dictionary, they are dropped from the unique term candidates.

The remaining unique term candidates are extracted as unique terms mixed with hiragana. In this example, the unique term candidates ``motion frame'', ``
Both of the words ``amount of movement'' are not found in the general language dictionary, and are extracted as unique words mixed with hiragana.

As described above, 2 using the unique term list ``1''
By using a step-by-step extraction method, that is, by focusing on the conjunctive conjugated ending hiragana and developing it into a unique character type code string, by applying narrowing down rules regarding the order of the code string, and by referring to a common word dictionary to narrow down the candidates.
It is possible to extract ``motion frame'' and ``amount of motion,'' which are unique terms mixed with hiragana.

〔Effect of the invention〕

According to the present invention, unique terms mixed with hiragana can be extracted.1 For example, a process of extracting document-specific terms from a document to be translated and creating a bilingual collection.

It can be expected to significantly improve the efficiency of creating glossaries and indexes for Japanese documents such as papers and copyrighted works.

For example, when creating a Japanese-English bilingual collection for Japanese-English translation, in the past, before translating a large number of documents to be translated, the technical terms and special terms used in the documents were manually extracted from the documents. Previously, bilingual collections were created by extracting words and giving translations to them.2 By using the present invention, terms unique to the document are extracted from the document to be translated in the preprocessing of translation, and are analyzed by morphological analysis and related translations. It becomes possible to extract data at high speed without performing analysis processing such as analysis. Also.

By performing morphological analysis of unknown words, it becomes possible to extract unique words that may cause analysis failure or extraction failure. In the present invention, the quality of translation can be expected to be significantly improved by registering and using a glossary that provides translations for extracted unique terms in a user dictionary.

Regarding automatic index creation, conventionally, when creating an index, the author picks out terms that are considered important from the manuscript he has written, sorts the extracted terms, and creates the index manually. The procedure was as follows. By using the present invention, an author can automatically extract index candidate terms by running an electronic Japanese document through this device.

The author can extract terms to be used in the index by deleting some unnecessary candidates from among the candidates presented by this device. The electronic candidate terms can be easily sorted in alphabetical order, and an index can be created quickly.

[Brief explanation of drawings]

FIG. 1 is a basic configuration diagram showing an embodiment of the present invention. Figure 2 is a schematic flowchart of the operation of the present invention, Figure 3 is an example of a Japanese document that is input to the present invention, Figure 4 is a list of unique terms that do not include hiragana that have been extracted in advance, and Figure 5 is a list of unique terms that do not include hiragana. An example of deleting a unique term extracted from an input document, Figure 6 shows a list of separated character strings created by separating the sequence of characters before and after the deleted unique term Figure 7 shows a list of 7 Japanese characters Example of content of character type court table divided into types, No. 8
The figure shows a list of separated character strings converted into codes using the character type code table, and FIG. 9 shows the separated character strings separated by character type codes F and J. Or enclosed A, B, C. Among character strings D and E, select only those containing B = 19
- Complementary list, FIG. 10 shows an example of the contents of a narrowing rule table applied to the coated character strings remaining as candidates, and FIG. 11 shows the final candidate list and the results of applying the narrowing rules. 1...Japanese document, 2...specific term list) rlJ. 3... Character string separation unit, 4... Hiragana mixed unique term candidate extraction unit, 5... Character type code table, 6... Hiragana mixed unique term candidate narrowing down unit A, 7... Narrowing down rule table , 8... Hiragana-mixed specific term candidate narrowing section B, 9... General word dictionary, 10... Specific term output unit, 11... Specific term list r2. +, 12・
・Japanese sentence-specific term extraction device. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (1)

[Scope of Claims] The system is configured to input a Japanese document written in Japanese and a unique term list "1" that does not include hiragana included in the document, the input Japanese document and the unique term list "1" a character string separation unit that separates and extracts the character strings sandwiched between the above-mentioned unique terms and unique terms; a character type code table that classifies characters used in Japanese documents into multiple specific character types; , a hiragana-mixed unique term candidate extraction unit that extracts hiragana-mixed unique term candidates for each character string separated by the character string separation unit using the character type code table; Hiragana-mixed proper term candidate narrowing down unit A that extracts only candidates that satisfy the selection conditions for narrowing down from the mixed proper term candidates
and a hiragana-mixed unique term candidate narrowing-down unit that compares the candidate terms narrowed down by the hiragana-mixed unique term candidate narrowing unit A with a general language dictionary and deletes words registered in the general-language dictionary to further narrow down the candidates. B, a hiragana-mixed unique term output unit that accumulates narrowed-down results and outputs a unique term list "2" to an output device; Extraction device.