JPH05120274A - Natural language processing method - Google Patents

Abstract

PURPOSE:To take the best syntax and meaning analysis corresponding to the class of each sentence and make the precision of the analyzing process high by utilizing syntax features that each class has for the syntax and meaning analysis. CONSTITUTION:This method is equipped with a one-sentence segmenting process means, a sentence class determining process means, a morpheme analyzing process means, and a syntax and meaning analyzing process means, which performs the best analyzing process 11 for sentences belonging to respective classes by utilizing class classifications determined by the sentence class determining process means when the syntax and meaning analysis is taken. Namely, when a one-sentence segmenting process 10 is performed, format information on centering, margins, indents, etc., is recognized and sentences are classified into a title sentence, an index sentence, list sentences, normal sentences, etc., according to the format information; when the syntax and meaning analyzing process 11 is performed, the syntax features that the respective classes have are utilized, so the optimum syntax and meaning analysis corresponding to the classes is taken.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a natural language sentence processing method, and more particularly to a natural language processing method for performing a syntax / semantic analysis process using information about a document format.

[0002]

2. Description of the Related Art A conventional natural language processing method is a processing method for processing character information of a document, and is not a processing method considering the appearance form (format) of a character code string in a document such as centering, margin and indent. There wasn't. However, in general documents, the format is also part of the information that the author seeks to convey to the reader. For example, centering, margins, indentation, etc. indicate that the sentence is a title, a headline, or a list sentence (such as item listing or item writing). These sentences have special roles not only in terms of format but also in terms of syntactic and content, and in many cases syntactic and semantic analysis require processing different from ordinary sentences.

To explain with a simple example, in the document shown in FIG. 7, the first line "VariousAccess Conflicts at
“I / OPorts” is a headline, so “Various
(Adjective) Access (Noun) Conflicts (Noun) at (Preposition)
I / O Ports (noun) "should be interpreted as a whole noun phrase, but if you analyze only a character string without considering the format," Various (Adjective) Access "
(Noun) Conflicts (Verb) at (Preposition) I / O Ports (Noun) ”
Will be misinterpreted.

As described above, although the format is the information that cannot be ignored in analyzing the document, the conventional natural language processing method has not sufficiently processed the format information. Therefore, even a headline or a list sentence is processed without being distinguished from a normal sentence at all, and an incorrect analysis result is often obtained.

[0005]

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned circumstances, and recognizes format features such as centering, margin, and indent at the time of processing to cut out one sentence from a document, and based on the recognition. , Classify sentences into title sentences, headline sentences, list sentences, ordinary sentences, etc.,
Also, at the time of syntax / semantic analysis, by utilizing the syntactic features of each class, it is possible to perform the optimum syntax / semantic analysis according to the class to which each sentence belongs, and to improve the accuracy of analysis processing. To do.

[0006]

A natural language processing method according to the present invention comprises a single sentence cutout processing means for extracting a sentence from an input natural language document, and a plurality of the sentence sentences based on their format and syntactical characteristics. A sentence class determination processing unit that classifies the sentence into a class, a morpheme analysis processing unit that divides the sentence into word units and assigns a part of speech, and a syntactic / semantic analysis processing unit that syntactically and semantically analyzes the sentence. In the provided natural language processing method, the syntactic / semantic analysis processing means uses the class classification determined by the sentence class determination processing means at the time of syntactic / semantic analysis, and is optimal for a sentence belonging to each class. It is a natural language processing method characterized by performing analysis processing.

[0007]

According to the natural language processing method of the present invention, format information such as centering, margin, and indentation is recognized at the time of processing to extract one sentence, and based on this format information, a sentence is classified into a title sentence, a headline sentence, Classes such as list sentences and ordinary sentences are classified, and at the time of syntax / semantic analysis, optimum syntactic / semantic analysis according to the class is performed in order to utilize syntactic features of each class.

[0008]

1 is a flow chart of natural language processing according to the present invention. The natural language processing according to the present invention is composed of one sentence cutout processing (10) and analysis processing (11).

Here, the sentence cut out by the one-sentence cutout process (10) refers to a unit which is an input to the syntax / semantic analysis process (11).

Further, FIG. 2 shows a flowchart of the one-sentence cutout process (10), and FIG. 5 shows a flowchart of the analysis process (11).

[Description of Operation of Embodiment] Before describing the embodiment, the operation principle of the embodiment according to the present invention will be briefly described.

The one-sentence cutout process (10) will be described with reference to the flow chart of FIG. 2. First, in the line recognition process (21), the morphological characteristics of the line are recognized, and each line is processed as shown in FIG. Row attributes as shown in are added. The morphological features of interest here are centering, margins, indented space between lines, typeface, character size, numbers / symbols, colons, and the like.

Next, in the sentence end recognition process (22), it is checked whether the line currently being processed has a sentence end of one sentence (23). When the end of the sentence is recognized here, the line attributes of the lines that form one sentence are collected, and then this one sentence is cut out in the cutout process (24), and the one sentence cutout process ends. If the end of the sentence is not recognized, the subsequent line is recognized until the end of the sentence is recognized.

In the sentence class determination process (25), the sentence class is determined according to the rules shown in FIG. 4 with reference to the line attributes of the lines forming the sentence.

Next, the analysis process (11) will be described with reference to the flowchart of FIG. Analysis process (1
In 1), the sentence cut out by the one sentence cutout process (10) is analyzed. At this time, different analysis processes are performed according to the class classification determined by the sentence class determination process (25).

First, in the morphological analysis process (50), the part of speech of each word is determined. When the part of speech for a word cannot be definitely determined by this processing, a plurality of candidates are secured and a final determination is made by the syntax / semantic analysis processing.

Next, in the syntactic / semantic analysis processing (51), general syntactic / semantic analysis such as grouping of noun phrases and verb phrases and resolution of disambiguation of a contact person is performed.

In the middle of the flow of the syntax / semantic analysis processing (51), analysis processing (53, 54, 5) corresponding to each class classification is performed.
5) exists, and the flow of processing is divided into these according to the classification determined in the sentence class determination processing (25) (52), and highly accurate analysis processing depending on the appearance form of the sentence is performed.

[Processing Operation of the Embodiment] A processing example for eliminating the ambiguity in the interpretation of the ing verb in the list sentence will be described below with reference to the document shown in FIG. Here, the list sentence refers to the form of a sentence used for enumerating items and itemizing, and in terms of format, 1) is indented, and 2) the number of characters per line is smaller than that of a general sentence.
3) It has a feature that the line immediately before is broken by a colon.

Before entering the description, the ambiguity of the interpretation of the "ing verb" will be described. S1, S2, S3, S4 of FIG.
Is expressed in the form of "ing verb + noun phrase", but in general, in such a sentence, there is ambiguity in the interpretation of the "ing verb", and it is difficult to identify its usage. These "in
There are three possible interpretations of "g-type verb + noun phrase", as shown in FIG.

When the ing verb is immediately followed by an article like S1 or S3 in FIG. 7, it can be said that "ing verb = verb usage" is often used, but as in S2 and S4, There is a possibility that the i ng verb is a transitive verb, and i
If a noun or adjective follows the ng verb,
Usage identification is very difficult.

In this embodiment, the present invention will be used to explain how such ambiguity is resolved.
Here, it is assumed that the natural language processing up to the position indicated by X (70) in the document of FIG. 7 has already been completed.

First, the one-sentence cutting process (10)
Recognizing each line that constitutes the input document is performed. Figure 8
FIG. 9 is a diagram showing a result in which each line is recognized by a line recognition process (21) and an attribute is given to each line. In FIG. 8, the line (80) is determined to be a list line (LATR5) because the previous line is terminated by a colon, the indent at the beginning of the line, and the following line match the symbol.

Next, in the sentence end recognition process (22), the line
The end of the line (80) is recognized as the end of the sentence because the (80) ends in the middle of the line, the indent at the beginning of the line (81), and the symbol match with the following line.

The sentence recognized by the sentence end recognition process (22) is cut out from the document by the cutout process (24).
Statement class determination processing based on the row attributes of the rows that make up the statement
The sentence class is determined by (25). Here, in order to derive the sentence class from the line attribute, a rule as shown in FIG. 4 is used.

In the case of the line (80), the sentence (90) is written in only one line.
And the attributes of the list line of line (80) (LAT
R5) is associated with the list sentence attribute (SATR4). Similar to the case of the line (80), the line (81), the line (82), and the line (83) are respectively sentence (9
1), sentence (92), and sentence (93) are provided, and the attribute (SATR4) of the list sentence is given. FIG. 9 shows the result of cutting out sentences from the document of FIG. 7 and the sentence class corresponding to each sentence.

As described above, the one-sentence cutout process (10) is completed, and then the analysis process (11) is performed.

FIG. 10 is a diagram showing a result of the part-of-speech determination performed on the sentence (93) by the morphological analysis processing (50). Here, for the word “missing”, the ing verb (Interpretation 1 in FIG. 6), the adjective (Interpretation 2 in FIG. 6), ing
There are three possible interpretations of the noun (Interpretation 3 in Figure 6),
Since the part of speech for the word cannot be reliably determined by this processing, the final determination is made by the syntax / semantic analysis processing.

In the syntax / semantic analysis process (51), general analysis processes such as grouping of sentence elements into phrase units and determination of the related party are performed. The sentence (93) uses the attributes of the list sentence class. In order to have the list, the list specific process (55) is performed. FIG. 11 shows an example of the list sentence specific processing rule in the syntax / semantic analysis processing.

Syntactically, each sentence forming the list has a kind of coordinate, and each sentence is often written in the same expression form. For example, if the first sentence is composed of noun phrases, in most cases, the second sentence and subsequent sentences are also noun phrases. Correct analysis can be performed by creating an analysis rule based on such characteristics of the list sentence class. The processing rule shown in FIG. 11 is obtained by regularizing the characteristics of such a list sentence in the syntactic analysis process.
i of more than half of the sentences with the form "ng verb + noun phrase"
If the ng verb is X usage, the remaining "ing verb +
It also shows that the usage of the ing verb of a sentence having a "noun phrase" form is also the X usage.

Speaking of sentence (93), in general, when a sentence immediately follows a verb, such as sentence (90) or sentence (92), the ing form takes a verb usage (Interpretation 1 in FIG. 6). ). Therefore, according to the above rules, if the sentence (90) and the sentence (92) take the verb usage (Interpretation 1 in FIG. 6), the sentence (91) and the sentence (93) also have the verb usage (FIG. 6).
Interpretation 1) can be taken.

[0032]

According to the natural language processing method of the present invention, format information such as centering, margin, and indentation is recognized at the time of processing to cut out one sentence, and based on this format information, a sentence is classified into a title sentence and a headline. Optimal syntax / semantic analysis according to the class is performed in order to classify sentences, list statements, ordinary statements, etc. into classes and utilize syntactic features of each class at the time of syntax / semantic analysis. Therefore, it is possible to perform analysis processing with higher accuracy than the conventional method.

[Brief description of drawings]

FIG. 1 is a diagram showing a flow of entire processing.

FIG. 2 is a diagram showing a flow of a single sentence extraction process.

FIG. 3 is a diagram showing row attributes.

FIG. 4 is a diagram showing a rule for determining a sentence class from a line attribute.

FIG. 5 is a diagram showing a flow of a syntax analysis process.

Figure 6: ing in the "ing verb + noun phrase" format
It is a figure showing the possibility of interpretation of a form verb.

FIG. 7 is a diagram illustrating an example of an input document.

FIG. 8 is a diagram showing line attributes corresponding to lines forming an input document.

FIG. 9 is a diagram showing sentence classes corresponding to sentences forming an input document.

FIG. 10 is a diagram showing a morphological analysis result of a list sentence including an ing verb.

FIG. 11 is a diagram showing an analysis process specific to a list sentence.

[Explanation of symbols]

 10 Single Sentence Extraction Process 11 Analysis Process 21 Line Recognition Process 22 Sentence End Recognition Process 24 Extraction Process 25 Sentence Class Determination Process 50 Morphological Analysis Process 51 Syntax / Semantic Analysis Process 55 List Sentence Specific Process

Claims (1)

[Claims] Claim: What is claimed is: 1. One sentence segmentation processing means for extracting a sentence from an input natural language document, sentence class determination processing means for classifying the sentence into a plurality of classes based on its format and syntactical characteristics, and the sentence. In a natural language processing method comprising: a morphological analysis processing unit that divides words into word units and adds a part of speech; and a syntactic and semantic analysis processing unit that syntactically and semantically analyzes the sentence. Processing means, at the time of syntax / semantic analysis,
A natural language processing method characterized by performing optimal analysis processing on a sentence belonging to each class using the class classification determined by the sentence class determination processing means.