JPH09160915A - Contextual processing system for natural language - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a contextual processing system for natural language which is capable of eliminating the ambiguity of a syntax structure and selecting a right analysis result. SOLUTION: The syntax meaning analysis of a sentence is performed (step S1 and S2), the sentence is decomposed into the modification elements of a word, the modification elements are integrated, and the contextual space data composed by having the relative confidence weighed by the evaluation point by the number of the appearance of the modification and a meaning analysis is prepared (S3). The modification element which is large in relative confidence is used as data to be believed. When the modification element which is the same as this element appears in an other sentence, the relative confidence of the modification element is raised. When the modification data which is the same as the modification data which is very small in the relative confidence appears in the other sentence, confidence adjusting processings (A, B and C) lowering the relative confidence of the modification element are performed (S4 to S6). From the contextual space data which is finally obtained from the processings, data with high confidence is selected and a syntax tree is selected (S7).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the validity of dependency between words in an analysis algorithm of an information processing apparatus which executes a certain task by inputting natural language and analyzing its syntactic structure and semantic content. The present invention relates to a processing method of a semantic analysis process and a context analysis process for determining.

[0002]

2. Description of the Related Art Generally, Japanese analysis can be considered in stages, such as morphological analysis, syntactic analysis processing, semantic analysis processing, and context analysis processing. When a syntactic grammar tree is generated in the parsing process, a plurality of syntactic trees are generally obtained. Semantic analysis is performed to select the correct syntax tree from among these. In the semantic analysis process, the dependency of a word is examined to determine whether or not the dependency is semantically valid.

However, in reality, the semantic analysis processing is 100%.
It is difficult to obtain the correct result of, and context analysis processing is performed to solve this problem. However, the context analysis processing is a technically immature field, and at present, only the processing up to the semantic analysis processing and the processing targeting the context is not performed so much.

[0004]

In the conventional semantic analysis processing, it may be difficult to obtain a correct syntactic analysis result. It may be difficult to perform semantic analysis processing because natural language inherently allows vague aspects, but on the other hand, the processing depends on only the notation content described in the sentence of interest. Doing and not using the information of the sentences before and after,
This is a major factor. In other words, the fact that a framework that uses context information well has not been realized is a major problem.

The present invention has been made in view of the above points, and an object thereof is to provide a natural language contextual processing method capable of eliminating ambiguity in syntactic structure and selecting a correct analysis result. is there. That is, the present invention provides a context analysis process that processes a plurality of sentences and uses not only the sentence of interest but also the contents of the sentences before and after the sentence for disambiguating the dependency.
Although the present invention is effective only in a limited number of cases, it also proposes a basic framework for developing various context processing in the future.

[0006]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention performs a syntactic and semantic analysis on a sentence and decomposes it into dependency elements of a word. Create context space data that has relative confidence weighted by the number and the evaluation score by the semantic analysis, and determine the dependency element having a large relative confidence,
It is used as belief data, and when the same dependency element appears in another sentence, the relative confidence factor of that dependency element is increased, and the same dependency factor as the dependency data with a very small relative confidence factor When it appears in a sentence, a certainty factor adjustment process that lowers the relative certainty factor of the dependency element is performed, and data with a high certainty factor is selected from the context space data finally obtained by the process to select a syntax tree. It is characterized by performing.

[0007]

Embodiments of the present invention will be described below with reference to the drawings. According to the present invention, a sentence is syntactically and semantically analyzed, and the result is decomposed and stored as data of the dependency set of words. Each of the dependency sets is referred to as a dependency element, and the data collected by the method described below is referred to as context space data. When a sentence is syntactically and semantically analyzed, plural syntactic trees are generally generated. This syntax tree is compared with the context space data obtained from other sentences, it is checked whether or not the same dependency element exists in the context space data, and this is used as a criterion for selecting the syntax tree. .

Next, a detailed description will be given with examples. Suppose multiple parse trees are obtained from a sentence. If you take a closer look at those contents, you can see that there are dependencies that are commonly included in all syntax trees, and that there are dependencies that appear only slightly.

For example, (sentence a) "The management class issues an instruction to link to all existing device classes."
It is assumed that the following statement is obtained by analyzing the sentence.

[0010]

[Table 1]

Table 2 is a breakdown of the state of this dependency.

[0012]

[Table 2]

The dependency element is composed of data such as word notation (two), a symbol indicating a semantic relationship, and a syntax tree number including the element. Five dependencies such as "issue a command" and "issue a management class" appear commonly in all syntax trees. Therefore, it can be judged that these are certain dependencies. It is considered that other dependencies have relative certainty depending on the number of occurrences, that is, how many syntactic trees in the generated syntactic tree.

Further, since each syntax tree is given an evaluation score by semantic analysis, the difference between the evaluation scores is used as a weight for the number calculation. The scores in Table 2 are the scores thus obtained. Then, based on this score, the relative certainty factor of the dependency element is calculated. The dependency factors are summarized in order from the data with the higher relative confidence.
Create a kind of tree structure data shown in (this is context space data).

[0015]

[Table 3]

In Table 3, for example, the relative existence factor of "existing" is 0.783, and the relative reliability of "existing command" is 0.73. 217.

Further, in the data of relative confidence level 0.783 (this is called a subspace), the related party of "device class" is "issue to device class" and the relative confidence level is 0.500. , The relative certainty factor of "put on the device class" is 0.500. We will proceed with the analysis while creating such context space data for each sentence.

Next, a process of checking based on the created context space data will be described. After a sentence is analyzed and context space data is created, a check process is performed with the context space data obtained from the sentence before that. The process consists of the following three steps A, B, and C. The context space data of the sentence of interest is CurrentConSp, and the list of context space data of the sentences before it is ConSps.

Process A: Believing the dependency element of the relative confidence of CurrentConSp of 1 and adjusting the value of the relative confidence of the dependency element of ConSps. Process B: Believing the dependency element of the relative confidence of ConSps of 1 and adjusting the value of the relative confidence of the dependency element of CurrentConSp. Process C: Fine adjustment of the relative certainty factor is performed between the data of the relative certainty factors of CurrentConSp and ConSps less than 1.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows the overall flow of processing of the present invention. The algorithms of Process A and Process B are the same except that the data is interchanged. The flow of this processing is shown in FIG. 2 and will be described in detail.

(Step S ₁ ) _One dependent element (hereinafter simply referred to as an element) is extracted from the believing context space data (CurrentConSp in process A, ConSps in process B).

(Step S ₂ ) If the relative certainty factor of the element (hereinafter simply referred to as the certainty factor) is 1, the process proceeds to step S ₃ . If not, the process ends.

(Step S ₃ ) An element having the certainty factor of 1 is set as the certainty element.

(Step S ₄ ) Context space data to be adjusted (ConSps in process A, Curren in process B)
One element from tConSp) is taken as the target element.

(Step S ₅ ) If the certainty factor of the target element is 1, this is also believed, and the process is returned to step S ₄ without any change. If less than 1, proceed to step S ₆ .

(Step S ₆ ) When the certainty factor of the target element is less than 1, there are a plurality of subspaces, and the sum of these certainty factors is 1. Therefore, it is checked whether or not the same element as the certainty element is included in each subspace. Whether or not they are the same element is checked by the notation and the semantic relationship between words. In addition, in the case of a decrypted noun, it is often effective to use a noun that follows. If a large thesaurus dictionary is available, the information that the thesaurus codes match can also be used.

(Step S ₇ ) If the same element as the certainty element does not exist in any subspace, the data is left as it is and the process returns to step S ₁ . If not, the process proceeds to step S ₈ .

(Step S ₈ ) If all subspaces include the same element as the certainty element, the process proceeds to step S ₉ . If not, the process proceeds to step S ₁₀ .

(Step S ₉ ) Each subspace is regarded as the context space data to be adjusted, and this process is applied to each subspace from step S ₁ . When it ends, the process returns to step S ₁ .

(Step S ₁₀ ) The certainty factor is increased for the subspace including the same element as the certainty element. The confidence level is lowered for subspaces that do not include it. However, the sum of the confidence factors is limited to 1 even after the adjustment.

(Step S ₁₁ ) The subspace containing the same element as the certainty element is regarded as the adjustment target space, and this processing is applied from step S ₁ . When it ends, the process returns to step S ₁ .

Next, the process C will be described. Process C is also the same as Processes A and B, CurrentConSp and ConS.
As the elements of p are compared, the certainty factor of the certainty factor is multiplied as it goes from the top to the bottom of the subspace. Since the certainty factor in the present invention is originally very uncertain information,
A lower limit value (called a positive lower limit, for example, 0.8) that determines as a correct element and an upper limit value (called a negative upper limit, for example, 0.2) that determines an incorrect element are provided, and intermediate data is treated as a conviction element Absent. CurrentConSp and C
If there is the same onSp element, the processing shown in Table 4 below is performed. However, the adjustment of the certainty factor is the process A and the process B.
Set the change width to be smaller than.

[0033]

[Table 4]

When all the element checking processes are completed as described above, the syntax tree selecting process (step S _{7 in} FIG. 1) is finally performed. This process targets the final list of context space data. Follow the subspace with the higher certainty in the context space, and when you reach the bottom element, see which parse tree this element was derived from and correct the parse tree of that number. Is a syntax tree of.

Further explanation will be given using an actual example. In the above example sentence (sentence A) “Management class issues an instruction to link to all existing device classes”, as shown in Table 1, the syntax tree of No. 2 has low confidence but The third tree has the same evaluation score and certainty factor, and it cannot be determined which is the correct answer. However, it is assumed that after this sentence, a sentence (statement a) "The instruction issued to the device class is executed after confirming that the condition flag is ON". Table 5 summarizes the analysis results of this sentence.

[0036]

[Table 5]

At this time, it can be seen that the element [issue for use, device class] exists with a certainty factor 1.
Also, in the above (sentence A), the same element as this has a certainty factor of 0.
It is present in 5 (Table 3). Therefore, increase the certainty factor of this element, and for the other element [use, device class,
It is possible to select the correct No. 3 syntax tree by lowering the certainty factor of [Enforce].

[0038]

As described above, according to the present invention,
A context space formed by performing a syntactic and semantic analysis on a sentence to decompose it into dependency elements of a word, collecting the dependency elements, and having a relative certainty factor weighted by the number of appearances of the dependency and the evaluation point by the semantic analysis. When data is created and the dependency element having a large relative certainty factor is used as the belief data, when the same dependency factor appears in another sentence, the relative certainty factor of the dependency factor is increased to increase the relative confidence. When a dependency element having the same degree as the dependency data appears in another sentence, a confidence factor adjusting process for lowering the relative confidence factor of the dependency factor is performed, and the context space finally obtained by the process. Since we selected high-confidence data from the data to select the syntax tree, we can resolve the ambiguity of the syntax structure that cannot be resolved by focusing on one sentence and select the correct analysis result. Rukoto can.

[Brief description of the drawings]

FIG. 1 is a flow chart showing an overall process according to an embodiment of the present invention.

FIG. 2 is a flowchart showing the certainty factor adjustment processing according to the embodiment of the present invention.

Claims (1)

[Claims] 1. In a natural language processing method for analyzing a syntactic structure, semantic content, etc. of a natural language as an input, a syntactic and semantic analysis is performed on a sentence to decompose it into word dependency elements, and the dependency elements are In summary, the context space data consisting of the relative certainty factor weighted by the number of appearances of the dependency and the evaluation point by the semantic analysis is created, and the dependency element having the large relative certainty factor is used as the belief data. When the same dependency element appears in another sentence, the relative confidence of the dependency element is increased, and when the dependency element with the same relative confidence as the dependency data appears in another sentence, Performing a certainty factor adjustment process to lower the relative certainty factor of the dependency element, from the context space data finally obtained by the process,
A natural language contextual processing method characterized by selecting high-confidence data and selecting a syntax tree.