JPS6279569A - Non-sentence extracting device - Google Patents

Abstract

PURPOSE:To improve a processing speed of syntactic analysis and to reduce the capacity of a grammar dictionary for the syntactic analysis by providing a non-sentence deciding means that compares the result of a statistics by plural kinds of statistics means with a decision condition and decides whether a sentence is the non-sentence or not. CONSTITUTION:A statistic device 7 counts the number of characters of an inputted sentence, posting them on a decision table 12, and counts the number of clauses, posting them on the decision table 12. At the next, a statistics device 8 counts the parameter of a parameter list 10 in the inputted sentence and a statistics device 9 counts the parameter of a parameter list 11, posting them on the decision 12 respectively. By comparing the statistics result of the decision table 12 posted by the above process and a threshold value, the decision of the non-sentence is performed, and when it is decided as the non-sentence, a data is transferred to an output device 5, displaying it is the non-sentence and when it is not the non-sentence, the syntactic analysis, a conversion and a generation, etc. are performed by a translation device 4, outputting other language to the output device 5. Thereby, the extraction of the non-sentence against the inputted sentence before the process of the syntactic analysis and also, the speed up of the process of the syntactic analysis and the saving of the capacity of the grammar dictionary for the syntactic analysis can be obtained.

Description

[Detailed Description of the Invention] Technical Field The present invention relates to a non-sentence extraction device, and particularly in natural language processing such as a machine translation system, when an input sentence has considerable ambiguity as Japanese, or a sentence that is completely strange as Japanese. The present invention relates to a non-sentence extraction device suitable for mechanically extracting non-sentences (hereinafter referred to as non-sentences).

Prior Art Conventionally, in machine translation systems, input Japanese sentences are first subjected to morphological analysis, then non-sentential analysis, etc.
The non-sententially parsed sentence will be translated into another language (eg, English, French).

However, since not all input Japanese sentences are correct Japanese, a check is made to see if they are non-sentences during the above-mentioned non-sentence analysis process. Alternatively, a method is adopted in which a non-sentence check is not performed and an input text is directly analyzed as a non-sentence and translated.

In this method of checking non-sentences during non-sentence analysis,
There was a problem in that unnecessary processing was performed until a non-sentence was found. If non-sentence analysis is performed using a grammar that adds tt to non-sentences, the non-sentence analysis becomes complicated and the processing speed decreases.

On the other hand, if non-sentence analysis is performed without checking non-sentences and a translation result is output, the translation result may be completely incorrect.

In this way, if a translation system is configured using a grammar that checks non-sentences during non-sentence analysis, the cost of the system will increase, and even if it takes a lot of time to analyze non-sentences, no results will be obtained in the end. There is a risk.

Purpose The purpose of the present invention is to solve such conventional problems, and in natural language processing such as machine translation systems, to perform non-sentence analysis on input sentences before performing non-sentence analysis processing with a simple hardware configuration. The object of the present invention is to provide a non-sentence extraction device that can extract the following, increase the processing speed of non-sentence analysis, and reduce the capacity of a grammar dictionary for non-sentence analysis.

Configuration In order to achieve the purpose of the non-sentence extraction device of the present invention, in a machine translation system that machine-translates a Japanese sentence into another language, conditions for determining it as a non-sentence are determined in advance and input A plurality of types of statistical means that take statistics corresponding to the conditions of the Japanese sentences that have been written, and a non-sentence judgment that compares the statistical results of the plurality of types of statistical means with the above conditions to determine whether or not it is a non-sentence. It is characterized by the provision of means.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings.

FIG. 2 is a schematic configuration diagram of a machine translation system to which the present invention is applied.

In FIG. 2, 1 is an input device for inputting natural language sentences, and 82 is a character input device such as a keyboard or OCR, which divides the input sentence into m words using a Japanese dictionary and connection table. This is a morphological analysis device that adds part of speech and other information. There are three types of non-sentence extraction equipment according to the present invention, which uses morphological statistics (large number of characters, many clauses, etc.), parameter lists 1 to 10, and parameter list 11 to extract grammatical,
It is determined whether the text has the characteristics of a non-sentence from a usage point of view, and if it is a non-sentence, this is displayed to the user, and if it is not a non-sentence, it is input to the translation device 4 and translated. 4 is a translation device;
Non-sentential analysis of input sentences using translation grammars and translation dictionaries,
Conversion, generation processing, and conversion from Japanese to other languages (e.g. English,
Translate to French).

5 is an output device that outputs the non-sentence extracted by the non-sentence extraction device w3 or the foreign language translated by the translation device 4, and is, for example, a CRT display.

Such as printers.

FIG. 1 is a block diagram of a non-sentence extraction device showing an embodiment of the present invention.

In FIG. 1, it is a non-sentence judgment device that judges non-sentences using the statistical results of the six-digit judgment table 12 and a threshold, and the processing is as follows:
Proceed to statistics device 7 → statistics device gf, 8 → standard fil device W9,
If any one of the statistical devices determines that it is not a sentence, the process is interrupted and displayed to the user. At this time, the reason why it was determined to be non-text is also displayed.

7 counts the number of characters and the number of phrases in the input sentence and makes a judgment table 12.
8 is a statistical device that uses the parameter list 10 to take statistics on each parameter in a sentence; 9 is a statistical device that uses the parameter list 11 to take statistics on each parameter in a sentence; 10 is a statistical device that takes non-sentence statistics. 11 is a parameter list in which words, attributes, and attribute values that are keys for determination are filled in; 11 is a parameter list in which relationships between words that are keys to judge non-sentences; 12 is a process name and parameters This is a judgment table that has items such as name, statistical result, and threshold value, and the statistical result is the count number of parameters in each process.

FIG. 3 is a diagram showing an example of the configuration of parameter lists 1 to IO in FIG. 2. This parameter list 10 is composed of three sets of word attribute names and one attribute value representation. For example, the hiragana "no" has an attribute name "part of speech", an attribute value "case particle",
Construct the parameter list from the notation "parameter".

FIG. 4 is a diagram showing an example of the configuration of the parameter list 1 to 11 in FIG. 2. This parameter list 1-11 is tli
It is used to show the context of a single word clause. First, the list is divided into two parts, front and back. The first part consists of an attribute name and two attribute values, and the second part consists of an attribute name and two attribute values. , consists of evaluation values.

FIG. 5 is a diagram showing a configuration example of the determination table 12 according to the present invention. This judgment table 12 is composed of a process name, a parameter name, a statistical result, and a threshold value. Statistical process l indicates a process by the statistical device 7, and the parameter name includes, for example, the number of characters in a sentence, the number of characters, and the threshold value. Enter the number of bunsetsu and determine the 5 threshold value in advance. Here, the threshold value for the number of characters is set to 0.6 for 40° number of clauses/number of characters. However, it is not limited to this value. Also, the main entrance,
- The results are filled in with the results of statistics taken by the statistical device 7 for the sentences that were actually input. For example, the number of ``no'', the number of discontinuations, and the number of parallel particles are entered in the parameter name, and the threshold value is 3' for each.

13 g, l 21. This numerical value is also not limited as in the above-described subsequent opening process 1.

Statistical processing 3 indicates statistical processing by the statistical device 3, in which the number of "r~ is" is entered in the parameter name, and the threshold value is 1'.

Therefore, when the non-sentence determining device 6 looks at the above-described determination table 12 and the statistical result is equal to or higher than the threshold value, the input sentence is deemed to be a non-sentence.

FIG. 6 is a flowchart showing non-sentence extraction processing according to the present invention. This shows the operation of the non-sentence extraction device shown in FIG. The process will be explained below according to the flowchart shown in FIG.

First, the number of characters in the sentence morphologically analyzed by the morphological analyzer W2 is counted and entered in the judgment table 12.Step 60
1) Count the number of phrases and enter them in the judgment table 12 (step 602), compare the statistical results of the judgment table 12 with the threshold value, and use the non-sentence judgment device 6 to judge whether it is a non-sentence (step 603). If the non-sentence determining device 6 determines that it is a non-sentence (step 604), the output device 5 shown in FIG. 2 displays it to the user. When the non-sentence determining device 6 determines that it is not a non-sentence (step 604), the statistical device 8 counts each parameter in the input sentence in the parameter list 10 and enters it in the determination table 12 (step 605). The device 6 compares the statistical results of the determination table 12 with the threshold value and determines whether it is a non-sentence (step 606). If it is determined that it is a non-sentence (step 6
07), display the non-sentence by the output device 5 (step 6
11). When it is not a non-sentence (step 607), the command R
1 device 9 counts each parameter in the input sentence with respect to the parameter list 11 and enters it in the judgment table 12 (step 608). A determination is made (step fi09). If it is determined that it is a non-sentence (step 6
10), displayed to the user by the output device 5 (step 6)
11). If it is determined that it is not unwanted (Step 6)
0), the process ends.

FIG. 7 is a diagram showing an example of an input sentence. Although it is immediately obvious that ``a book on the bookshelf on his desk'' as shown in this example has four clauses, an ambiguity arises when this sentence is analyzed. In other words, ■His desk, 1 bookshelf on the desk, 1 bookshelf,
Q) Books on his bookshelf, 2 bookshelves on the desk bookshelf, fi + his books, 1 bookshelf on the desk bookshelf.

■One possible interpretation is his desk, 2 books on the desk, 2 books on the shelf.

Therefore, such input sentences are extracted as non-sentences,
Processing degradation can be prevented by not performing non-sentence analysis. The non-sentence extraction process until this "book on the bookshelf on his desk" is deemed to be a non-sentence by the book-nonsentence extraction device 3 will be explained below. First, when "a book on the bookshelf on his desk" is inputted by the input device 1, the morphological analysis is performed by the morphological analysis device 2.
It will be done.

This morphologically analyzed sentence "a book on the bookshelf on his desk" is input to the non-sentence extraction device 3 and checked to see if it is a non-sentence. The non-sentence extraction process will be described below with reference to the non-sentence determining device 3 shown in FIG.

First, for the morphologically analyzed sentence "a book on the bookshelf on his desk," the statistical device 7 calculates the number of characters, the number of clauses, and the number of clauses/number of characters. Then, the results are entered in the determination table 10.

The statistical result is that number of characters = 82, number of clauses = 41, number of clauses/number of characters = 0.5. An example of this judgment table 12 is shown in FIG.

The non-sentence determining device 6 looks at the determination table 12 as shown in FIG. 8 and determines whether the statistical result is greater than a threshold value,
If it is above the threshold value, it is considered a non-text. However, in this example, since both are smaller than the threshold value, it is assumed that the result of Statistics 1 is not a non-sentence. Processing proceeds to the statistical device 8.

The statistical device 8 uses the parameter list 10 to search for "a book on the bookshelf on his desk" using the parameters, sends it to Callan 1, and enters it in the determination table 12. The statistical results obtained by this statistical instrument W8 are shown in FIG. Now Harameter List 1
Among the parameters of 0, pay attention to the parameter whose part of speech is a case particle and whose notation is "no" (for the sake of explanation, other parameters are ignored). Then, the input sentence "books on the bookshelf on his desk" contains three case particles "no".

Enter this in the judgment table 12. Therefore, the non-sentence determining device 6
Now, if we compare the statistical result of the case particle "no" with its threshold value, the statistical result is equal to the threshold value, so "the book on the bookshelf on his desk" may be a non-sentence, and we cannot process it. The process is interrupted at this point, and the output device 5 shown in FIG. 2 displays to the user that the "book on the bookshelf on his desk" is a non-text, and also displays the reason why it is determined to be a non-text.

FIG. 10 is a configuration diagram of a non-sentence determining device 3 showing another embodiment. Here, since the configuration is the same as that in FIG. 1 except for the non-sentence determining device 6, the explanation will be omitted, and the non-sentence determining device w6 will be explained.

Once the non-sentence judgment device w6 transfers the processing to the statistical device 7, it uses the judgment table 1 until it receives a processing end signal from the statistical device W9.
I'm not going to see 2. In other words, statistical device 7 → statistical device 8 →
When the processing with the statistical device 9 is completed, go to the judgment table 12 and judge whether it is a non-sentence, and if it is judged to be a non-sentence, extract all the statistical results from the judgment table 12 that are equal to or greater than the threshold value -L. , all the non-sentential factors are displayed to the user by the output device 5.

FIG. 11 is a flowchart of non-sentence extraction processing showing another embodiment. This shows the non-sentence extraction process by the non-sentence extraction device 3 of FIG. 10. Hereinafter, the explanation will be made according to the brooch yard shown in FIG.

First, the statistical device 7 counts the number of characters in the input sentence,
Fill in the judgment table 12 (step 1101), count the number of clauses, and write in the judgment table 12 (step 1102)
. Next, the statistical device 8 counts each parameter in the input sentence for the parameter list 10, and the judgment table 12
(Step 1103). Furthermore, the statistical device 9
With respect to the parameter list 11, each parameter is counted in the input sentence and entered in the judgment table 12 (step 1104). Judgment table 1 filled in through the above process
The statistical result of step 2 is compared with the threshold value to determine whether it is a non-sentence (steps 1105 and 1106). If it is determined that it is a non-sentence, the data is transferred to the output device 5 to display that it is a non-sentence (step 1107).

If it is not a non-sentence, the translation device 4 analyzes it as a non-sentence.

It performs conversion, generation, etc., and outputs the other language to the output device 5.

Here, the difference between the other embodiments and this embodiment is that a plurality of factors that are determined to be non-sentences are displayed to the user. In the judgment table 12, all the items of statistical processing 1゜2.3 are filled in, so the non-sentence judgment device 6 looks at this and
Non-sentence factors are extracted (those whose statistical results are above the threshold) and all their parameter names are sent to the output device 5 and displayed.

In this way, in this embodiment, the hardware determines whether or not the morphologically analyzed input sentence is a non-sentence by comparing it with predetermined non-sentence conditions, and then Since non-sentence analysis is performed, the capacity of grammar dictionaries and the like required for non-sentence analysis can be reduced. Moreover, since this non-sentence extraction can be processed mechanically to some extent, the processing speed can be improved.

Furthermore, it eliminates the need for unnecessary processing such as non-sentence analysis.
The processing speed of non-sentence analysis can also be improved. Furthermore, in this embodiment, since input sentences other than those that are considered non-sentences are translated, the quality of the translated sentences can be improved in the end.

According to the present invention, simple Ql-7'J! is used in natural language processing such as machine translation systems. Perform non-sentence analysis processing using hardware configuration 1) In 1F, non-sentences can be extracted from input sentences, the processing speed of non-sentence analysis can be increased by -16, and a grammar dictionary for non-sentence analysis can be used. You can save on transportation costs.

[Brief explanation of drawings]

Fig. 1 is a block diagram of a diplomatic extraction device showing a fourth embodiment of the present invention, Fig. 2 is a block diagram of a machine translation system to which the present invention is applied, and Figs. FIG. 5 is a diagram showing a configuration example of a judgment table according to the present invention, and FIG. 6 is a diagram showing a non-sentence extraction process by the diplomatic extraction device of FIG. The figure shows an example of an input sentence, FIGS. 8 and 9 do not show an example of the structure of a judgment table showing the result of the input sentence synthesis 81 in FIG. 7, and FIG. 10 shows another embodiment of the present invention. 11 is a flowchart showing the non-sentence extraction processing by the diplomatic extraction device of Fig. 10. 4. Brief explanation of the drawings 1: Input device, 2: Morphological analysis Device, 3: Diplomacy extraction device, 4: Translation device, 5: Output device, 6: Non-sentence determination device, 7
, 8.9: Statistical equipment, 11.12: Parameter list. Fig. 1 Fig. 2 Fig. 1b - Fig. 3 Fig. 4 Fig. 5 Fig. 6 Fig. 7 Fig. 8 Fig. 9 Fig. 10
Published. , Li statistical device
Fixed value 9 views 11 statistics device 4! II Schedule Figure 11

Claims (4)

[Claims] (1) In a machine translation system that performs machine translation of Japanese sentences into other languages, multiple types of statistical means are used to determine conditions for determining non-sentences in advance and to obtain statistics corresponding to the conditions for input Japanese sentences. , and non-sentence determining means for comparing the statistical results obtained by the plurality of types of statistical means with the above-mentioned conditions to determine whether or not it is a non-sentence. (2) The non-sentence extraction device according to claim 1, wherein the statistical means counts the number of characters and the number of phrases in the Japanese sentence. (3) The above-mentioned statistical means counts each parameter using a parameter list in which parameters consisting of words, attributes, and attribute values for determining non-sentences are entered. The non-sentence extraction device described in Section 1. (4) The statistical means counts each parameter using a parameter list in which parameters consisting of word relationships for determining non-sentences are entered. non-sentence extraction device.