JPH0821034B2 - Language analyzer - Google Patents

Description

TECHNICAL FIELD The present invention relates to a language analysis device, and more particularly to a language analysis device useful for an automatic translation device.

Conventional technology For example, when creating a Japanese sentence corresponding to a foreign language sentence such as English, the morpheme of the input English sentence is analyzed, its syntax is analyzed, the sentence structure is converted, and then the Japanese sentence is converted. Generate a translation of.

That is, morphemes such as words constituting an input sentence are analyzed by searching a dictionary, and information such as a part of speech about these morphemes is obtained. After that, based on the obtained information such as the part of speech, the structure of the input sentence is analyzed, that is, the syntactic analysis is performed, by analyzing each word or the modification relation between blocks formed by these words by the grammar rule. . Furthermore, the sentence structure of the input sentence is converted into the order of the Japanese syntax array based on the analyzed syntax, and the Japanese morphemes are generated according to the converted array to generate the Japanese sentence.

In such a morphological analysis performed when generating a translated sentence, the input sentence is divided into dictionary lookup units by delimiters such as spaces, commas, and colons, and a dictionary is searched for this dictionary lookup unit to search for morphemes such as words. We have obtained the part of speech and other information. In this case, many ordinary nouns, verbs, etc. can be stored in the dictionary, so that they can be easily searched and information can be obtained.

However, when such a dictionary is searched for proper nouns, there are so many kinds of proper nouns that it is impossible to store all of them in the dictionary. Therefore, proper nouns not registered in the dictionary could not be processed as proper nouns.

In addition, proper nouns, such as Mr.Brown and Lake Biwa, may sometimes represent proper nouns together with other common nouns. In such a case, proper nouns and common nouns are searched by searching the dictionary. If each is analyzed separately, it cannot be recognized as a proper proper noun.

Conventionally, as described above, there has been a drawback that proper nouns cannot be properly recognized and analyzed.

Aim The present invention solves the above-mentioned drawbacks of the prior art, can process unregistered proper nouns, and can appropriately analyze proper nouns in consideration of the relationship with the preceding and following phrases. The purpose is to provide.

Structure In order to achieve the above object, the present invention has an input means for inputting a character string in a predetermined language, a dictionary means used for searching a character string input from the input means, and a dictionary for the input character string. After searching the means, it has a property information analysis means for analyzing the property information of the character string, and the property information analysis means analyzes the property information of the character string in consideration of the property information of the character strings before and after the character string. It is characterized by doing. Hereinafter, a specific description will be given based on an embodiment of the present invention.

FIG. 1 shows an embodiment in which the language analysis device according to the present invention is applied to an English-Japanese automatic translation device. The present invention can be used for other than morphological analysis of an English input sentence, and is not limited to an English-Japanese translation device that translates English into Japanese, but an automatic translation device that translates one language into another language. It goes without saying that it can be effectively applied to.

This embodiment has an input processing unit 14, and data is input to the input processing unit 14 from the input device 12. The input device 12 includes, for example, a keyboard having character keys such as alphanumeric keys and function keys, an optical character reading device for reading an alphabetic text recorded on paper, a magnetic disk reading device, and the like.

The input processing unit 14 has an input character string buffer 14a, and stores an English input sentence input from the input device 12 in the input character string buffer 14a. The input processing unit 14 is the input character string buffer 1
The input sentence stored in 4a is read and output to the unit cutout unit 16. The unit cutout unit 16 is a functional unit that searches the delimiter table 18 and performs dictionary lookup unit cutout of the input sentence sent from the input processing unit 14. Delimiter table
18 stores delimiters such as spaces and commas.

The unit cutout unit 16 reads the delimiter from the delimiter table 18 and divides the input sentence sent from the input processing unit 14 by the portion in which the delimiter exists, so that the reference dictionary 20
Is divided into character strings that are the unit for searching. The divided character strings are input to the dictionary search unit 22.

The dictionary search unit 22 searches the reference dictionary 20 for the input sentence divided by the dictionary lookup unit sent from the unit cutout unit 16. In the reference dictionary 20, for example, as shown in FIG. 2, an entry of an English character string, its part of speech, property information, etc. are recorded. In addition to the proper nouns shown in the figure, the reference dictionary 20 also stores character strings of other parts of speech, such as verbs and adjectives. In the figure, the record of proper noun as a part of speech means that a registered proper noun process described later is performed, and does not represent a proper proper noun in grammar. Further, the property information indicates what the proper noun represents, and 1
Not necessarily one.

The dictionary search unit 22 searches the reference dictionary 20 for a character string divided in dictionary lookup units, outputs the character string to the proper noun processing unit 24 when the character string is a proper noun, and performs proper noun processing described later. If it is not a proper noun, it is output to the processing unit 36 and stored in the dictionary information storage table 36a of the processing unit 36.

The proper noun processing unit 24 includes an immediately preceding sentence end processing unit 26, an immediately preceding proper noun processing unit 28, and a proper noun own processing unit 30.

The preceding sentence end processing unit 26 determines whether the character string immediately before the character string searched and input by the dictionary search unit 22 is the sentence end, and if the immediately preceding character string is the sentence end, the character to be processed. The uppercase letters at the beginning of the sequence are converted to lowercase letters and sent to the dictionary search unit 22 to cause the dictionary search unit 22 to search the reference dictionary 20 again. The character string that is not searched by this search again is determined as an unregistered proper noun, is sent to the processing unit 36, and is stored in the dictionary information storage table 36a. When the immediately preceding character string is not the end of the sentence, it is sent to the processing unit 36 as a proper noun with unknown property information as described later, and the dictionary information storage table
Register for 36a.

The immediately preceding proper noun processing unit 28 analyzes the property information of the immediately preceding character string sent from the immediately preceding sentence end processing unit 26, and outputs it to the proper noun own processing unit 30. The proper noun processing unit 30 examines the property information of the proper noun to be analyzed, and as described later, when the property information is not registered in either the proper noun or the previous proper noun, the other registered property is registered. The proper noun and the immediately preceding proper noun are collectively analyzed by the information and stored in the dictionary information storage table 36a of the processing unit 36.

The processing unit 36 has a dictionary information storage table 36a, stores the data sent from the immediately preceding proper noun processing unit 28 or the dictionary search unit 22 in the dictionary information storage table 36a, and then reads the stored data and parses it. Output to the unit 38. The syntactic analysis unit 38 performs syntactic analysis of the input sentence that has been morphologically analyzed and read from the dictionary information storage table 36a.

The operation of this apparatus will be described with reference to the flowchart shown in FIG.

First, an English input sentence is read from the input device 12 into the input processing unit 14 (100). The input sentence read by the input processing unit 14 is stored in the input character string buffer 14a. The input sentence stored in the input character string buffer 14a is read by the unit cutout unit 16.

When the input sentence is input, the unit cutout unit 16 reads the delimiter from the delimiter table 18 and cuts it out in dictionary lookup units (102). That is, the character string that constitutes the input sentence that has been input is delimited by parts with delimiters such as spaces and colons, so that the search key character string, which is the unit for searching the reference dictionary 20, is specified from the beginning of the character string. It is divided in order. It is judged whether the divided dictionary lookup unit, that is, the search key character string is the end (104),
If the search key character string still exists (if it is not the end), the search key character string is sent to the dictionary search unit 22.

When the search key character string is sent to the dictionary search unit 22, the dictionary search unit 22 searches the reference dictionary 20 for this search key character string (106). It is judged whether or not there is a search key character string in the entry of the reference dictionary 20 as shown in FIG. 2 (108), and if there is an entry, the part-of-speech information stored in the reference dictionary 20 is read out and the search key is read. It is determined whether the character string is a proper noun (110).

If the search key character string is not a proper noun, the dictionary search unit 22 sends the data read from the reference dictionary 20 to the processing unit 36 and records it in the dictionary information storage table 36a (112). When the data is stored in the dictionary information storage table 36a, the processing unit 3
From 6, the unit cutout unit 16 is input that the data has been saved and the data of the search key character string that was saved immediately before is input. As a result, the process returns to step 102, and the unit extraction unit 16 cuts out the dictionary unit.

When the search key character string is the proper noun in step 110, the dictionary retrieval unit 22 retrieves the data of the proper noun (hereinafter, simply referred to as proper noun) read from the reference dictionary 20 from the dictionary information storage table 36a of the processing unit 36. Unit cutout unit 16
It is sent to the proper noun processing unit 24 together with the immediately preceding search key character string data input to the dictionary search unit 22 via the proper noun processing unit 24 for processing the dictionary registered proper noun (114).

The processing of dictionary registered proper nouns will now be described with reference to the flowchart shown in FIG.

The data sent from the dictionary retrieval unit 22 to the proper noun processing unit 24 is input to the immediately preceding proper noun processing unit 28 through the immediately preceding sentence end processing unit 26. The immediately preceding sentence end processing unit 26 does not function in processing the dictionary registered proper noun.

In the immediately preceding proper noun processing unit 28, the search key character string immediately before the proper noun is a proper noun not registered in the reference dictionary 20,
That is, it is judged whether or not the dictionary unregistered proper noun processing described later has been performed (200), and if it is an unregistered proper noun, the proper noun and the immediately previous unregistered proper noun are all It is determined that the proper noun has property information of the proper noun (202), these data are sent to the processing unit 36, and recorded in the dictionary information storage table 36a (214).

In the immediately preceding proper noun processing unit 28, when it is determined in step 200 that the search key character string immediately preceding the proper noun is not an unregistered proper noun, the search key character string immediately preceding the proper noun is the reference dictionary. It is determined whether the proper noun registered in 20 (204). When the search key character string immediately before the proper noun is the registered proper noun, whether or not the property information of the proper noun immediately before this is unknown (Unknwn), that is, whether it is not registered in the reference dictionary 20 or not. It is judged (206).

If the property information of the immediately preceding proper noun is unknown, the process proceeds to step 202, and the proper noun and the immediately preceding proper noun are all judged to be proper nouns having the property information of the proper noun (202),
The immediately preceding proper noun processing unit 28 sends these data to the processing unit 36. The data sent to the processing unit 36 is recorded in the dictionary information storage table 36a (214).

In step 206 in the immediately preceding proper noun processing unit 28, when it is determined that the property information of the immediately preceding proper noun is not unknown (that is, registered in the reference dictionary 20), these data are processed in the immediately preceding proper noun processing. The part 28 is sent to the proper noun processing unit 30. The proper noun processing unit 30 determines whether the property information of the proper noun is unknown (208). When the property information of the proper noun is unknown, the proper noun processing unit 30 determines that the proper noun and the immediately preceding proper noun are all proper nouns having the property information of the immediately preceding proper noun (210). Data processing part 3
Send to 6. The data sent to the processing unit 36 is recorded in the dictionary information storage table 36a (214).

When the proper noun processing unit 30 determines that the property information of the proper noun itself is not unknown, that is, the proper noun is registered in the reference dictionary 20, the proper noun processing unit 30 determines the proper noun immediately before the proper noun. Separately, it is determined that the proper noun has the property information retrieved from the reference dictionary 20 (212), and the data is sent to the processing unit 36. The data sent to the processing unit 36 is recorded in the dictionary information storage table 36a (214).

Returning to FIG. 3, if there is no search key character string in the entry of the reference dictionary 20 in step 108, it is judged whether the first character of the search key character string is uppercase (116). The dictionary search unit 22 determines that this search key character string is an unregistered word, sends it to the processing unit 36, and records it in the dictionary information storage table 36a (118).

When the first character is a capital letter, the data of this search key character string is sent from the dictionary search unit 22 to the proper noun processing unit 24 together with the data of the previous entry recorded in the dictionary information storage table, and the dictionary is not registered. Proper noun processing is performed (120).

Processing of dictionary unregistered proper nouns will be described with reference to FIG.

The data of the search key character string is sent to the previous sentence end processing unit 26 together with the data of the previous entry recorded in the dictionary information storage table, and the end of the previous entry recorded in the dictionary information storage table in the previous sentence end processing unit 26. It is determined whether is a sentence end candidate (300). Whether or not the sentence end candidate is determined is determined by determining whether or not the end of the previous entry recorded in the dictionary information storage table is a sentence end candidate such as a single period (.).

If the end of the immediately preceding entry recorded in the dictionary information storage table is a sentence end candidate, data is sent from the immediately preceding sentence end processing unit 26 to the immediately preceding proper noun processing unit 28, and the immediately preceding proper noun processing unit 28 causes the dictionary information. The last entry recorded in the storage table is judged to be the end of sentence (302), and the search key character string is set to 1
The uppercase letters are converted to lowercase letters and sent to the dictionary search unit 22.

The dictionary search unit 22 searches the reference dictionary 20 for the search key character string converted to lowercase (304), and determines whether or not there is an entry in the reference dictionary 20 (306). If there is an entry, the dictionary retrieval unit 22 sends the data retrieved from the reference dictionary 20 to the processing unit 36 and records it in the dictionary information storage table 36a (308). If there is no entry, the dictionary search unit 22
The first character of this search key character string is returned to upper case, sent to the processing unit 36 as an unregistered proper noun, and recorded in the dictionary information storage table 36a (310).

In step 300, when the immediately preceding sentence end processing unit 26 determines that the end of the immediately previous entry recorded in the dictionary information storage table is not a sentence end candidate, the data is transferred from the immediately preceding sentence end processing unit 26 to the immediately preceding proper noun processing unit 28. The immediately preceding proper noun processing unit 28 judges that the immediately preceding entry recorded in the dictionary information storage table is not the end of sentence (312). Data is sent from the immediately preceding proper noun processing unit 28 to the proper noun processing unit 30, and the proper noun processing unit 30 determines that the search key character string is a proper noun whose property information is unknown (314).

The proper noun processing unit 30 processes the data immediately before the proper noun processing unit.
Returning to 28, the immediately preceding proper noun processing unit 28 processes the dictionary registered proper noun (316). The processing of the dictionary registered proper noun is the same as that shown in FIG.

Returning to FIG. 3, when the dictionary lookup unit cut out in step 104 is the end, the processing unit 36 outputs the data recorded in the dictionary information storage table 36a to the syntax analysis unit 38 (122), The morphological analysis of the embodiment is completed. The operation of the present apparatus described above will be described with an example of an input sentence.

For example, a case where the input sentence “In Tokyo Station Mr. Walter ...” is input will be described with reference to FIG. First,
Input processing to read this input sentence into the input processing unit 14 (100)
I do. Next, cut out the dictionary lookup unit (102),
The above input sentence is divided into each word by a space. First, the reference dictionary 20 is searched for "In" (106). "I
As for “n”, the entry is not in the reference dictionary 20 as it is, and the processing proceeds to the dictionary registration proper noun. Then, when "In" is changed to "in" and the dictionary is searched, there is an entry (10
8) Since it is not a proper noun (110), the data retrieved from the reference dictionary 20 is recorded in the dictionary information storage table 36a (112).

Next, search the reference dictionary 20 for "Tokyo" (10
6). "Tokyo" has no entry in the reference dictionary 20 (10
8) Since the first character is a capital letter (116), the dictionary end registered proper noun is processed (120). Proceed to FIG.
Since it is “n” and there is no sentence end candidate (300), “In” is recognized as not the end of the sentence (312), and “Tokyo” is unknown (Unkn).
The proper noun (own) is assigned (314), and the dictionary-registered proper noun is processed (316).

Proceeding to FIG. 4, since the immediately preceding “In” is not an unregistered proper noun (200) and is not a registered proper noun (204), the proper noun having its own property information alone, that is, property information unknown Record as the proper noun of (Unknown) (2
16).

Returning to FIG. 3, the reference dictionary for "Station" 20
Is searched (106). Since "Station" has an entry in the reference dictionary 20 (108) and is a proper noun (110), the dictionary registered proper noun is processed (114). Proceeding to Fig. 4, since "Tokyo" just before is an unregistered proper noun (200),
The entire "Tokyo Station" is recorded as a proper noun having the property information "place" of "Station" (202).

Next, the reference dictionary 20 of FIG. 3 is searched for "Wr." (106). "Mr." has an entry in the reference dictionary 20,
Since it is a proper noun (110), the dictionary registered proper noun is processed (114). Proceeding to FIG. 4, the immediately preceding "Station" is not an unregistered proper noun (200), is a registered proper noun (204), and the property information "place" is not unknown (206).
Since "Mr." has property information of "person" and is not unknown (208), "Mr." is registered alone as a proper noun of property information "person" (212).

Next, returning to FIG. 3 again, reference dictionary for "Walter"
Search for 20 (106). Since "Walter" has an entry in the reference dictionary 20 (108) and is a proper noun (110), the dictionary registered proper noun is processed (114). Proceeding to FIG. 4, the immediately preceding "Mr." is not an unregistered proper noun (200), is a registered proper noun (204), and the property information is "person" and not unknown (206), "Walter". Since the property information of is unknown, (20
8), "Mr. Walter" is recorded together as a proper noun with property information "person" (210).

As described above, according to the present embodiment, the English input sentence is divided into the search key character strings, and first searched by the reference dictionary 20,
When the reference dictionary 20 has an entry as a proper noun, the registered proper noun is processed. The processing of the registered proper noun considers the entry just before recorded in the dictionary information saving table, and when the entry immediately before recorded in the dictionary information saving table is a proper noun, immediately before being recorded in the dictionary information saving table. Entry and the property information of the proper proper noun are examined, and if either property information does not exist, the other property information is added, and if both have property information, each has its own property information. Proper noun.

Therefore, it is possible to appropriately add property information to proper nouns without property information, and to narrow down the added property information to more appropriate property information. As a result, in the later parsing, the parsing can be effectively performed and appropriate translation can be performed.

In addition, regarding a character string not registered in the reference dictionary 20, when it is determined that the first character is uppercase and the previous character string is the end of sentence, the uppercase is converted to lowercase and the reference dictionary 20 is searched again. Therefore, the reference dictionary 20 can be searched for the character string at the beginning of the sentence. Furthermore, if a character string starting with a capital letter appears at a position that is not the beginning of a sentence, it is determined as a proper noun, and the property information of this proper noun is given by this when there is a proper noun with property information registered before and after that. It Therefore, proper nouns not registered in the reference dictionary 20 can be analyzed to some extent.

According to the present invention, for a proper noun whose property information is not registered in the dictionary, if there is a proper noun with the property information registered before and after the proper noun, the property information of the registered proper noun is used. Adds property information to unregistered proper nouns, and analyzes property information of proper nouns by the property information of preceding and following proper nouns, so gives proper property information to proper nouns and interprets the input sentence effectively. Can be done.

[Brief description of drawings]

1 is a block diagram showing an embodiment of a language analysis device according to the present invention, FIG. 2 is a diagram showing an example of data stored in the reference dictionary shown in FIG. 1, and FIG. 3 is a diagram showing the device shown in FIG. FIG. 4 is a flowchart showing processing of a dictionary-registered proper noun in the operation shown in FIG. 3, and FIG. 5 is processing of a dictionary-unregistered proper noun in the operation shown in FIG. FIG. 6 is a diagram showing an example in which an input sentence is processed in the apparatus of FIG. 1 and the data stored in the dictionary information storage table changes. Description of main part code 14 …… Input processing unit 16 …… Unit cutout unit 18 …… Delimiter table 20 …… Reference dictionary 22 …… Dictionary search unit 24 …… Proper noun processing unit 26 …… Previous sentence end processing unit 28 ...... Previous proper noun processing unit 30 …… Proper noun processing unit 36 …… Processing unit 36a …… Dictionary information storage table

Claims (2)

[Claims] 1. An input unit for inputting a character string of a predetermined language, a dictionary unit used for searching the character string input from the input unit, and a search for the dictionary unit for the input character string. A search unit, and, as a result of the search by the search unit, adds property information to a character string that has not been registered in the dictionary unit as property information indicating a property that changes according to the context,
A character string before and / or after the character string when the property information is added to the character string whose property information is not registered in the dictionary means. When the property information is registered, the property information of the character string before and / or after is used,
A language analysis device characterized by adding property information to the unregistered character string. 2. The apparatus according to claim 1, wherein the character string to which the property information is added by the analyzing means, and before and / or after the character string used to add the property information to the character string. The character string is a proper noun, which is a language analysis device.