JPH052601A - Preprocessor for japanese language processing - Google Patents

Abstract

PURPOSE:To improve processing speed and exactness by judging whether a repetitive mark is converted into a character previous to said mark or not, converting it, and judging whether a hiphen mark existing after KATAKANA (square form of KANA (Japanese syllabary)) is corrected to a long vowel mark or not so as to correct it. CONSTITUTION:A word including the repetitive mark is registered in a word dictionary 24 by correcting the repetitive mark to a word converted into the previous word. A repetitive mark processing part 22 provided in a front end processor 21 detects the repetitive mark existing in the Japanese sentence 10 of a described natural language, judges whether the repetitive mark is converted into the previous character or not and converts is. A hyphen mark processing part 23 detects the hyphen mark existing after KATAKANA in the described natural language and judges whether the hyphen mark is corrected to the long vowel mark or not so as to correct it.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a preprocessing device for Japanese processing such as machine translation.

[0002]

2. Description of the Related Art In conventional machine translation, Japanese natural language is processed by an analysis system, decomposed into words, and converted into another language by a dictionary. At this time, the repeated symbol is interpreted as a symbol or converted to another character, it does not become a correct word, and that word does not exist in the dictionary, so correct analysis cannot be done and translation cannot be done There was a problem.

For example, when "Americans" is input correctly, it is correctly decomposed into words, "US" and "Zosin". Although "US" is in the dictionary, "Zosin" is not in the dictionary. Translation fails. However, in reality, the characters "" are treated as symbols, so "US people" is decomposed into the input "US", "people" and "people", and the translation fails. Such a failure lies in the fact that the repeated symbols “s” could not be preprocessed correctly. That is, a repeat symbol must be decomposed into words after it has been replaced by the preceding character to be repeated.

Further, when a hyphen symbol is erroneously input when a katakana character word in a natural language is input, the hyphen symbol existing in the katakana character word is not regarded as a long sound symbol, and the katakana character is not included. Since it is processed as a combination of and hyphen symbols, the corresponding word does not exist in the dictionary, and it is processed as an unregistered word, resulting in the fact that translation cannot be performed. For example, if "Computer" is mistakenly entered as "Computer", it will be treated as a combination of two words "Compu" and "Ta" and the translation will fail.

FIG. 3 is a diagram showing a configuration of Japanese language processing in conventional machine translation. In the figure, 10 is a natural language Japanese sentence input by the Japanese processing device 18, and 11 is a natural language Japanese sentence in a form suitable for the Japanese processing device 18 for inputting into the Japanese processing device 10. A human preprocessing unit for converting the input, 12 is an input natural language Japanese sentence 10
A word analysis unit that analyzes (hereinafter abbreviated as input data) into word elements, 13 is a structure analysis unit that determines a sentence structure from the sequence of words in the input data 10 decomposed into words, and 14 is a part of speech of a Japanese word , A word dictionary that holds word attributes such as translated words,
Reference numeral 15 is a translation processing unit that replaces the input data 10 with translated words by the dictionary 14, 16 is a post-processing unit that rearranges the input data 10 converted into translated words, 18 is a word analysis unit 12, a structure analysis unit 13,
The Japanese processing device 18 includes a word dictionary 14, a translation processing unit 15, and a post-processing unit 16.

In the Japanese language processing apparatus 18 having the above-described structure, the input data 10 is decomposed into word elements by the word analysis unit 12, and the sentence structure is analyzed from the word array of the input data 10 decomposed by the structure analysis unit 13. And the translation processing unit 15 inputs the input data 10
Was replaced with the translated word by the word dictionary 14, and the input data 10 converted into the translated word by the post-processing unit 16 was rearranged to complete the sentence.

Here, conventionally, a manual preprocessing unit 11 for converting a Japanese sentence 10 in a natural language into a form suitable for the Japanese language processing apparatus 18 for inputting into the Japanese language processing apparatus 18 is required. This is because the Japanese sentence 10 in natural language has the characteristic of natural language that it is not always described according to the grammar, so the correction is done manually.

As the object of such correction, there are the above-mentioned repeated symbol processing and hyphen symbol processing, which reduce the processing speed of machine translation or erroneously process the accuracy of translation. It was.

[0009]

However, the conventional manual preprocessing of Japanese processing has a problem in that the processing speed of machine translation is reduced or the accuracy of translation is impaired by misprocessing.

In view of the above points, the present invention has an object to provide means for improving the processing speed and accuracy by pretreatment by a machine.

[0011]

The above-mentioned problems can be solved by a preprocessing device for Japanese language processing configured as follows. FIG. 1 is a principle diagram of the present invention. (1) In a Japanese processing preprocessing device that decomposes a natural language Japanese sentence 10 into words included in a word dictionary, and collates and translates the words into a word dictionary. The word converted to the character is corrected and registered in the word dictionary in advance, and the repeat symbol existing in the written Japanese sentence 10 of the natural language is detected, and the repeat symbol is set to the character in front of it. A repeat symbol processing unit 22 for determining whether or not to convert is provided, and a word including a repeat symbol existing in a written Japanese sentence of a natural language is converted into a same-character repeated word included in the word dictionary and translated. To configure.

(2) A hyphen symbol existing after a katakana character in a described natural language is detected in a preprocessing device for Japanese processing for decomposing and translating a natural language Japanese sentence into words included in a word dictionary. Then, a hyphen symbol processing unit 23 for determining whether to correct the hyphen symbol to a long-sound symbol is provided, and a katakana word including the hyphen symbol existing in the described natural language is included in the word dictionary. It is configured to be corrected and translated into a katakana word including.

[0013]

(Operation) (1) A word including a repeating symbol is corrected to a word obtained by converting the repeating symbol into a character immediately before it and registered in the word dictionary 24. The repeat symbol processing unit 22 provided in the preprocessor 21 detects a repeat symbol existing in the written Japanese sentence 10 of the natural language, and determines whether or not to convert the repeat symbol with the preceding character. And then convert.

As a result, a word containing a repetitive symbol existing in the written Japanese sentence in the natural language is converted into the same-character repetitive word included in the word dictionary, which enables translation.

(2) The hyphen symbol processing unit detects a hyphen symbol existing after a katakana character in the described natural language, and determines whether or not the hyphen symbol should be corrected to a long sound symbol and corrects it.

Therefore, the katakana word including the hyphen symbol existing in the described natural language is corrected to the katakana word including the long sound symbol included in the word dictionary, so that it can be translated.

[0017]

FIG. 2 is a block diagram showing the configuration of an embodiment of the present invention. In the figure, 22 is a repetition symbol processing unit that detects a repetition symbol existing in a written Japanese sentence (10) in natural language, determines whether or not to convert the repetition symbol with a preceding character, and performs conversion. , 23 detect a hyphen symbol existing after a katakana character in the written natural language, and determine whether or not to correct the hyphen symbol to a long-sound symbol, and 21 corrects the hyphen symbol. And a hyphen symbol processing unit 23 for automatically performing preprocessing, 24 is a word dictionary in which a word containing a repeating symbol is replaced with a word in which the repeating symbol is converted to a character in front of the word and registered. is there. In addition, the same reference numerals as those in FIG. 3 are the same.

The repetition symbol processing unit 22 of the preprocessing device 21 recognizes that "is" is a repetition symbol when there is a sentence "American people" in the input data 10, and the repetition symbol for the Kanji character is used. Therefore, it is judged that the repetition of the preceding kanji "country" has been specified, and "every"
Is replaced with "country". Then, the input data 10 "Americans" is converted into "Americans". The converted "American citizen" can be decomposed into two words "American" and "national" by the word analysis unit 12, and the structure analysis unit 13 determines that it is a noun clause, and the translation processing unit 15 At `` The United States
converted to "people".

Repeat symbols may be used within the same word. For example, the word "Chibato" is converted to "Chakuto" by the repeat symbol processing unit 22, so the word dictionary 24
In the inside of "Kimonoto", it is necessary to replace "Kimonoto" with "Kimonoto". Further, the repetition symbols include "A" and "A" in addition to "A", which are a repetition symbol for Kanji and a repetition symbol for Hiragana, respectively, and perform similar processing.

In the long sound symbol processing section 23 of the preprocessing device 21,
If it is detected that there is a hyphen symbol immediately after the katakana when there is a sentence "Computer" in the input data 10, it is first cut in front of the hyphen symbol and regarded as a word up to that point, and the word dictionary 24 is displayed. If there is a relevant word by reference, it is treated as a hyphen symbol. If the word is not found by referring to the word dictionary 24, the hyphen symbol is converted to a long sound symbol, and it is regarded as a part of the katakana word. The hyphen symbol is regarded as an erroneous long-sound symbol and is replaced with a long-sound symbol and sent to the next word analysis unit 12. By automatically performing such processing by the preprocessing device 21, the preprocessing that has been conventionally performed manually can be mechanized, and the processing speed and translation accuracy can be improved.

[0021]

As is apparent from the above description, according to the present invention, there is a remarkable industrial effect that the pretreatment conventionally performed manually can be mechanized, the processing speed can be improved, and the translation accuracy can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing the principle of the present invention.

FIG. 2 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 3 is a diagram showing a configuration of Japanese processing in conventional translation.

[Explanation of symbols]

10 Natural language Japanese sentence or input data 11 Manual preprocessing unit 12 Word analysis unit 13 Structure analysis unit 14 Dictionary 15 Translation processing unit 16 Postprocessing unit 18 Japanese processing 21 Preprocessing device 22 Repeat symbol processing unit 23 Hyphen Symbol processing unit 24 word dictionary

Claims (2)

[Claims] 1. A preprocessing device for Japanese processing for decomposing a natural language Japanese sentence (10) into words included in a word dictionary, collating the words with a word dictionary, and translating the words. Correct the symbol into the word converted to the character just before it and register it in the word dictionary in advance,
A repetition symbol processing unit (22) is provided which detects a repetition symbol existing in the written Japanese sentence (10) of the natural language and judges whether or not to convert the repetition symbol into a character before it. A preprocessing device for Japanese processing, characterized in that a word including a repetitive symbol existing in a written Japanese sentence in a natural language is converted into a same-character repetitive word included in the word dictionary and translated. 2. A hyphen symbol existing after a katakana character in the described natural language is detected in a preprocessing device for Japanese processing for decomposing and translating a natural language Japanese sentence into words included in a word dictionary. , A hyphen symbol processing unit that determines whether to correct the hyphen symbol to a long-sound symbol and corrects the hyphen symbol
(23) is provided, and the Japanese processing is characterized in that a katakana word including a hyphen symbol existing in the described natural language is corrected and translated into a katakana word including a long sound symbol included in the word dictionary. Processing equipment.