JPH0231422B2 - - Google Patents

Description

[Detailed Description of the Invention] (Technical Field) The present invention relates to an independent word detection method, and in particular, the present invention relates to an independent word detection method, and in particular, the present invention relates to an independent word detection method, and particularly to This invention relates to an independent word detection method for detecting independent words from character strings containing kanji and kana.

(Prior Art) Character strings containing kanji, kana, and kanji in Japanese sentences do not have the custom of separating words into words as in English sentences, and are usually written in solid writing. Therefore, when mechanically analyzing a character string containing kanji and kana in a Japanese sentence using an electronic computer, first, the given character string containing kanji and kana is compared with the notation of each word in a word dictionary. It is necessary to detect words from a given string of characters mixed with kanji and kana.

Here, adjunctive words have fewer words than independent words (the number of adjunct words is from several hundred to several thousand words, and the number of independent words is tens of thousands or more) and appears frequently, so conventionally, a word dictionary is used as an independent word dictionary. First, the attached word is detected by searching the attached word dictionary, and then the remaining character strings that are not recognized as attached words are checked against the independent word dictionary to detect independent words. By doing so, a method is adopted that aims to improve processing efficiency by minimizing the number of times the independent word dictionary is searched, which requires time. FIGS. 1a and 1b are diagrams showing first and second examples of independent word detection using a conventional independent word detection method, respectively. In the first example, the character strings ``wo'' and ``mu'' are searched in the adjunct dictionary.
are recognized as adjunctive words “wo (case particle)” and “mu (five-stage conjugated final form),” respectively, and an independent word dictionary is created for the remaining character strings “hon” and “yomi” that were not recognized as adjunct words. Independent word detection is performed by matching with the independent word “book (noun)”.
and yomi (five-stage verb stem)" are detected. These independent words can be connected with the adjacent "(case particle)" or "mu (five-stage conjugated final final form)", and the correct independent word is detected. It is considered to have been detected (indicated by a circle).

However, in the conventional independent word detection method as described above, the starting position of matching a given character string containing kanji and kana with the notation part of a self-reliance dictionary is divided into character strings that have already been recognized as words and character strings that have not yet been recognized as words. Since it is set at the boundary between Kana and non-Kana characters,
As in the second example, if some kana characters constituting an independent word were recognized as attached words, there was a drawback that the correct independent word could not be detected (indicated by an x). That is, in the character string ``for example'' in Figure 1b, ``eba'', which is originally a part of the independent word ``for example (adverb)'', is replaced by the adjunct word string ``e (five-stage conjugated ending hypothetical form)'', ``ba ( Since the character string ``Example'' must be checked against the independent word dictionary, the correct independent word will not be detected. In this case, it is possible to obtain the desired independent word by performing an independent word dictionary search while shifting the range of character strings in which the independent word is presumed to exist, such as "example" → "example" → "for example". However, in the character string ``sudden rain shower'' in Figure 2b, the range of character strings in which the independent word is presumed to exist is ``shower rain'' →
“Shower rain” → “Ka rain” → “Ka rain” →……→
The drawback was that a blind search had to be performed, shifting the search to "showers", which resulted in extremely low search efficiency.

(Objective of the Invention) The object of the present invention is to eliminate the above-mentioned drawbacks by setting the matching start position to the last non-kana character in an independent word that includes non-kana characters in the notation, and to distinguish between kana characters and non-kana characters. An object of the present invention is to provide an independent word detection method that can correctly and efficiently detect independent words that are written in a mixed manner.

(Structure of the Invention) According to the present invention, an independent word is detected from a given character string containing kanji and kana by comparing the given character string containing kanji and kana with the notation of each independent word in an independent word dictionary. In the independent word detection method,
The main notation part of the notation part of an independent word whose notation contained in the independent word dictionary includes a non-kana character, after removing the last kana character string from the notation part, and the part that is the last kana character string. and a residual notation part,
When detecting an independent word that includes a non-kana character in the notation from the given kanji-kana mixed character string, start matching the possible position of the last non-kana character in the independent word that includes a non-kana character in the notation. a collation start position setting means for setting a position, and a main unit for searching the independent word dictionary for an independent word whose main notation matches a character string forward from the collation start position in the given kanji-kana mixed character string. a notation part search means, and a residual notation that matches the character string after the matching start position in the given kanji-kana mixed character string with the remaining notation part for each independent word searched by the main notation part search means. and a part matching means for detecting an independent word including a non-kana character in the notation by starting matching from the last non-kana character in the independent word including the non-kana character in the notation. An independent word detection method is obtained.

(Example) Next, the present invention will be described with reference to FIGS. 2 to 7.

FIG. 2 is a block diagram of an apparatus showing an embodiment of the independent word detection method of the present invention, FIG. 3 is a flowchart showing the processing operation of each part in FIG. 2, and FIG. 4 is an independent word detection method of the present invention. Figure 5 is a block diagram showing an example of the configuration of the non-kana independent word detection section in Figure 2, and Figures 6a and b are the collation in Figure 5. FIG. 7 is a flowchart showing the details of the process 34 in FIG. 3.

In FIG. 2, the device includes a character string input section 21 for inputting a character string mixed with kanji and kana given as a target for independent word detection, and a character string storage section 22 for storing the character string from the character string input section 21. , an independent word determination unit 23 that determines whether the independent word to be detected from the character string read from the character string storage unit 22 is an independent word expressed in kana or an independent word containing non-kana characters in the notation. , a kana independent word dictionary storage unit 24 that stores a kana independent word dictionary that stores independent words written in kana, and an independent word determination unit 23 that is activated when it is determined that an independent word written in kana should be detected. The given kanji-kana-mixed character string is obtained by comparing the notation part in the kana-independent word dictionary in the dictionary storage unit 24 with the partial kana character string in the given kanji-kana-mixed character string from the character string storage unit 22. a kana independent word detection unit 25 that detects independent words in kana notation from , a non-kana independent word dictionary storage unit 26 that stores a non-kana independent word dictionary containing independent words whose notation includes non-kana characters, and an independent word determination unit It is activated when the unit 23 determines that an independent word containing a non-kana character in the notation should be detected, and the notation unit and character string storage unit 22 in the non-kana independent word dictionary in the non-kana independent word dictionary storage unit 26 is activated.
Non-kana that detects an independent word whose notation includes a non-kana character from the given Kanji-kana-mixed character string by comparing it with a partial string containing non-kana characters in the given Kanji-Kana-mixed character string. Part of speech, pronunciation, accent information, etc. regarding independent words in kana notation or independent words containing non-kana characters in the notation detected by the independent word detection unit 27 and the kana independent word detection unit 25 or the non-kana independent word detection unit 27 an information output unit 28 that reads and outputs word information from the kana independent word dictionary stored in the kana independent word dictionary storage unit 24 or the non-kana independent word dictionary stored in the non-kana independent word dictionary storage unit 26; ，
-28.
Note that the character string given as the object of independent word detection is a character string containing kanji and kana that is converted into a character code string through an input device such as a keyboard, OCR, magnetic tape device, etc., and the attached word detection is performed by referring to an attached word dictionary. or even partially subjected to independent word detection. Further, for the character string storage section 22, the kana independent word dictionary storage section 24, and the non-kana independent word dictionary storage section 26, for example, an IC memory, a magnetic disk device, a magnetic tape device, etc. are used. Furthermore, the independent word determination unit 23 includes the character string storage unit 2
When the remaining character strings that are not recognized as words in the given kanji-kana mixed character string stored in step 2 are all kana characters, it is determined that an independent word in kana notation should be detected, and When a kana character is included, it is determined that an independent word whose notation includes a non-kana character should be detected. For example, for the character string ``difficult/ii (adjective conjugation adjunctive form)/hon (noun)'' containing kanji and kana, the remaining character string ``difficult'' other than the character string recognized as a word as shown in (noun) '' contains the non-kana character `` difficult'', so it is determined that an independent word that includes the non-kana character in the notation should be detected, and for example, the character string ``Omoshiro/ii (adjective conjugated adjunctive form)/hon ( As for "Noun)", since the remaining character string "Omoshiro" is all kana characters, it is determined that an independent word written in kana should be detected. These determination results are sent to the control section 29.

Here, the structure of the kana independent word dictionary stored in the kana independent word dictionary storage unit 24 and the kana independent word detection unit 25 are known, and are not a feature of the independent word detection method of the present invention. The features of the independent word detection method of the present invention include the configuration of the non-kana independent word dictionary used to detect independent words containing non-kana characters in the notation, and the non-kana independent word detection unit 27.
It is.

Note that the kana independent word dictionary and the non-kana independent word dictionary do not necessarily have to be configured as separate dictionaries, but can also be configured as one independent word dictionary.

Next, in FIG. 4, each record corresponding to an independent word whose notation includes a non-kana character stored in the non-kana independent word dictionary 4 has a notation section 41 and a word information section 42, and the notation section 41 has a notation section 41 and a word information section 42. A main notation part 411 of the part from which the last kana character string is removed from the notation part 41 and a residual notation part 412 of the part which is the last kana character string.
It consists of In addition, the word information section 42 stores part-of-speech information used in grammatical testing or syntactic analysis of detected independent words, pronunciation and accent information used in audio output, etc. It is not necessarily used for word detection.

Next, in FIG. 5, a character string storage section 22, a non-kana independent word dictionary storage section 26, an information output section 28, and a control section 29 correspond to the same reference numerals in FIG. 2, respectively. The non-kana independent word detection unit 27 detects the possible position of the last non-kana character in the independent word whose notation includes a non-kana character for a given kanji-kana mixed character string 220 stored in the character string storage unit 22. A verification start position setting circuit 271 that sets the verification start position 221 as the verification start position 221, and a given kanji-kana mixed character string 220 stored in the character string storage unit 22.
Character string 2 before the matching start position 221 in
22 and the main notation part 411 in the non-kana independent word dictionary stored in the non-kana independent word dictionary storage unit 26. For each independent word searched by the partial search circuit 272,
Verification start position 22 in the given kanji/kana mixed character string 220 stored in the character string storage unit 22
It is comprised of a residual notation part collation circuit 273 that collates the character string 223 after 1 with the residual notation part 412 in the non-kana independent word dictionary 4 stored in the non-kana independent word dictionary storage part 26. Here, the main notation part search circuit 272 and the remaining notation part matching circuit 273 can be realized using a known word dictionary search method and character string matching method, respectively. The position of each independent word searched by the main notation part search circuit 272 in the non-kana independent word dictionary 4 is passed from the main notation part search circuit 272 to the remaining notation part matching circuit 273.
From the residual notation search circuit 273 to the information output unit 28 is a non-kana independent word dictionary 4 of independent words whose notation detected by the non-kana independent word detection unit 27 includes non-kana characters.
The position in is passed. Furthermore, when the verification start position setting circuit 271 sets the verification start position 221,
The collation start position determined as the possible position of the last non-kana character in an independent word whose notation includes a non-kana character is indicated by a triangle in FIGS. 6a and 6b, respectively. That is, as shown in Figure 6a, the position of the non-kana character immediately before the kana character (however, katakana is treated as a non-kana character), and as shown in Figure 6b, the position of the character string that has already been recognized as a word. The position of the immediately preceding non-kana character is the matching start position in the first and second examples. In the same figure, ()
The characters inside represent part-of-speech information regarding words recognized by word detection performed in advance.

Next, in this embodiment, for example, in the case of performing independent word detection of a character string containing kanji and kana, ``suddenly cold/ii (adjective conjugated ending adjunctive form)/rain (noun)/ga (case particle)'', FIG. This will be explained along the flowcharts of FIGS. 3 and 7 using FIGS. 4 and 5.

Step 1 (process 31 in FIG. 3): The character string input unit 21 is activated, and the character string ``suddenly cold/ii (adjective conjugated suffix)/rain (noun )/ga (case particle)'' (part of the words have already been recognized from the rear) is written into the character string storage section 22.

Step 2 (process 32 in FIG. 3): The independent word determination unit 23 is activated, checks the character type of the remaining character string "suddenly cold" that is not recognized as a word, and determines whether the remaining character string contains non-kana characters. Therefore, it is determined that an independent word containing non-kana characters in the notation should be detected, and the determination result B is sent to the control unit 29.

Step 3 (Process 34 in FIG. 3): Upon receiving the determination result B that an independent word containing non-kana characters in its notation should be detected, the non-kana independent word detection unit 27
is activated, and the following steps 3-1 to 3-3 are performed.

Step 3-1 (process 341 in FIG. 7): The matching start position setting circuit 271 is activated, and the non-kana character immediately before the kana character in the remaining character string "suddenly cold" of the given kanji-kana mixed character string 220 is activated. The position (that is, the position of "rei") is determined as the position of the last non-kana character in the independent word whose notation includes a non-kana character, and is set as the matching start position 221.

Step 3-2 (process 342 in FIG. 7): The main notation search circuit 272 is activated and searches for a character string 222 preceding the collation start position 221 (position of "chi") in the given kanji-kana mixed character string 220. The non-kana independent word dictionary 4 is searched for an independent word whose main notation 411 in the non-kana independent word dictionary 4 matches "suddenly cold." Here, there is no independent word that matches the main notation part 411 for "Sudden Rei" and "Zenrei" with one character deleted, and further, there is no independent word that matches the main notation part 411 for "Rei" with one character deleted. Since there are "cold", "cold", and "cold", these independent words are searched (however, when using the non-kana independent word dictionary 4 with the contents shown in Figures 4 to 6), do). Regarding each of the searched independent words “chill”, “cold”, “cold”,
The position in the non-kana independent word dictionary 4 is sent to the residual notation matching circuit 273. At this time, if no matching independent word in the main notation part 411 is retrieved, the search failure occurs in the remaining notation part matching circuit 2.
Sent to 73.

Step 3-3 (process 343 in FIG. 7): The residual notation matching circuit 273 is activated, and for each of the independent words "cold", "cold", and "cold" searched by the main notation searching circuit 272, The character string 223 "Tai..." after the verification start position 221 in the given kanji/kana mixed character string 220 is verified against the remaining notation portion 412 . At this time, character strings corresponding to the length of the kana character strings in each residual notation section 412 are compared. In other words, for the independent word "chi", the residual notation part 412 is empty (space), so the matching is successful, and for the independent word "chit", the "ta" in the residual notation part 412 matches "ta" in the character string, so the matching is successful. Successfully, for the independent word "cold", "yaka" in the residual notation section 412 and "tai" in the character string do not match, resulting in a matching failure. Therefore, the position in the non-kana independent word dictionary 4 is sent to the information output unit 28 for each of the independent words "rei" and "rei" that have been successfully matched. Note that if a search failure is sent from the main notation search circuit 272 instead of the position of the searched independent word in the non-kana independent word dictionary 4, the remaining notation part 412 is not checked and the search fails. The information is sent to the information output section 28.

Step 4 (Process 35 in FIG. 3): The information output unit 28 is activated and performs further analysis (grammatical test, voice output, etc.) regarding the independent words “chi” and “cold” received from the residual notation matching circuit 273. The word information required for this is read from the non-kana independent word dictionary 4 and output.

In this example, the following word information is output.

“Cold” Part of speech: Verb stem Pronunciation: Hiya Accent: Type 2 “Cold” Part of speech: Adjective stem Pronunciation: Tsumeta Accent: Type 0 However, when a search failure is sent from the residual notation matching circuit 273, an independent word is detected. Failure is output.

Furthermore, if we were to perform a grammatical test on the independent words "chi" and "cold" detected through the above steps, the following character strings "i (adjective conjugation suffix) / rain (noun) / ga ( The independent word ``cold (adjective stem)'' that can be connected with ``case particle)'' is selected as the correct analysis result. Then, word detection is further performed for the remaining character string "sudden".

When the independent word determination unit 23 determines that an independent word written in kana should be detected in step 2 and sends the determination result A to the control unit 29, the step
After the process 3' is performed, the flow returns to step 4.

Step 3' (process 33 in FIG. 3): Upon receiving the determination result A that an independent word written in kana should be detected, the kana independent word detection unit 25 is activated, and detects the word written in kana from the given character string containing kanji and kana. Independent words are detected and sent to the information output section 28.

As explained above, according to this embodiment, the starting position of matching a given character string containing kanji and kana with the notation part of the non-kana independent word dictionary is determined between a character string that has already been recognized as a word and a character string that has not yet been recognized as a word. Because it is set to the last non-kana character in an independent word that includes a non-kana character in its notation, rather than the boundary with a character string, the conventional independent word detection method could not obtain the correct independent word. It is clear that the correct independent word can be obtained even if part of the independent word is recognized as an erroneous adjunct, as shown in (a). In other words, in the character string "for example" in FIG. According to the method, not only "example (noun)" but also "for example (adverb)" can be detected, so correct independent words can be obtained.

Additionally, independent words with different matching start positions, such as "example" and "for example," or "rain" and "shower," which previously could not be obtained without repeating multiple searches, sometimes blindly, many times. Also, in this embodiment, since the same matching start position of the last non-kana character in the independent word is set, simultaneous detection is possible and efficiency is high.

Note that in this example, the character string containing kanji and kana that is the target of independent word detection is a character string containing kanji and kana that has been detected as an adjunct word in advance. Even if no detection has been made, it is possible to apply the independent word detection method of the present invention by setting the transition point of the character type from a non-kana character to a kana character as the matching start position. For example, for the character string ``suddenly showering'' that includes kanji and kana, the character ``ran'', which is the transition point from a non-kana character to a kana character,
By applying the independent word detection method of the present invention with "rain" and "rain" as matching starting positions, "sudden (adverb)", "shower (noun)" and "rain It is possible to detect independent words that include non-kana characters in the notation, such as "(noun)" and "desu (five-dan verb stem)." Therefore, whether or not the target character string containing kanji and kana has been subjected to adjunct word detection in advance is not a constraint for realizing the independent word detection method of the present invention.

(Effects of the Invention) As is clear from the above explanation, according to the independent word detection method of the present invention, matching starts from the last non-kana character in an independent word that includes non-kana characters in the notation, so This has the effect that even independent words written in combination with non-kana characters can be detected correctly and efficiently.

[Brief explanation of drawings]

FIGS. 1a and 1b are diagrams showing first and second examples of independent word detection using a conventional independent word detection method, respectively, and FIG. 2 is a block diagram showing an embodiment of the independent word detection method of the present invention. FIG. 3 is a flowchart showing the processing operations of each part in FIG. 2, FIG. 4 is a diagram showing an example of a non-kana independent word dictionary used in the independent word detection method of the present invention, and FIG. FIGS. 6a and 6b are a block diagram showing an example of the configuration of a non-kana independent word detection section, and FIGS. Figure 7 shows processing 34 in Figure 3.
This is a flowchart showing the details. In the figure, 21... character string input section, 22...
Character string storage unit, 220...Character string containing kanji and kana, 221...Verification start position, 222...Character string before the verification start position, 223...Character string after the verification start position, 23...Independent word Judgment part, 2
4...Kana independent word dictionary storage unit, 25...Kana independent word detection unit, 26...Non-kana independent word dictionary storage unit, 2
7...Non-kana independent word detection unit, 271...Verification start position setting circuit, 272...Main notation part search circuit,
273... Residual notation section collation circuit, 28... Information output section, 29... Control section, 31, 32, 33, 3
4,341,342,343,35...processing, 4
...Non-kana independent word dictionary, 41...Notation section, 411
...Main notation part, 412...Remaining notation part, 42...
...Word Information Department.

Claims (1)

[Claims] 1. In an independent word detection method that detects an independent word from a given character string containing kanji and kana by comparing a given character string containing kanji and kana with the notation of each independent word in an independent word dictionary, For the notation part of an independent word containing non-kana characters in the notation stored in the dictionary, the main notation part of the part after removing the last kana character string from the notation part and the remaining notation of the part that is the last kana character string. When detecting an independent word that includes a non-kana character in the notation from the given character string containing kanji and kana characters, it is necessary to detect the existence of the last non-kana character in the independent word that includes a non-kana character in the notation. a verification start position setting means for setting a position to be obtained as a verification start position; and a verification start position setting means for setting the obtained position as a verification start position; A main notation part search means for searching from a dictionary, and a character string after the matching start position in the given kanji-kana mixed character string and the remaining notation part for each independent word searched by the main notation part search means. residual notation part matching means for matching the notation, and detecting an independent word that includes a non-kana character in the notation by starting matching from the last non-kana character in the independent word that includes the non-kana character in the notation. An independent word detection method characterized by: