JP2684138B2 - Japanese morphological analysis system and headline extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to Japanese morphological analysis processing performed as part of processing such as a parser, and is particularly characterized by a clipping method for clipping a partial character string corresponding to a morpheme to be identified from an input sentence. The present invention relates to a Japanese morphological analysis system.

[0002]

2. Description of the Related Art A morphological analysis system used for a Japanese parser or the like generally comprises a dictionary storing various information about morphemes and a dictionary search unit for searching the dictionary. Then, referring to the dictionary information etc. stored in the above dictionary, the input Japanese sentence is converted into an appropriate morpheme (word).
Then, the morphemes to which predetermined information is added are passed to the syntactic analysis system or the like.

Here, the above-mentioned conventional dictionary search unit provided in the morpheme analysis system generally employs a method of searching by the first character of the morpheme heading. This dictionary search method has the advantage that the number of searches is small and the processing algorithm can be relatively simple.

[0004]

However, the conventional morphological analysis system based on the above-mentioned dictionary search method is not always efficient in analysis. This is because in large dictionaries with many registered words, words with the same first letter inevitably increase (because the type of kana is small compared to kanji etc., this tendency is particularly noticeable in dictionaries with many kana headings). When the method of searching by the first character of the headline is adopted, this is because the morpheme search information output at the same time becomes huge. Therefore, there have been the following problems.

1) Problem of lack of memory for output information As the amount of information retrieved simultaneously increases, the main memory area becomes insufficient, which may affect the analysis speed.

2) Useless Matching of Output Information When the amount of information retrieved at one time increases, the matching process inevitably becomes enormous. Therefore, if the amount of useless search information increases, the amount of matching useless increases, which may reduce the analysis efficiency.

3) Retrieval Problem of Meaningless Information in Morphological Analysis Adopting the Longest Matching Method Using this retrieval method, for example, the input sentence becomes "ABCD".
If the correct morpheme to be identified at the beginning is "ABCD" in the case of "EFG", not only "ABCD" but also
If it starts with "A", it also searches for morphemes shorter than "ABCD", such as "ABC" and "AB". However, in a morphological analysis system that employs the longest matching method, which is a general morpheme extraction method, and finds only one set of optimal morpheme division patterns, these do not need to be output as the results of morphological analysis, so it is meaningless. Yes, it was a waste of search.

4) Problem of impossible information retrieval When the input sentence is "ABCDEFG", "ABCDEF"
Even if you search for an unnecessarily long morpheme such as "GHIJ", or when the input sentence is "ABCD, FG" and there is a delimiter character after "D", "ABCDFG"
There is a problem that a morpheme longer than necessary such as "D" is searched.

5) Problem of useless search time for output information Since all applicable morphemes are searched in one search, the number of searches itself is small, but it is useless due to the useless searches in 2) to 4) above. Search time was also required.

By the way, since the above-mentioned problems are all caused by performing the dictionary search with the first character of the morpheme heading, it can be solved by adopting the dictionary search method by specifying the morpheme heading in full spelling. it can.

However, it is usually difficult to efficiently determine the range of a partial character string (hereinafter, this character string is referred to as a “heading-corresponding character string”) in an input Japanese sentence corresponding to one morpheme heading. Therefore, in morphological analysis that uses the longest match method and employs a search method that performs a full-spelling dictionary search, as a result, the longest range of heading-corresponding character strings in Japanese character strings is preferentially prioritized by trial and error. It will be cut out into, and the dictionary will be searched each time. Therefore, the number of dictionary searches will increase at an accelerating rate as the length of the sentence becomes longer.

[0012] Regardless of the number of dictionary searches, one dictionary search requires a physically constant time.
When the dictionary search method with full spelling is adopted, there is a problem that a long analysis time may be required to analyze a long sentence. Therefore, after all, it has been common to adopt a dictionary search method using the first character of the headline and then improve the analysis efficiency by devising the matching procedure and the like.

By the way, the above-mentioned problem in the dictionary search system in which a morpheme heading is specified by full spelling is due to the inability to efficiently determine the range of heading-corresponding character strings.

Therefore, in order to solve the above-mentioned problem, a method is adopted in which a dictionary is searched by designating a morpheme heading with full spelling, and a range of heading corresponding character strings is efficiently determined to reduce the number of dictionary searches. It is an issue to realize.

The Japanese morphological analysis system of the present invention is predicated on the case where the input sentence is a Japanese sentence, and the length of the headline-utilizing character string that can be obtained when the headings of the morphemes registered in the dictionary are used at the end. It is an object of the present invention to solve the above problem by efficiently cutting out a heading-corresponding character string using information regarding the size and the like.

[0016]

In order to achieve the above object, the Japanese morphological analysis system of the present invention is provided with a dictionary capable of searching morpheme information by full spelling of morpheme headings, and a character string of an input Japanese sentence. About the beginning of the sentence, it is cut out in order from the partial character string in the predetermined range and the ending is transformed,
In the Japanese morphological analysis system, which identifies the morpheme corresponding to the extracted partial character string while searching the dictionary with the heading of the format registered in the dictionary created by the ending modification, registers in the dictionary For all possible headline utilization character strings of all the morphemes that have been stored, information about the length of the headline utilization character string is stored separately for at least each group in which the first character of the headline utilization character string is the same. When attempting to cut out a partial character string from the long presence determination table and the character string of the Japanese sentence, refer to the information in the heading length presence determination table corresponding to at least the first character of the partial character string, A form that can take a headline utilization character string in which at least the first character of the partial character string matches and the lengths of the partial character strings match Element is characterized in that it comprises a heading cutout portion going cut out the partial string only when it is determined to be registered in the dictionary.

In the above invention, the headline length existence determination table divides at least the headline utilization character strings into groups according to the difference in the first character for all the headline utilization character strings that can be taken by all the morphemes registered in the dictionary, Stores heading length existence information for determining whether or not a heading utilization character string of a specific length belongs to the group, which is created by aggregating the lengths of heading utilization character strings for each group. In addition, the maximum length of the headline utilization character string belonging to the group is stored for each group.

Further, a character string of an input Japanese sentence is sequentially cut out from a partial character string in a predetermined range on the sentence head side and subjected to word ending modification, and a full spelling heading in a format registered in a dictionary created by the word ending modification. Used in a Japanese morphological analysis system that identifies the morphemes corresponding to the extracted partial character strings while searching the dictionary with
In the headline cutout method used when cutting out a partial character string in the range corresponding to one morpheme heading from the character string of the Japanese sentence, when a partial character string is to be cut out from the character string of the Japanese sentence, From the heading length existence determination table in which information on the lengths of all possible heading utilization character strings registered in all the above morphemes stored in the dictionary is distinguished by at least the first character of the heading utilization character string and stored, At least the first character of the partial character string to be cut out coincides with the step of extracting the information corresponding to the partial character string to be output and the information extracted from the heading length existence determination table, and A step for determining whether or not a morpheme that can take a headline-utilizing character string whose lengths of the partial character strings match is registered in the dictionary. Then, by the above determination, the partial character string is changed only when it is determined that a morpheme that can take a headline utilization character string that may match the partial character string to be cut out is registered in the dictionary. And a step of cutting out.

[0019]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a morphological analysis system according to an embodiment of the present invention.

As shown in the figure, the morpheme analysis system of this embodiment is provided with a dictionary 10 in which morphemes are registered and stored together with various information about the morphemes, and can be searched by full spelling of morpheme headings. A headline cutout unit 2 that cuts out a character string in a predetermined range on the assumption that it is a character string, and a headline in a format in which the headline corresponding character string cut out by the headline cutout unit 2 is changed and registered in the dictionary 10 (hereinafter, this headline will be referred to as (It is called "word ending utilization headline")
And a dictionary search unit 4 that searches the dictionary 10 for the ending utilization headings created by the ending utilization unit 3, and a connection determination unit 5 that determines whether or not adjacent morphemes can be connected. The headline cutout unit 2, the word ending utilization unit 3, the dictionary search unit 4, and the morpheme identification control unit 1 for controlling the concatenation determination unit 5 are provided. Also, the headline cutout unit 2
Head length existence determination table 21 referred to in the processing of
And the ending ending table 3 referred to by the ending ending part 3
1 is provided.

Morphological information is stored in the dictionary 10, and the heading of each morpheme is registered as a final heading when the ending is used and as it is when the ending is not used. The morpheme information stored in the dictionary can be searched by specifying the full spelling of the morpheme heading.

The morpheme identification control unit 1 sequentially pays attention to the characters on the sentence head side of the input sentence, and sequentially identifies the morphemes on the sentence head side by a series of processes from the headline cutout unit 2 to the concatenation determination unit 5. The morpheme identification control unit 1 includes a backtrack processing unit, an unknown word processing unit, and a morpheme determining unit (all are not shown).

Here, when the immediately preceding morpheme is identified and in the case of the first processing for the input sentence, the morpheme identification control unit 1 newly pays attention to the next character in the determined range, and the morpheme extraction unit 2 The character string after the character under attention and the position of the character under attention are passed. Then, when the morpheme that includes the character of interest cannot be identified, if no suitable morpheme can be retrieved by the dictionary retrieval unit 4, or no morpheme can be concatenated by the concatenation determination unit 5. In this case, the character string after the character under attention and the position of the character under attention are passed again to the headline cutout unit 2.

On the other hand, if the headline cutout unit 2 cannot cut out, the backtrack processing is started. If the backtrack is successful, the next character in the range identified by the backtrack processing is focused on. If backtracking fails, unknown word processing is started, and then the next character in the range identified by unknown word processing is focused.

The headline cutout unit 2 is passed from the morpheme identification control unit 1 the character string after the character of interest in the input sentence and the position of the character of interest in the input sentence. When the character string and the position of the focused character are passed, the headline cutout unit 2 assumes that the focused character is the first character of the headline-corresponding character string, and determines partial characters within a predetermined range that are considered as the headline-corresponding character string. The string is cut out from the character string (character string after the character under attention) passed from the morpheme identification control unit 1.

Here, the headline cutout unit 2 cuts out a character string by using a generally used "longest matching method" (priority is given to the longest morpheme), but
In order to reduce unnecessary cutting, further, in the present embodiment, all the character strings that can be obtained when the headings of all the morphemes registered in the dictionary are used as endings (hereinafter, this character string will be referred to as “heading utilization character A range corresponding to the headline of one morpheme is efficiently determined by using information obtained by aggregating information about the length of a “column”) by a predetermined procedure, and the character string in the range is cut out as a headline-corresponding character string.

In order to realize such means, the headline cutout unit 2 has a headline length existence determination table 21,
Newly noticed characters, that is, for the heading corresponding character string to be cut out, at least the first character of the heading corresponding character string and the length of the heading corresponding character string are clues,
The corresponding information in the index length existence determination table 21 is referred to.

The headline length existence determination table 21 classifies all the headline utilization character strings that can be taken by all the morphemes registered in the dictionary into groups according to at least the difference in the head character, and the groups thereof. The headline length presence information created by aggregating the lengths of the headline utilization character strings belonging to the group is stored for each. Further, in each group, information on the maximum length of the headline utilization character string belonging to each group is also stored. here,
The headline length existence information stored for each group is information that can be used to determine how many characters of the headline utilization character string belong to each group.

Therefore, when the heading-corresponding character string is to be cut out, referring to the heading length existence information of the group corresponding to the first character of the heading-corresponding character string, the length of the heading-corresponding character string to be cut out is referred to. It is possible to determine whether the headline utilization character string equal to the length belongs to the group. This is because whether or not a morpheme that can take a headline utilization character string in which at least the first character of the headline-corresponding character string to be cut out matches and the headline-corresponding character string has the same length is registered in the dictionary. It means that it can be determined. The heading length existence determination table 21 is sufficiently small and can be provided in the main memory.

Here, the heading length existence determination table 21
Is created in advance as follows and is loaded on the main memory before a series of morphological analysis processing. First, for all morphemes registered in the dictionary, the heading with inflection is the length of all the inflection-use character strings obtained by inflectioning the inflection of the heading (end-form heading) registered in the dictionary. For a heading that has no endings, the length of the character string of the heading itself is extracted.

For example, if a morpheme beginning with "report" is registered only in the dictionary 10 as "reward", "reward", and "report", the information on the headline utilization character string corresponding to "report" Is as shown in Table 1.

[0032]

[Table 1]

Next, at least the head character of the headline utilization character string is divided into groups, and the lengths of the headline utilization character strings existing in the respective groups and the maximum lengths thereof are obtained and stored in the headline length existence determination table 21. In Table 1, "report"
The length of the headline inflection character string corresponding to those beginning with is 2
Since there are only letters, three letters, and four letters,
The information of the heading length existence determination table corresponding to “report” is
It becomes like Table 2.

[0034]

[Table 2]

Here, the heading length existence information in Table 2 is information represented by 8 bits, and each bit is a heading utilization that belongs to that group (in this case, a group having "report" as the first character). Corresponds to the number of characters in the string. The dictionary 10 has a morpheme that can take a headline utilization character string of a predetermined length belonging to the group, and has 1 if it does not exist.

In Table 2, for example, the length of the headline utilization character string corresponding to "report" includes only two characters, three characters, and four characters. Therefore, the second bit from the left, the third bit, the fourth bit, and the fourth bit.
Only the 1st bit is 1, and the others are 0.

When the heading length existence information is set to have an 8-bit area, if there is a heading utilization character string of 9 characters or more, it cannot be expressed by the heading length existence information. In such a case, for example, the maximum length is 10, and virtually all 9th to 10th bits are 1
Will be interpreted as follows.

For example, the heading starting with "O" is the dictionary 1
If only "open fund" (9 characters) is registered in 0, the information of the heading length existence determination table 21 corresponding to "o" is as shown in Table 3.

[0039]

[Table 3]

The size of the area of the heading length existence information in the heading length existence determination table 21 can be freely set, and if the heading utilization character string that can be taken by the morphemes registered in the dictionary is often 9 characters or more. Is, for example, if the heading length existence information is 12 bits or 16 bits,
It is better to have a larger area, and if the heading length existence information is set to have a 16-bit area, a heading utilization character string of 17 characters or more will be treated in the same way.

It should be noted that, as in the case of the verb "kuru", the head character of the heading-utilized character string when the word is utilized is different from the head character of the morpheme heading registered in the dictionary 10 (end-type heading, etc.). In some cases. Temporarily "come" to the dictionary 10
If only registered, the length of the headline utilization character string corresponding to "Kuru" and the headline length existence information are as shown in Tables 4 and 5.

[0042]

[Table 4]

[0043]

[Table 5]

In Table 5, if a dictionary other than "Kuru" is registered in the dictionary 10, for example, "Kodomo" (three characters), from the left of the heading length existence information corresponding to "Ko" in Table 5. The place of the 3rd bit becomes 1. FIG. 2 shows an example of the headline length existence determination table 21 in the case where information is divided into groups by the first character, which is created as described above.

It is to be noted that, instead of grouping by the first one character of the headline utilization character string as described above, it is better to divide the headline utilization character string into smaller groups, for example, by the first two characters of the headline utilization character string. Since the number of characters becomes smaller, it is possible to more strictly determine whether or not the headline utilization character string of the corresponding length belongs to the group. Further, it is possible to make a stricter determination by grouping with the first three characters rather than the first two characters, and finally, it is determined whether or not there is a corresponding one by putting all the headline utilization character strings in the main memory. It will be the most rigorous decision. However, if this is done, the memory consumed for that information may become enormous and it is difficult to implement.

Therefore, in this embodiment, in order to make the heading length existence determination table 21 placed in the main memory small and to make more strict determination possible, in principle, grouping is performed by the first character. Only when the first character is a hiragana, the first two characters are grouped together to some extent and grouped by these combinations.

Specifically, if the first character at the beginning is hiragana and the second character at the beginning is hiragana, all the characters in the first character at the beginning are classified, and if the second character at the beginning is hiragana, "A line", " If the first second character is a character other than hiragana (for example, Kanji), the heading length exists for each combination with the grouping such as “Kanji”, “Katakana”, etc. Information is set.

Generally, this is because (1) the average length of a dictionary heading (or heading-use character string) is about 2 characters, and therefore it can be determined quite strictly by referring to the first two characters. (2) Hiragana heading , Being registered more in the dictionary than the headings of other types of characters (3) It is effective to distinguish when the type of characters changes (4) The type of hiragana is extremely different from kanji etc. It is based on the fact that even if the information consisting of the combination of the first two characters is set, the amount of the information itself is small and it does not affect the memory used because it is small.

The heading length existence determination table 21 is
As described above, the unit of information setting can be freely changed, and when it is not necessary to consider the memory constraint, the units may be divided into smaller groups. Further, it is sufficiently effective to provide the main memory with the heading length existence determination table 21 grouped by only the first character.

FIG. 3 shows an example of the heading length existence determination table 21 in which the group is subdivided according to the type of the second head character only when the first head character of the heading corresponds to hiragana.

The headline cutout unit 2 refers to the headline length existence determination table 21 created as described above, and the headline length existence information corresponding to the newly noticed character (including the next character in some cases). Take out. Then, within the range to be cut out (the range that does not exceed the end of sentence etc.),
In accordance with the "longest match method," the longest string is sequentially cut out as a heading-compatible character string.

For example, if the input sentence is “must report” and the user is paying attention to “report” and coming to the headline segmentation section 2, normally, “must report” “must report” "I must report": It will be cut out like "not report" and "report", but in the present embodiment, the headline length existence determination table 2 for "report" is assumed.
If the information of No. 1 is as shown in Table 2 above, it can be seen that by referring to this, the headline utilization character string starting with "report" has only 4 characters or less, so at first "Shin" is cut out, and then "report" is cut out.

When there is no headline utilization character string having a specific character as the first character, that is, when a morpheme that can take a headline utilization character string having the specific character as the first character is not registered in the dictionary 10. Indicates that the maximum length of the index length existence determination table 21 corresponding to the specific character is 0. For example, if a morpheme that can take a headline inflection character string starting with "ru" is not registered in the dictionary 10, the information of the headline length existence determination table 21 corresponding to "ru" is as shown in Table 6.

[0054]

[Table 6]

Therefore, when trying to cut out a heading-corresponding character string starting with a specific character, if the maximum length in the information of the heading length existence determination table 21 corresponding to the specific character is 0, the specific character It is possible to immediately determine that a morpheme that can take a headline inflection character string beginning with is not registered in the dictionary 10. In this case, there is an effect that the headline-corresponding character string starting with a specific character cannot be cut out without cutting out (searching the dictionary) even once, and an unregistered word can be efficiently determined.

The headline slicing unit 2 stores information about the range of the sliced character string, that is, the information such as the character under consideration in the input sentence, the position under consideration, the length of the extracted character string, etc. . This information is used to distinguish between cutting out a shorter range heading-compatible character string starting with the character at the same focus position or cutting out the maximum range heading-compatible character string starting at the character at the newly focused position. To do.

When the headline cutout unit 2 can cut out a character string corresponding to one or more characters from the character string passed from the morpheme identification control unit 1, the cutout corresponding character string is used by the ending part utilization unit 3. Pass to. When it is not possible to cut out the heading-corresponding character string of one or more characters, it is determined that cutting out is not possible, and the process returns to the morpheme identification control unit 1.

The word ending part 3 is used by the headline cutout part 2.
Receives a heading-compatible character string from. The word ending utilization unit 3 has a word ending utilization table 31 as shown in FIG. 4, and refers to the word ending utilization table 31 for the hiragana part of 1 to 4 characters at the end of the heading corresponding character string, and is estimated by this. An ending ending heading (heading in the format registered in the dictionary 10) is created. Of course, a heading that does not utilize the ending is also regarded as a single ending using the heading. FIG. 5 shows an example of creating an ending inflection heading.

After the above processing, the ending ending section 3 collects the obtained arbitrary ending ending headings as one list and sends it to the dictionary searching section 4.

The dictionary search unit 4 searches the dictionary 10 for all the passed inflection use headings, and retrieves the morpheme information of the morphemes corresponding to each inflection use heading. And
When an appropriate morpheme is retrieved, the list of retrieved morpheme information is sent to the concatenation determination unit 5. On the other hand, when an appropriate morpheme cannot be retrieved, the process proceeds to the processing of the headline cutout unit 2 via the morpheme identification control unit 1.

The concatenation determination section 5 determines whether or not adjacent morphemes can be concatenated based on the information passed from the dictionary retrieval section 4, and identifies an appropriate morpheme. Then, when an appropriate morpheme is identified, the information is passed to the morpheme identification control unit 1. When an appropriate morpheme cannot be identified, the process proceeds to the processing of the headline cutout unit 2 via the morpheme identification control unit 1.

Next, the operation of the present embodiment will be described in detail by showing a concrete example of processing of an example sentence. FIG. 6 and FIG. 7 show a processing example in the case where the sentence “she is dominant in the present situation” is subjected to the morphological analysis processing according to the present embodiment. In the present analysis example, when the first character is a hiragana character, the grouping is performed by combining the above two characters up to the first two characters, and when the first character is not a hiragana character, the heading character is grouped by the first character. An example using the table 21 will be shown.

First, under the control of the morpheme identification control unit 1, the headline cutout unit 2 causes the headline length existence determination table 21.
Referring to FIG. 3, since the headline utilization character string starting with the first character “now” has a maximum of three characters, the headline corresponding character string “current state” is cut out. Then, with respect to this headline-corresponding character string, the processing of the word ending utilizing unit 3, the dictionary searching unit 4, and the concatenation judging unit 5 is performed, but the morpheme cannot be identified,
The process returns to the processing of the headline cutout unit 2.

Therefore, the headline cutout section 2 is "current state".
One character at the end of is deleted, and "Imamono" is cut out as a new heading-compatible character string. After all, even for this headline-corresponding character string, the morpheme could not be identified in the subsequent processing,
The process returns to the processing of the headline cutout unit 2. After repeating the same processing, the morpheme "now" is identified. Then, paying attention to the next character “NO”, the process moves to the extraction of the heading corresponding character string.

Since the first character of the succeeding character string is the hiragana "no", the heading cut-out part 2 looks at the second character "shape" and refers to the heading length existence determination table 21 for the character string "no". Since the headline utilization character string starting with "no + kanji" has a maximum of 0 characters, the headline corresponding character string "no" is not cut out and there is a headline utilization character string corresponding to "no" (one character) Cut out the character string "no". Then, the heading-corresponding character string is processed by the word ending utilizing unit 3, the dictionary searching unit 4, and the concatenation judging unit 5 to identify the morpheme "no" or "da". Then, the process proceeds to the cutting out of the next heading corresponding character string.

The headline slicing unit 2 refers to the headline length existence determination table 21 for the first character "character" of the following character string, and since the headline utilization character string starting with "character" has a maximum of three characters, the headline corresponding character string Cut out "by situation".
In this case, the morpheme cannot be identified in the subsequent processing for this headline-corresponding character string, and the headline cutout unit 2 is again used.
Return to the processing of.

Therefore, the headline cutout unit 2 deletes the last character of "in the situation" and cuts out the "status" as a new headline-corresponding character string. Then, the morpheme “situation” is identified by the subsequent processing. Then, the process proceeds to the cutting out of the next heading corresponding character string.

The headline slicing section 2 refers to the headline length existence determination table 21 for the character string "a" by looking up to the second character "ha" since the first character of the following character string is a hiragana "de", Since the headline utilization character string starting with "+ is a line" has a maximum of 0 characters, the headline-corresponding character string "de" is cut out without cutting out the headline-corresponding character string "de". Then, the headword corresponding character string is processed by the word ending part 3, the dictionary searching part 4, and the concatenation judging part 5 to identify the morpheme “de” “de” or “da” (however, “de”, "Da" is later rejected as not connectable). Then, the process proceeds to the cutting out of the next heading corresponding character string.

The headline cutout unit 2, the word ending utilization unit 3, the dictionary search unit 4, and the concatenation determination unit 5 repeat the same processing,
The morphemes "ha", "she", "ga", and "predominant" are identified.

In the process of the above processing, the headline cutout unit 2 cuts out 12 types of headline-corresponding character strings as shown in FIG. 6 and FIG.

Next, FIGS. 8 to 10 show the heading length existence determination table 21 (FIG. 2) in which the example sentence “she is dominant in the present situation” similar to the above analysis example is grouped only by the first character. The example used is shown below.

In this analysis example, the heading corresponding character string in the range where the first character is “no” is cut out, the heading corresponding character string is cut out in the range where the first character is “de”, and the first character is “ha”. In cutting out the heading corresponding character string in the range and cutting out the heading corresponding character string in the range in which the first character is “ga”, the heading length existence determination table 21 is referred to only by the first character as shown in FIG. And cut out more character strings corresponding to the headings than in the analysis example of FIG.
Do a dictionary search.

In cutting out the heading corresponding character string in the range where the first character is "no", the heading cutout unit 2
Since the headline utilization character string starting with "no" has a maximum of 5 characters, the headline corresponding character string "in the situation of" is cut out. After all, with respect to this headline corresponding character string, the morpheme cannot be identified in the subsequent process, and the process returns to the process of the headline cutout unit 2. Then, based on the longest match method, the same processing is sequentially repeated for the character strings "in the situation", "the situation", "state", and "no", and then the morpheme "no" or "da" is identified. Then, the process proceeds to the cutting out of the next heading corresponding character string.

In cutting out the heading corresponding character string in the range in which the first character is "de", the heading cutout unit 2
Since the headline utilization character string starting with “de” has a maximum of 5 characters, the headline-corresponding character string “wa she” is cut out. After all, with respect to this headline corresponding character string, the morpheme cannot be identified in the subsequent process, and the process returns to the process of the headline cutout unit 2. Then, based on the longest match method, the string "was him"
The same processing is sequentially repeated for “de” and “de”.
Here, as shown in the heading length existence determination table 21 of FIG. 2, since the heading utilization character string starting with “de” does not have four characters, “then she” is not cut out.

After the above processing, the morpheme "de,""de," or "da" is identified (however, "de,""da" are later rejected as non-connectable). Then, the process proceeds to the cutting out of the next heading corresponding character string.

Similarly, in cutting out the heading-corresponding character string in the range in which the first character is "ha", "ha she is excellent" is cut out, and the heading-in correspondence character string in the range in which the first character is "ga" is cut out. In, first cut out "is superior". Then, the morphemes are identified by sequentially performing processing based on the longest match method. Note that the maximum number of character strings used for headings starting with "ga" is 5 characters, but since there are only 4 characters by the end of the sentence, "ga is dominant" is cut out first.

In the process of the above processing, the headline cutout unit 2 cuts out 26 types of headline-corresponding character strings, as shown in FIGS. To do.

Next, FIGS. 11 to 16 show examples in which the same sentence as in the above-mentioned analysis example, “she is dominant in the present situation”, is analyzed without using the heading length existence determination table 21. In this case, the headline cutout unit 2 is 5 as shown in the figure.
Seven types of heading-corresponding character strings are cut out, and a dictionary search is performed 92 times with the endings-use heading corresponding to them.

Comparing this with the two analysis examples of the present embodiment described above, in the analysis examples of FIGS. 6 and 7, 45 cuts are made by cutting out the heading corresponding character string and 68 searches are made by the dictionary search. It can be omitted. Also, in the analysis examples of FIGS. 8 to 10 in which the analysis efficiency is lower than this, 31 times of extraction of the headline-corresponding character string and 48 times of dictionary search are performed.
This means that the search can be omitted twice, and the number of times the headline-corresponding character string is cut out and the number of times the dictionary is searched are halved as compared with the case where the index length existence determination table 21 is not used.

The example sentence used in this analysis example is relatively short, but in general, since a longer sentence is analyzed,
The reduction effect in the number of cuts and the number of dictionary searches is greater.

[0081]

As described above, the Japanese morphological analysis system and heading cutout method of the present invention are provided with the heading length existence determination table 21, and the range of the heading corresponding character string to be cut out is determined with reference to this. Therefore, the headline-corresponding character string that is not identified as a morpheme is not wastefully cut out, the range of the headline-corresponding character string is efficiently determined, and the number of dictionary searches can be significantly reduced. Therefore, there is an effect that it is possible to adopt a method of searching a dictionary by designating a morpheme heading with full spelling.

Further, by referring to the heading length existence determination table 21, it is possible to immediately detect the number of characters that do not exist as a heading utilization character string, and a morpheme that can take a heading utilization character string starting with a specific character is a dictionary. There is an effect that it is possible to immediately determine that it is not registered in.

Further, according to the present invention, in order to reduce the waste itself of searching the dictionary by referring to the heading length existence determination table 21, even if it is assumed that all the dictionary information can be installed in the main memory, the dictionary can be searched. Considering that takes a certain time, there is an effect that the analysis efficiency is still high.

[Brief description of the drawings]

FIG. 1 is a block diagram showing the configuration of a morphological analysis system according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of a heading length existence determination table 21 of FIG.

FIG. 3 is a diagram showing an example of a heading length existence determination table 21 of FIG.

FIG. 4 is a diagram showing a word inflection table in FIG. 1.

FIG. 5 is a diagram showing an example of creating a ending inflection head by the ending inflection unit in FIG. 1;

6 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

FIG. 7 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

8 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

9 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

10 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

11 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

12 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

13 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

14 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

15 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

16 is a diagram showing an example of analysis processing by the morphological analysis system of FIG.

[Description of sign]

 DESCRIPTION OF SYMBOLS 1 Morphological identification control unit 2 Heading cutout unit 3 Word ending utilization unit 4 Dictionary search unit 5 Concatenation determination unit 10 Dictionary 21 Heading length existence determination table 31 Word ending utilization table

Claims (3)

(57) [Claims] 1. A morpheme information dictionary is provided which can be searched by full spelling of a morpheme heading, and a character string of an input Japanese sentence is sequentially cut out from a partial character string in a predetermined range on the sentence head side and the word is transformed, and the word is transformed. In the Japanese morphological analysis system that identifies the morpheme corresponding to the extracted partial character string while searching the dictionary with the heading of the format registered in the dictionary created by Heading length existence determination in which information about the length of the heading utilization character string is stored for all the heading utilization character strings that can be taken by all morphemes, at least for each group in which the first character of the heading utilization character string is the same When attempting to cut out a partial character string from the table and the above Japanese sentence character string, at least the beginning of the partial character string Using the information in the heading length existence judging table corresponding to the character,
Only when it is determined that a morpheme that can take a headline-utilizing character string in which at least the first character of the partial character string matches and the lengths of the partial character strings match is registered in the dictionary. A Japanese morphological analysis system characterized by comprising a headline extraction unit for extracting a column. 2. The heading length existence determination table divides at least the heading utilization character strings into groups according to the difference in the first character for all the heading utilization character strings that can be taken by all the morphemes registered in the dictionary. For each item, the headline utilization character string length information is stored, which is created by aggregating the lengths of the headline utilization character strings and used for determining whether or not the group has a specific length of the headline utilization character string. The Japanese morphological analysis system according to claim 1, wherein, for each group, the maximum length of a headline utilization character string belonging to the group is stored. 3. A full-spelling headline in a format registered in a dictionary created by cutting out a character string of an input Japanese sentence in order starting from a partial character string in a predetermined range on the sentence head side and deforming the word end. It is used in a Japanese morphological analysis system that identifies the morphemes corresponding to the cut out partial character strings while searching the dictionary with, and the part of the range corresponding to the heading of one morpheme from the character strings of the Japanese sentence. In the headline cutout method used when cutting out a character string, when trying to cut out a partial character string from the character string of the above Japanese sentence, all the headline utilization character strings that can be taken by all the morphemes registered in the above dictionary Information is extracted from the heading length existence determination table in which information on the length of the At least the first character of the partial character string to be cut out coincides with the step of extracting the information corresponding to the partial character string to be extracted and the information extracted from the heading length existence determination table, and the partial character A step of determining whether or not a morpheme that can take a headline-utilizing character string having the same column length is registered in the dictionary, and the determination can match the partial character string to be cut out. And a step of cutting out the partial character string only when it is determined that a morpheme capable of taking a useful headline-use character string is registered in the dictionary.