JP2897942B2 - Japanese morphological analysis system and morphological analysis method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a morphological analysis system and a morphological analysis method for performing a morphological analysis process performed as part of a process such as a machine translation system or a parser. The present invention relates to a Japanese morphological analysis system and a morphological analysis method that are characterized by the time of determining whether or not connection between two adjacent morphemes is possible and a determination unit.

[0002]

2. Description of the Related Art In a machine translation system or a parser, it is indispensable to first divide an input sentence into morphemes (words) and perform a morphological analysis process of adding syntax information and semantic information to each morpheme. Normally, a machine translation system that translates Japanese into another language or a morphological analysis system that is used for parsing Japanese sentences, etc., determines an optimal division pattern of an input Japanese sentence while referring to dictionary information. Then, the morpheme is divided into morphemes, syntax information and semantic information are added to each morpheme, and the analyzed morpheme information is passed to a syntax analysis system or the like.

Conventionally, such a morphological analysis system has
A dictionary that stores syntax information and semantic information about Japanese morphemes, a dictionary search unit that searches the dictionary, a heading cutout unit that cuts out a character string from an input Japanese sentence,
A concatenation determining unit that determines the possibility of adjacency between morphemes corresponding to the cut-out character string based on dictionary information obtained by searching the dictionary and determines an optimal division pattern. The dictionary search unit generally employs, as a dictionary search method, a method of searching for a morpheme using the first character of a character string to be searched. In addition, the dictionary often adopts a method of registering a morpheme registration method by separating a stem of a morpheme and an inflected ending. In addition, the connection determining unit prepares a special rule or table according to the dictionary registration method to make the determination.

[0004]

However, the conventional morphological analysis system described above is not always stable and efficient, and cannot be said to be easy to use for the user. This is mainly due to problems in the dictionary search method of the dictionary search unit, the connection determination information of the connection determination unit, and the determination method as described below.

[0005] 1) Problems in the dictionary search system [0005] The above-described conventional system for searching with the first character of a morpheme by the dictionary search unit has the following disadvantages. First, in a practical dictionary having a large number of dictionary registration words, the number of words having the same first character is inevitably increased. Therefore, a single search for the designation of the first character may result in an enormous number of words (morphemes) to be output. In this case, the search may take a long time. For example, the input sentence is "ABCDEFG"
And the correct morpheme is “ABCD”. At this time, in this method, as long as the word starts with "A", even if the word is "AFG" or "AF", the search is performed. For this reason, not only wasted searching, but also matching was wasted. Further, as the search information increases, there is a possibility that the main storage area used in the system becomes insufficient or the analysis speed becomes slow.

Second, in a morphological analysis system that employs the longest matching method, which is a general character string extraction method, morphemes shorter than the extracted character string are also searched, so that there is a lot of search waste. In the example above,
If "A", "AB", and "ABC" are registered, they are searched.

Third, in a morphological analysis system employing the longest match method, a morpheme longer than necessary is searched for before a correct character string is cut out. In the above example, "ABCDFGD"
Search for a heading longer than "ABCD".
In particular, in this case, if the input sentence is "ABCD, FG" and there is a delimiter or the end of a sentence (punctuation, etc.) next to "D", similarly, an unnecessarily long morpheme such as "ABCDFGD" is used. There was a problem that would be searched.

[0010] Fourthly, in this method, since morphemes corresponding to character strings are searched for and then matched, it seems to be efficient at first glance. However, in a practical dictionary having a large number of dictionary registration words, the morphemes to be searched are determined. Since the number increases, extra matching processing becomes enormous, and analysis efficiency may be reduced.

Further, the above-mentioned disadvantages are particularly remarkable in a dictionary having many hiragana headings or a dictionary having a large number of registered words.

Since the above-mentioned drawback is caused by performing a search using the first character of a morpheme, all of the above drawbacks can be solved by adopting a method of performing search by designating a morpheme heading by full spelling. You.

However, in this search method, a dictionary search is performed for each of the headings having the same first character but different other characters. Therefore, the number of searches is large, and a physically fixed time is required for the dictionary search. Therefore, there is a possibility that an enormous amount of time may be required for the analysis.

[0012] This disadvantage is caused by the fact that the present search method has a larger number of dictionary searches than the dictionary search method using the first character described above. For this reason, the conventional morphological analysis system generally employs a dictionary search method based on the first character and attempts to improve the analysis efficiency by devising a matching procedure and rules.

[0013] Therefore, in order to solve the above-mentioned drawbacks, it is a first object of the present invention to adopt a method of searching a dictionary by specifying a morpheme by a full spelling heading and to realize means for reducing the number of times of searching the dictionary. Becomes

2) Problem in connection determination information and determination method Conventionally, only part of speech information was often used as connection determination information. In this case, since the concatenation determination condition is loose, an unusual arrangement of morphemes is usually allowed, and erroneous morpheme division is performed, or a large number of morphemes are passed to parsing or the like with insufficient morpheme narrowing. there were.

In order to avoid this drawback, in addition to the part-of-speech information of morphemes, utilization information and other morphological information (for example, headings) are added to the connection determination information, and special rules and tables (matrix) are set. It is also considered that the connection determination is performed by using one fixed connection determination for any combination of various types of information such as part of speech, inflection, and heading. In many cases, a code (for example, an index of a connection determination matrix) is provided.

Therefore, even when referring to the same concatenation determination matrix, for example, only a part of speech for a certain morpheme, part of speech and inflection type for a certain morpheme, and part of speech and heading for a certain morpheme, etc. The connection determination matrix is referred to by one connection determination code individually set for the combination of.

For this reason, there is a drawback that the connection determination code is hard to be given depending on how to set rules and tables, and it is difficult for the user to use. In addition, since one connection determination code is assigned to one arbitrary combination, there is a limit to support all combinations, and when a new style appears, a new connection determination code is constantly added. Have to keep setting,
There was also a drawback that maintenance was troublesome.

Therefore, in order to solve the above-mentioned drawbacks, the information used for judging concatenation is classified into a part of speech, an inflected type, and an inflected type, and further separated into independent three-dimensional information. For the shape, the part of speech of the later morpheme,
It is desirable to set a concatenation table defined by the information on the ending utilization level that determines whether the conjugation type can be concatenated, and this is the second problem.

[0019]

To achieve the above object, a Japanese morphological analysis system of the present invention has the following features.

The Japanese morphological analysis system according to the present invention comprises a dictionary storing at least information on morpheme headings, parts of speech, and ending use types for Japanese morphemes. In the Japanese morphological analysis system that cuts out the sequence in order from the beginning of the sentence, searches the dictionary as needed, and identifies the morpheme corresponding to the cut out character string, the cut-out character string is deformed by ending. A ending use unit that outputs at least the headings, parts of speech, inflection type, and inflected form of the dictionary registration format estimated to be associated with it as morpheme candidate information corresponding to the extracted character string, A connection table for determining whether or not connection at the part of speech level is possible using information on the part of speech of the morpheme candidate and information on the part of speech of the morpheme identified immediately before; Morpheme part of speech such candidates, ending utilizing type information and immediately preceding the identified morphological part of speech, ending conjugations,
Referencing a concatenation table that determines concatenation at the inflection level using information of conjugation, and determining whether concatenation is possible between the morpheme candidate and the morpheme identified immediately before, and determining that concatenation is not possible And a pre-search connection determination unit for rejecting the morpheme candidate.

In the above invention, the morpheme corresponding to the morpheme candidate obtained by searching the dictionary is obtained by using the part-of-speech information of the morpheme corresponding to the morpheme candidate and the part-of-speech information of the morpheme identified immediately before. A morpheme corresponding to the morpheme candidate and a post-search concatenation determination unit for determining whether or not the morpheme identified immediately before can be connected.

The post-search concatenation determining unit uses the part-of-speech information of the morpheme corresponding to the morpheme candidate obtained by searching the dictionary and the part-of-speech information of the morpheme identified immediately before, at the part-of-speech level. A connection table for determining whether connection is possible or not is provided.

In order to solve the above-mentioned problem, the morphological analysis method of the present invention in the Japanese morphological analysis system as described above modifies the ending of the cut-out character string and estimates at least accompanying the ending. Outputs the headings, parts of speech, inflection type, and inflection type in the dictionary registration format as information on morpheme candidates corresponding to the extracted character strings,
Before a dictionary search, a concatenation table that determines whether or not concatenation is possible at the part of speech level using information on the part of speech of the morpheme candidate and information on the part of speech of the morpheme identified immediately before, and a part of speech and ending of the morpheme candidate The morpheme candidate is identified immediately before by referring to a concatenation table that determines whether or not concatenation is possible at the ending conjugation level using the information on the type and the information on the part of speech, the inflection type, and the conjugation type of the morpheme identified immediately before. It is determined whether the morpheme candidate is connected with the morpheme candidate, and when it is determined that there is no possibility that the morpheme candidate and the morpheme identified immediately before are connected, the morpheme candidate is rejected, and the morpheme candidate is identified immediately before the morpheme candidate. If it is determined that there is a possibility that the morpheme is concatenated with the morpheme candidate, the dictionary is searched for the morpheme candidate, and the part-of-speech information of the morpheme corresponding to the morpheme candidate obtained by searching the dictionary is identified immediately before. By using the morpheme of the part of speech of the information,
The morpheme corresponding to the morpheme candidate and the morpheme identified immediately before are determined, and the morpheme corresponding to the extracted character string is identified based on the determination result.

[0024]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a morphological analysis system according to one embodiment of the present invention.

As shown in the figure, the morphological analysis system of the present embodiment includes a dictionary 10 in which morphemes are registered and stored together with various information on the morphemes, and a headline cutout for cutting out a character string from an input Japanese sentence. Part 2
And a ending use unit 3 that makes use of the ending of the character string cut out by the heading extraction unit 2 and converts it into a heading in the format registered in the dictionary 10; and a morpheme candidate corresponding to the character string deformed by the ending usage unit 3. A pre-search concatenation determination unit 4 that determines whether connection with a morpheme is possible before dictionary search and narrows down to only morpheme candidates that may be connected, a dictionary search unit 5 that searches the dictionary 10 for the morpheme candidate, and a dictionary search unit 5, a post-search concatenation determination unit 6 for determining whether or not connection between the morpheme identified by the search in step 5 and the immediately preceding morpheme is possible, an exception processing unit 7 for performing exceptional processing in a predetermined case, and these units. And a control unit 1 for controlling.

The ending use table 31 and the particle table 32 are referred to in the process of the ending use unit 3, and the part-of-speech class connection table 41 is referred to in the process of the pre-search connection determining unit 4.
And a utilization-utilization connection table 42, and a part-of-speech-part-of-speech connection table 61 as a determination table referred to in the processing of the post-search connection determination unit 6. In the present embodiment, “a morpheme corresponding to a cut-out character string” refers to a heading and an inflection that is actually searched and identified from a dictionary when a morpheme to be corresponded to the cut-out character string has already been identified. The term “morphological candidate corresponding to the extracted character string” refers to information including information and the like. The term “morphological candidate corresponding to the extracted character string” means that the morpheme to be used for the extracted character string has not been identified as a morpheme yet. It refers to a combination of two pieces of information, the headline candidate character string estimated at the time and the end use information.

The dictionary 10 stores, for the registered morphemes, syntactic information, semantic information, and the like, as well as ending usage information used in morphological analysis such as morpheme headings, parts of speech, and inflection types. In the case of a morpheme having an inflection, the morpheme heading registered in the dictionary 10 is the final form of the morpheme. Further, at least the part of speech of the morpheme and the inflection type are stored as the inflection utilization information. On the other hand, in the case of a morpheme having no inflection, the heading is used as it is, and the inflection type as the inflection information is a non-inflection type. As will be described later, the ending usage information can be used as it is as connection determination information without any deformation, together with a usage form added later.

In the dictionary 10, one morpheme is registered without separating the stem and the inflected ending.

The control unit 1 focuses on the input Japanese sentence in order from the character at the beginning of the sentence, and the heading cutout unit 2
The morphemes identified by a series of processes of the exception processing unit 7 are stored in a morpheme sequence stack (not shown). Thereafter, the morphemes stored in the morpheme sequence stack can be freely referred to in the processing of the connection determination unit 4 before search and the connection determination unit 6 after search. Further, the control unit 1 includes a backtrack processing unit, an unknown word determination unit, and a morpheme determination unit (all are not shown).

Here, when the immediately preceding morpheme is determined, or in the case of the first processing for the input sentence, the control unit 1 pays attention to the next new character in the determined range and sends the The character string following the character of interest and the position of the character of interest are passed. When the morpheme cannot be determined for the character of interest, 1) when no morpheme candidate can be connected in the pre-search connection determination unit 4, and 2) the morpheme candidate corresponds in the dictionary search unit 5. If no morpheme to be searched could be retrieved or none could be matched, 3) if no morpheme could be concatenated in the concatenation determination unit 6 after search, and 4) an appropriate morpheme was If none of them can be specified, the character string after the focus and the position of the character under the focus are handed again to the headline clipping unit 2.

On the other hand, if the heading clipping unit 2 does not allow clipping, the backtracking process is started. If backtracking is successful, the character of interest is moved to the next character in the range identified by the backtracking process. If backtracking fails, the unknown word processing is activated, and then the character of interest is moved to the next character in the range identified by the unknown word processing. The heading cutout unit 2 includes a control unit 1
From, the character string after the current character in the input sentence and the position of the current character in the input sentence are passed.

When receiving the character string and the position of the character under consideration, the heading cutout unit 2 assumes that the character under consideration is the first character of the heading of the morpheme candidate, and determines a predetermined range that can be considered as the heading of the morpheme candidate. Is extracted from the character string (the character string following the character of interest) passed from the control unit 1.

Here, since the heading extraction unit 2 basically uses the “longest matching method” (priority is given to the longest morpheme) which is generally used, the character after the newly focused character is used. When a column is passed from the control unit 1,
First, within a predetermined range that can be considered for characters after the character under consideration (for example, within a range that does not exceed the character representing a delimiter, or within a range that does not exceed the maximum heading length of a morpheme registered in the dictionary, etc.) Assuming the longest range of character strings as morpheme candidate headings, cut out the range of character strings.

However, if an appropriate morpheme corresponding to the cut-out character string is not found by the subsequent processing, the character string after the focused character is controlled again while the focused character remains unchanged. (See 2) to 4) above), and in this case, a character obtained by deleting one (arbitrarily if necessary) the last character of the cut-out character string is replaced with a shorter new character. It is cut out again as a morpheme candidate heading.

The headline cutout unit 2 stores information relating to the range of the cut-out character string, that is, information which can determine the character of interest in the input sentence, the position of interest, the length of the cut-out character string, and the like. Keep it. With this information,
It is possible to determine whether or not the same character string passed from the control unit 1 is the same as the character string extracted immediately before and the focused character is unchanged. If the character string is unchanged, a character shorter than the character string extracted immediately before is determined. The line is cut out. Needless to say, there is no such restriction when the character string extracted immediately before and the focused character, that is, the focused position are different.

When a character string of one or more characters can be cut out from the character string passed from the control unit 1, the heading cutout unit 2 uses the cutout character string (a morpheme heading candidate character string) as a ending use unit. Pass to 3. When a character string of one or more characters cannot be cut out, it is determined that cutout is impossible, and the process returns to the control unit 1. FIG. 2 briefly shows a specific example of the above-described headline clipping method.

The ending part 3 receives the morpheme heading candidate character string from the heading cutting part 2. Then, the presence or absence of inflection is determined from the last form of the morpheme heading candidate character string.

When it is determined that the morpheme heading candidate character string utilizes the ending, the ending utilization unit 3 assumes that the morpheme heading candidate character string is a heading utilizing a dictionary-registered heading (end-shaped heading). The morpheme heading candidate character string is transformed into a terminal heading. At the same time, the part-of-speech class estimated based on the deformation, the ending utilization information such as the ending utilization type and the utilization form, and the like are taken in. Of course, if it is determined that there is no ending usage, the morpheme heading candidate character string is not transformed, and it is assumed that the morpheme heading is a dictionary registration format heading, and the ending usage information or the like corresponding to the unmodified is fetched. In the following, a morpheme heading candidate created from the morpheme heading candidate character string by modification or non-deformation in the ending utilization unit 3 is hereinafter referred to as an “ending utilization heading”.

The ending utilization unit 3 has an ending utilization table 31 as shown in FIG. 3, and refers to the table for the hiragana portion of the last one to four characters of the morpheme heading candidate character string, and is estimated therefrom. 1) Inflection heading 2)
Information on the morpheme candidate (morpheme candidate information) including information such as part of speech, 3) inflection type, and 4) inflection type is created.

The morpheme candidate information corresponds to the part of speech to be used at the end of the word, that is, to one of a verb, an adjective, an adjective, and an auxiliary verb. On the other hand, there is usually a case where the morpheme heading candidate character string is a heading as a morpheme that does not utilize the ending. Therefore, the ending usage unit 3 sets the part of speech of the morpheme candidate that does not use the ending to be a non-utilizing class, sets the ending usage type to the non-utilization type, and sets the usage type to the non-utilization type in the same format as the information in the above 1-4. Morphological candidate information of morphological candidates to be used is also created.
Here, the part-of-speech is obtained by roughly grouping the part-of-speech of an actual morpheme, and since the dictionary search has not been performed for the morpheme candidate, this part-of-speech is used.

The morpheme that does not utilize the endings refers to morphemes such as nouns, adverbs, and particles, and corresponds to morphemes of all parts of speech other than predicates. However, among them, particles often have a close connection with neighboring morphemes. For this reason, the ending use unit 3 of this embodiment further distinguishes morpheme candidate information that does not utilize endings into morpheme candidate information corresponding to particles and other morpheme candidate information that does not utilize endings.

That is, the ending use unit 3 has a particle table 32 as shown in FIG. 4, and sequentially matches the morpheme heading candidate character string to the particle table 32, and estimates the particle The morpheme candidate information corresponding to the class is taken. As a result of collation with the particle table 32, if there is information as a particle class, not only morpheme candidate information as an ineffective class but also morpheme candidate information as a particle class can be obtained. Note that the particle classes referred to here are case particles, connecting particles, final particles, and auxiliary particles, which are roughly grouped into parts of speech corresponding to actual particles.

After the above processing, the ending use unit 3 puts the obtained arbitrary morpheme candidate information into one list (morpheme candidate group). FIG. 5 shows an example of a morpheme candidate group.

Then, the ending use unit 3 passes the morpheme candidate group obtained by using the ending to the pre-search connection determination unit 4.

The pre-search concatenation determination unit 4 determines whether each morpheme candidate of the morpheme candidate group of interest can be connected to each morpheme of the morpheme group adjacent to the head of the sentence identified immediately before. Judge one and use ending part 3
Out of the morpheme candidates received from the morpheme candidates, only the morpheme candidates that may be connected can be selected and the rest are excluded.

The pre-search connection determination unit 4 has a part-of-speech-part-of-speech connection table 41 as shown in FIG. 6, and determines whether or not the part-of-speech of the morpheme immediately before and the part-of-speech of the morpheme candidate can be connected. First, by referring to the morpheme sequence stack storing the already identified morpheme, 1
One morpheme is taken out, one morpheme candidate is taken out from the morpheme candidate group, and the part-of-speech-speech is obtained by combining the part of speech of the morpheme (<before> in FIG. 6) and the part of speech of the morpheme candidate (<after> in FIG. 6) The data is collated with the similar connection table 41 and the data at the corresponding position is fetched.

The above processing is performed for all combinations of morphemes and morpheme candidates of the immediately preceding morpheme group and the morpheme candidate group of interest. The data of the part-of-speech-part-of-speech connection table 41 in FIG. 6 includes “0” indicating that connection is not possible, “1” indicating that connection is possible, and connection indefinite (participation is possible at the part-of-speech level, but connection is undefined at the ending use level). It is any numerical value of "2" shown.

Further, the pre-search concatenation determining unit 4 has a conjugation-conjugation connection table 42 as shown in FIGS. 7 and 8, and the part of speech of the morpheme under concatenation determination and the part of speech of the morpheme candidate. -If the data of the part-of-speech type connection table 41 corresponds to "2 (connection undefined)", the data is collated with the utilization-utilization connection table 42. In this case, the data of the conjugation-conjugation table 42 is referred to by the part-of-speech (kind), conjugation type, conjugation type of the immediately preceding morpheme, and the part-of-speech class and conjugation type of the morpheme candidate of interest.

In this case, 1 of the "probable form 1" is the same probable form. For example, in the case of the five-step inflection type verb "buy", the first verb such as "buy" and the verb "buy" Since there is a fifth stage, and the possibility of connection is different from each other, they are subdivided to distinguish them. The same applies to other inflected forms. Further, when referring to the data of the conjugation-conjugation connection table 42, the "part of speech" of the immediately preceding morpheme is passed as one of the arguments. However, the data of the concatenation table is common to "part of speech" and is passed. The data of “part of speech” corresponding to the “part of speech” is referred to. Similarly, as the morpheme utilization type, for example, “Ba line lower one-stage utilization type” is passed, but the data in the connection table is common to the “lower one-stage utilization type”, and the data may be referred to. is there.

The data in the utilization-utilization connection table 42 in FIGS. 7 and 8 is either "0" indicating that connection is impossible or "1" indicating that connection is possible. "Possible". In the figure, for convenience of explanation, only some combinations of data are extracted and described. In addition, only data for a specific part of speech of a morpheme candidate of interest that can be connected is described for a specific part of speech, inflection type, and inflection form of the immediately preceding morpheme. For example, FIG. 7A shows only verbs, auxiliary verbs that can be concatenated with the five-row inflection type, and forms 1 to 3, and auxiliary verbs not present here cannot be concatenated.

Similarly, FIG. 7B shows only auxiliary particles, auxiliary particles having no auxiliary verb, and only particles which can be connected to the conjunctive forms 1-3. Become. FIG. 7C shows only auxiliary verbs that can be concatenated with auxiliary verbs, conjugations without auxiliary verbs, and conjunctive forms 1 to 3. All auxiliary verbs that are not included here cannot be concatenated. FIG. 8 (D) shows verbs, lower one-step inflection type, predecessor form 1
3 shows only the auxiliary verbs that can be concatenated, and all the auxiliary verbs that are not included here cannot be concatenated. FIG. 8 (E)
Indicates only the auxiliary verbs that can be concatenated with the verbs, the lower one-step conjugation type, and the conjunctive forms 1 to 3, and all the auxiliary verbs that are not included here cannot be concatenated. Further, FIG. 8F shows that there is no auxiliary verb that can be concatenated with the verb, the lower one-step utilization type, and the imperative forms 1-2.

Therefore, for example, when the immediately preceding morpheme has information as shown in Table 1 below, even if the morpheme candidate under consideration is an auxiliary verb, the ending usage type is “auxiliary verb conjugation type” or “auxiliary verb conjugation type”. 7 (A) in cases other than "conjugation type,""conjugation type without auxiliary verbs," and "conjugation type without auxiliary verbs"
As shown in (1), data cannot be connected ("0"), and this morpheme and morpheme candidate cannot be connected. In this case, since "buy" is "probable form 1", "auxiliary verb conjugation type" which can be connected to "probable form 3" is irrelevant.
Although not shown in FIGS. 7 and 8, there is only an auxiliary verb that can be connected to the probable forms 1 to 3 of the verb. Therefore, for the morphemes shown in Table 1, naturally, parts of speech other than auxiliary verbs cannot be concatenated.

[0053]

[Table 1]

Further, for example, in order to determine whether or not the morpheme “examine (verb instruction form“ examine ”)” and the morpheme candidate “u (auxiliary verb ending form“ u ”)” can be connected, “examine” is used.
It is assumed that the information of the morpheme corresponding to “” and the information of the morpheme candidate corresponding to “U” are obtained as shown in Table 2 below. When this is collated with the utilization-utilization connection table 42, the following Table 3 is obtained.
It is determined that these cannot be connected. Therefore, the morpheme candidate corresponding to "U" can be rejected before the dictionary search, and it is not necessary to perform the dictionary search.

[0055]

[Table 2]

[0056]

[Table 3]

The utilization-utilization connection table 42 described above
Pure grammar, that is, collation is performed using data determined by three types of independent general information such as morpheme part-of-speech, inflection type, and inflection type, and there is no need to set unique data in the system. Even morphemes that do not use inflections, such as particles, that have a certain rule in their concatenation, can be treated in the same way.

As described above, the pre-search concatenation determination unit 4 determines a new morpheme candidate group that has been narrowed down to only morpheme candidates that can be connected to any morpheme of the immediately preceding morpheme group among the morpheme candidates of the passed morpheme candidate group. create. by this,
The number of morpheme candidates can be significantly reduced before the dictionary search, and the number of dictionary searches can be reduced. In addition, since the pre-search concatenation determination unit 4 performs the concatenation determination using the information on the inflection level, it rejects those that could not be rejected by the concatenation determination using only the part-of-speech level information such as "verb and auxiliary verb". In many cases, analysis accuracy can be improved.

After the above processing, the pre-search connection determination unit 4
If one or more morpheme candidates in the morpheme candidate group can be connected to the immediately preceding morpheme, information on the morpheme candidates that can be connected to the immediately preceding morpheme is passed to the dictionary search unit 5 as a new morpheme candidate group. On the other hand, when none of the morpheme candidates in the morpheme candidate group can be connected to the immediately preceding morpheme, it is determined that there is no suitable morpheme candidate, and the processing proceeds to the heading clipping unit 2 via the control unit 1.

The dictionary search unit 5 includes a pre-search connection determination unit 4
First, the dictionary 10 is searched for the full spelling headings (ending usage headings) transformed by the ending usage unit 3 for all the morpheme candidates in the morpheme candidate group passed from (Morpheme information). Then, only those whose morpheme information obtained by the dictionary search matches the morpheme candidate information obtained by the processing of the ending utilization unit 3 are taken in as morphemes, and the rest are excluded.

Here, for the same cut of the input sentence,
Since polysemy and polyspeech are accepted, a plurality of morphemes are taken in as information in the same column. Further, since the morpheme information obtained by the dictionary search does not store the inflected form, if the morpheme information matches the morpheme candidate information, the inflected form in the morpheme candidate information is stored in the morpheme information. Output information. For example, in Table 4 below, when the character string of the input sentence is “buy” and the ending use headings of the morpheme candidate information are “buy” and “buy”, only the dictionary search result “buy” is registered in the dictionary. (Morphological information), the output information shown in Table 4 is obtained.

[0062]

[Table 4]

In some cases, the morpheme candidate information includes a plurality of morpheme candidates of the same inflection heading. However, the dictionary search unit 5 of this embodiment devises that the same inflection heading is searched only once. It is. For example, in Table 4 above, when the character string of the input sentence is “none” and the ending ending heading of the morpheme candidate information is “none”, there are two ending usage headings “none”. Is subjected to a dictionary search only once to obtain the output information shown in the (morphological information) table. In this case, since the end of the character string of the input sentence is "tsu", it is devised not to output the information of the morpheme candidate as the ineffective class corresponding to "none".

After the above processing, if the morpheme information is searched and one or more matches with the morpheme candidate information are found, the dictionary search unit 5 searches the list (morpheme group) of the morpheme information. It is passed to the connection determination unit 6. If no morpheme information can be retrieved, or if the morpheme information is retrieved but no morpheme candidate information is found, it is determined that there is no appropriate morpheme, and the control unit 1 determines that there is no suitable morpheme. To the heading clipping unit 2.

The post-search connection determination unit 6 includes a dictionary search unit 5
It is determined whether or not the morpheme of interest searched in the dictionary can be connected to the morpheme identified immediately before, that is, whether or not it can be positioned in the sentence as a syntactically adjacent morpheme.

The post-search connection determination unit 6 has a part-of-speech connection table 61 as shown in FIG. 9, and stores the part-of-speech (not part-of-speech) of the immediately preceding morpheme and the part-of-speech (part of speech) of the searched morpheme. ) Can be connected, first, one morpheme is extracted from the immediately preceding morpheme group with reference to the morpheme sequence stack, and one morpheme is extracted from the morpheme group of interest.
Two morphemes are extracted, and the part of speech of the morpheme immediately before (<previous> in FIG. 9) and the part of speech of the morpheme under focus (<post> in FIG. 9)
Is compared with the part-of-speech part-of-speech connection table 61, and the data at the corresponding position is fetched. Note that the part of speech referred to here is different from the part of speech, which is obtained by subdividing the part of speech, and the data in the part of speech part-of-speech connection table 61 is set separately even if the parts of speech belong to the same part of speech. You. Therefore, a stricter connection determination can be performed based on the part-of-speech-part-of-speech type connection table 41 used by the connection determination unit 4 before search.

The above processing is performed on a combination of all the morphemes of the immediately preceding morpheme group and all of the morphemes of the morpheme group of interest searched in the dictionary. The data of the part-of-speech-part-of-speech connection table 61 in FIG. 9 is a numerical value of either “0” indicating that connection is not possible or “1” indicating that connection is possible. For example, as shown in Table 5 below.

[0068]

[Table 5]

By the above processing, a set of morphemes of interest that is finally determined to be connectable to any one or more morphemes of the immediately preceding morpheme group is set as a new morpheme group of interest.

After that, when one or more morphemes remain as elements of the morpheme group of interest, the post-search concatenation determination unit 6 assigns those morphemes of interest to character strings in a predetermined range in the input sentence. The morpheme group of interest is passed to the control unit 1 via the exception processing unit 7. If one of the morphemes searched in the dictionary cannot be connected to the immediately preceding morpheme, the process proceeds to the headline clipping unit 2 via the control unit 1 without going to the exception processing unit 7.

The exception processing unit 7 activates individual procedural rules as necessary to eliminate the adverse effects of the longest match method and performs processing. If there is an inappropriate morpheme, Reject.

The control unit 1 activates the backtracking process when the extraction of the morpheme starting from the character of interest has failed. That is, if no morpheme starting with the focused character is found after the focused character, or if there is no morpheme that can be connected, the temporarily determined morpheme is re-divided.

Finally, when a morpheme including the character of interest as a heading is found, the new cutout method and the morpheme group are passed to the control unit 1. If a morpheme containing the character of interest as a heading is not found, a portion corresponding to a predetermined range of character strings including the character of interest is set as a dictionary unregistered word, and morpheme candidate information estimated before the search is used. The part of speech or the like is estimated for reference, identified as one morpheme, and the information is passed to the control unit 1.

As described above, the control unit 1 sequentially stores the morphemes (groups) identified as described above in the morpheme sequence stack. However, if the previous morpheme is also a dictionary unregistered word, the character of interest is absorbed and stored in the range of the morpheme heading immediately before.

Next, the operation of the present embodiment will be described in detail with reference to a specific example of morphological analysis processing of an example sentence. [First Analysis Processing Example] FIGS. 10 to 14 show processing examples in which a sentence (noun phrase) “Do you want to buy?” (Morphological analysis processing) is performed by this embodiment. In the Japanese sentence, a series of processes of the control unit 1 to the exception processing unit 7 based on the longest match method first focus on the character “bu” at the beginning of the sentence, and sequentially see FIG.
The character string is cut out as shown in FIG. 0 and FIG. 11, and the morpheme is identified.

Here, the character string "buy" is finally cut out and identified as a verb (use type: wa-row five-step use type, use type: unpredictable form 1) which makes the headline "buy". Usually, except for adjuncts such as particles and auxiliary verbs, it is possible to place them at the beginning of a sentence. Therefore, even if a concatenation judgment is performed before a dictionary search, it is located in the sentence that morpheme candidates at the beginning of a sentence can be rejected. Less than morpheme candidates. Therefore, also in this embodiment, as shown in the figure, the morpheme “buy”
Are processed until 22 is identified, but there is no such thing that the pre-search connection determination unit 4 determines that connection is impossible.

Incidentally, in the present embodiment, the ending utilization section 3 uses the ending utilization table 31 and the particle table 3
2 to determine the possibility of the existence of particles and auxiliary verbs. In this case, especially in this case,
It rejects the possibility that morphemes such as particles and auxiliary verbs correspond to morpheme heading candidate character strings starting with "buy". For this reason, the morpheme candidate information corresponding to the particles or auxiliary verbs is not passed to the pre-search concatenation determination unit 4.

However, if the pre-search connection determination unit 4
If a morphological candidate for a particle or auxiliary verb is passed to (for example, if "ha hurts" is analyzed and "ha" is cut out, "ha" , A morpheme candidate as a particle class is passed), the morpheme candidate as a particle class is determined not to be connected by the processing of the pre-search concatenation determining unit 4.

Next, as shown in FIG. 12, an analysis process is performed on "Isn't I Want?", And an auxiliary verb (Utilization type:
A morpheme of the inflected form without auxiliary verb, inflected form: continuous form 3) is identified. As shown in the figure, 16 types of inflection headings are processed until the morpheme “none” is identified, and of the 15 types of inflection headings, it is determined that concatenation is impossible by the processing of the pre-search concatenation determination unit 4. Search is skipped.

In the present embodiment, even when there are a plurality of morpheme candidate information having the same inflection heading,
In consideration of efficiency, a dictionary search is performed only once for the ending use headline. Therefore, when there is a plurality of morpheme candidate information having the same inflection heading, a dictionary search is performed using the inflection heading unless all of the morpheme candidate information is determined to be inconsistent by the processing of the pre-search concatenation determination unit 4. It will be. For example, in FIG.
There are two types of morpheme candidate information for the adjectives and the auxiliary verbs, but since only the adjectives have been rejected by the processing of the pre-search concatenation determination unit 4, the dictionary search is performed with "no". I have.

Similarly, as shown in FIG. 13, an analysis process is performed on “Taika”, and an auxiliary verb with the heading “Ta” (conjugation type: auxiliary verb Inflection type, inflection type: A morpheme of terminal type 1 or continuous type 1) is identified. As shown in the figure, 11 types of inflection headings are processed by the time the morpheme “ta” is identified. Of these, ten types of inflection headings are determined by the pre-search concatenation determination unit 4 to be incapable of concatenation. Search is skipped.

Normally, in the articulation judgment using only the part-of-speech level information (or when the articulation judgment is not performed), since the auxiliary verb can be connected after the auxiliary verb, for example, "no" (auxiliary verb) followed by "tai" (Auxiliary verb) is also determined to be concatenated, and a dictionary search must be performed for the morpheme candidate “tai”. In this embodiment, however,
Since the concatenation determination using the information on the inflection level is performed in the pre-search concatenation determination unit 4, these morpheme candidates can be rejected before the dictionary search as shown in FIG.

Finally, as shown in FIG. 14, an analysis process is performed for “squid”, and for the character string of “squid”, a noun whose heading is “squid” (conjugation type: non-conjugation type,
The morpheme “utilization type: non-utilization type” is identified. In addition,
Here, a dictionary search is also performed for a morpheme candidate corresponding to the verb “Iku”, and a morpheme corresponding to this is found, but is rejected (ignored) in the subsequent connection determination with the end of the sentence (punctuation).

As described above, conventionally, dictionary search was performed for all types of morpheme heading candidate character strings. In the present embodiment, however, the character string determined to be unconnected by the pre-search connection determination unit 4 is not used. Since a dictionary search is not performed for a column, the time required for the entire analysis process is greatly reduced.

[Second Analysis Processing Example] Next, a processing example in which a sentence “Look up” is subjected to morphological analysis processing according to this embodiment will be described. In the Japanese sentence, a morpheme for “check” is first identified by a series of processes of the control unit 1 to the exception processing unit 7 based on the longest match method, and then a morpheme candidate for “u” is processed by the ending use unit 3. And the morpheme information and the morpheme candidate information for them are as shown in Table 2 above.

The pre-search concatenation determination unit 4 determines “check” in Table 2.
First, the morpheme information for “U” and the morpheme candidate information for “U” are checked against the part-of-speech-part-of-speech connection table 41. Here, it is determined that the concatenation is indefinite due to the combination of the verb and the auxiliary verb. Next, the morpheme information and the morpheme candidate information in Table 2 are collated with the utilization-utilization connection table 42, and
Obtain the information shown in.

In this case, since no auxiliary verb can be connected to the lower one-stage utilization type imperative form 2, it is determined that connection is impossible.
Therefore, a dictionary search is not performed for "U", and a heading shorter than "U" cannot be cut out, so that the morpheme corresponding to "Look up" identified immediately before is rejected, and backtracking is performed. , And morpheme candidate information for “you” are obtained as shown in Table 6.

[0088]

[Table 6]

The pre-search connection determination unit 4 first compares the morpheme information for “check” and the morpheme candidate information for “you” in Table 6 with the part-of-speech-part-of-speech connection table 41. Here, it is determined that the concatenation is indefinite due to the combination of the verb and the auxiliary verb. Next, when the morpheme information and the morpheme candidate information in Table 6 are collated with the utilization-utilization connection table 42, the following Table 7 is obtained.

[0090]

[Table 7]

In this case, since the auxiliary verb conjugation type can be concatenated with the lower one-stage inflected form 2 and the auxiliary verb conjugation type cannot be concatenated in the lower one-stage conjugation type 1, the two morphemes corresponding to "investigation" can be used. Of these, the morpheme of the continuous form 1 is rejected, and the morpheme of the probable form 2 is identified. In the above processing, if only the concatenation relation using only part-of-speech level information is determined, for example, since verbs and auxiliary verbs can always be concatenated, the accuracy of morpheme division is low, and unnecessary dictionary searches are rejected. You can't. However, as described above, according to the present embodiment, such an erroneous division pattern can be rejected, and an accurate division pattern can be obtained. Further, since this processing is performed before the dictionary search, In addition, the number of dictionary searches can be significantly reduced.

[0092]

As described above, according to the Japanese morphological analysis system and morphological analysis method of the present invention, a cut-out character string is transformed into a heading in a dictionary registration format by utilizing the ending,
Based on the relationship between the information of the inflection level estimated in that case and the information of the inflection level of the morpheme identified immediately before, before the dictionary search, the morpheme candidate and the morpheme identified immediately before The morpheme candidate is discarded when the possibility of concatenation is determined, and when it is determined that there is no possibility of concatenation, the number of dictionary searches can be significantly reduced and the analysis efficiency can be improved. .

Further, the concatenation determination data at the inflection level is defined by three types of independent information in a form conforming to a general grammar, so that there is an effect that the concatenation table can be easily set.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a Japanese morphological analysis system of the present invention.

FIG. 2 is a diagram illustrating an example of a process of extracting a morpheme heading candidate character string according to the embodiment;

FIG. 3 is a diagram illustrating an end-of-word utilization table used in the embodiment.

FIG. 4 is a diagram showing a particle table used in the embodiment.

FIG. 5 is a diagram illustrating an example of a morpheme candidate group obtained by a process of an end use unit.

FIG. 6 is a diagram showing a part-of-speech-type part-of-speech connection table used in the embodiment;

FIG. 7 is a diagram illustrating an inflection-part of speech class connection table used in the embodiment.

FIG. 8 is a diagram showing an inflection-part of speech class connection table according to the first embodiment;

FIG. 9 is a diagram illustrating a part-of-speech-part-of-speech connection table used in the embodiment.

FIG. 10 is a diagram illustrating a utilization-utilization connection table used in the present embodiment.

FIG. 11 is a diagram showing a utilization-utilization connection table according to the embodiment.

FIG. 12 is a diagram illustrating an example of an analysis process according to the embodiment;

FIG. 13 is a diagram illustrating an example of an analysis process according to the embodiment;

FIG. 14 is a diagram illustrating an example of an analysis process according to the embodiment;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Control part 2 Heading extraction part 3 Inflection part 4 Concatenation judgment part before search 5 Dictionary search part 6 Concatenation judgment part after search 7 Exception processing part 10 Dictionary 31 Inflection table 32 Particle table 41 Part-of-speech-part-of-speech class concatenation table (part-of-speech level 42 Utilization-utilization concatenation table (concatenation table specified by ending usage level information) 61 Part-of-speech-part-of-speech concatenation table (concatenation table specified by part-of-speech level information)

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁶ , DB name) G06F 17/20-17/28 JICST file (JOIS)

Claims (4)

(57) [Claims] A Japanese morpheme is provided with a dictionary storing at least morpheme headings, parts of speech, and inflection type information, and a predetermined range of character strings in an input Japanese sentence is cut out in order from the beginning of the sentence. In a Japanese morphological analysis system that searches the dictionary as needed and identifies a morpheme corresponding to the cut-out character string, the cut-out character string is ending deformed, and is estimated at least accompanying it. Ending part that outputs the headings, parts of speech, inflection type, and inflected form of the dictionary registration format as morpheme candidate information corresponding to the extracted character string. A concatenation table that determines whether or not concatenation is possible at the part of speech level using information and information on the part of speech of the morpheme identified immediately before; The morpheme candidate was identified immediately before referring to the morpheme candidate by referring to the information and the part of speech of the morpheme identified immediately before, the inflection type, and the concatenation table that determines whether or not concatenation at the inflection level using the information of the inflection form. A Japanese morphological analysis system comprising: a pre-search concatenation determination unit that determines whether or not connection with a morpheme is possible and rejects the morpheme candidate when it is determined that connection is not possible. 2. A morpheme corresponding to a morpheme candidate obtained by searching a dictionary, using the part-of-speech information of the morpheme corresponding to the morpheme candidate and the information of the part-of-speech of the morpheme identified immediately before. The Japanese morphological analysis system according to claim 1, further comprising a post-search connection determination unit that determines whether connection is possible between the morpheme corresponding to the candidate and the morpheme identified immediately before. 3. A part-of-speech-level concatenation unit using a part-of-speech information of a morpheme corresponding to a morpheme candidate obtained by searching a dictionary and information of a part of speech of a morpheme identified immediately before. 3. The Japanese morphological analysis system according to claim 2, further comprising a connection table for determining whether or not the Japanese morphological analysis is possible. 4. A Japanese morpheme is provided with a dictionary storing at least morpheme headings, parts of speech, and inflection type information, and a predetermined range of character strings in the input Japanese sentence is cut out in order from the beginning of the sentence. In the morphological analysis method of the Japanese morphological analysis system that searches the dictionary as needed and identifies a morpheme corresponding to the cut character string, the cut character string is ending deformed, and at least Heading, part-of-speech, inflection type, and inflected form of the dictionary registration format estimated as the morpheme candidate corresponding to the cut-out character string, and output the morpheme candidate part-of-speech A concatenation table that determines whether or not concatenation is possible at the part of speech level using information and information on the part of speech of the morpheme identified immediately before; And the concatenation table that determines whether or not concatenation is possible at the inflection level using information on the part of speech of the morpheme identified immediately before, the inflection type, and the inflected form, and is identified with the morpheme candidate immediately before. The morpheme candidate is rejected, and if the morpheme candidate and the morpheme identified immediately before are determined to have no possibility of being connected, the morpheme candidate is rejected and the morpheme candidate is identified immediately before. When it is determined that there is a possibility that the morpheme is concatenated with the morpheme candidate, the dictionary is searched for the morpheme candidate, and the part-of-speech information of the morpheme corresponding to the morpheme candidate obtained by searching the dictionary is identified immediately before. Using the part-of-speech information of the morpheme, it is determined whether the morpheme corresponding to the morpheme candidate and the morpheme identified immediately before are connected or not. Based on the determination result, the morpheme corresponding to the cut-out character string Morphological analysis method, characterized in that it will be identified.