JP4036064B2 - Morphological sequence processing apparatus and method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a morpheme sequence processing technique for discriminating a person name from a morpheme sequence.
[0002]
[Prior art]
Like other proper nouns, personal names often have important meanings in the text. For example, by extracting and discriminating the names of people in the text, it is possible to determine the importance of the text, etc., and to classify the text by purpose and content. Therefore, it is desired to extract a person name in the text. However, it is necessary to perform complicated processing such as syntax analysis and semantic analysis in order to determine whether the name is a person's name. Conversely, if the organization name can be grasped before syntactic analysis or semantic analysis, syntactic analysis or the like can be performed easily and reliably.
[0003]
[Problems to be solved by the invention]
The present invention has been made in view of the above circumstances, and an object thereof is to provide a technique for discriminating a person name from a morpheme string.
[0004]
[Means for Solving the Problems]
According to this invention, in order to achieve the above-mentioned object, the configuration as described in the claims is adopted. Here, prior to describing the invention in detail, supplementary explanations of the claims will be given.
[0005]
According to one aspect of the present invention, in order to achieve the above-mentioned object, the morpheme processing device: means for storing a personal name morpheme string part discrimination rule for discriminating a morpheme string part corresponding to a person name from a morpheme string The first characteristic of the preceding boundary candidate preceding the person name, the second characteristic of the succeeding boundary candidate following the person name, and the third characteristic of the person name part candidate corresponding morpheme or morpheme group of the morpheme sequence; A characteristic allocating means for assigning to the morpheme string morpheme or morpheme group, and applying the personal name morpheme string partial discrimination rule to the first characteristic, the second characteristic, and the third characteristic corresponding to the person name And a means for discriminating the morpheme row portion to be provided.
[0006]
In this configuration, it is determined whether or not a morpheme or morpheme group in a morpheme string has a characteristic of a candidate boundary of a person name or a characteristic of a candidate constituting a person name, and based on this, a predetermined morpheme string part is a person name It is determined whether it corresponds to. Therefore, it is possible to easily distinguish a person's name. Characteristics are also called features and attributes.
[0007]
The target person names are mainly first and last names, but either the last name or first name may be used. Not only Japanese but also other languages such as Korean and Chinese are applicable.
[0008]
As candidates for person name boundaries, morphemes or morpheme groups having characteristics of a person's title (such as job title) can be used.
[0009]
Based on the characteristics of the title prefix candidates (such as “original” and “previous”), the characteristics of the title suffix candidates that follow the title (such as “government” and “phase”), and the characteristics of the title part candidates The title characteristics can be given to the morpheme group.
[0010]
However, there are rare cases such as “Defendant” that are combined with other morphemes to form a long title, and it is preferable to assign a single title characteristic to such a case.
[0011]
Candidate characteristics of a person name boundary following a person name can be assigned to a word representing a title, which is a suffix that follows the person name, such as “san”, “sama”, “m”.
[0012]
In addition, the characteristics of candidate names can be assigned to particles, punctuation marks, midpoints, hyphens, and parentheses.
[0013]
The characteristics of the personal name boundary preceding the personal name can be assigned to a kanji string composed of a plurality of characters representing a career such as “management”.
[0014]
According to another aspect of the present invention, in order to achieve the above object, the morpheme string processing device includes: means for inputting a morpheme string; and characteristics required for at least one of the following first and last name discrimination rules: And a morpheme sequence part sandwiched between the following left boundary characteristic part and the following right boundary characteristic part by applying at least one of the following plural name discrimination rules: And a means for discriminating as a first and last name.
[0015]
The first and last name discrimination rules are as follows.
[0016]
[Rule 1]: | left boundary | family name (place name), name body, given name | right boundary |
[0017]
[Rule 2]: | left boundary | family name (place name), give name, name body [grb: +] | right boundary |
[0018]
[Rule 3]: | left boundary | family name (place name), give name | right boundary |
[0019]
[Rule 4]: | left boundary | family name (place name), name body + | right boundary |
[0020]
[Rule 5]: | left boundary | name body [flb: +], name body +, given name | right boundary |
[0021]
However, the characteristics of the leading boundary candidate preceding the first and last names are the left boundary, the characteristics of the subsequent boundary candidate following the first and last names are the right boundary, the characteristics of the surname or place name candidate are the family name (place name), and the first name candidate characteristics. Given name, the characteristics of the first and last name part candidates as name body, the characteristics of the first and last name part candidates following the common name as name body [grb: +], and the characteristics of the first and last name part candidates preceding the general last name as name body [ flb: +]. Furthermore, “+” after the characteristic represents one or more repetitions.
[0022]
The present invention can be realized not only as an apparatus or a system but also as a method. Of course, a part of the invention can be configured as software. Of course, software products used to cause a computer to execute such software are also included in the technical scope of the present invention.
[0023]
The above described aspects of the invention and other aspects of the invention are set forth in the appended claims and are described in detail below with reference to examples.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
Examples of the present invention will be described below.
[0025]
FIG. 1 shows the morpheme processing apparatus of the embodiment as a whole. In this figure, the morpheme processing apparatus includes a Japanese text input unit 10, a morpheme analysis unit 11, a characteristic assignment unit 12, a chunking unit 13, and a person name extraction. Unit 14, person name output unit 15, morphological analysis dictionary 16, chunking dictionary 17, and the like. The morpheme processing apparatus is realized by installing a computer program recorded on the recording medium 101 or sent via a communication network (not shown) in a computer system 100 such as a personal computer or a workstation. can do.
[0026]
The Japanese text input unit 10 inputs Japanese text to the morpheme processing device. The morpheme analysis unit 11 divides the input Japanese text into morphemes with reference to the morpheme analysis dictionary 16 and gives grammatical information. For example, “ChaSen” developed at Nara Institute of Science and Technology Graduate University can be used for the morphological analysis unit 11. The characteristic imparting unit 12 refers to the chunking dictionary 17 and imparts characteristics used for chunking to the corresponding morpheme.
[0027]
In this example, character candidates constituting a person name are registered in the chunking dictionary 17. That is, for example, 1990 and 1600 characters are extracted from the last name of the last name (flb) and the last character (frb) of the last name, for example, from 2437 Japanese surnames and registered. In addition, for example, 5550 and 4030 characters are extracted and registered from the 59183 Japanese names, for example, the first character (glb) and the last character (grb). The character characteristics flb, frb, glg, grb and the personal name character candidate characteristic name-body obtained by integrating them are assigned to the characters in the morpheme string.
[0028]
An example of what is registered in the dictionary (morpheme analysis dictionary 16) as the first name and last name is shown in FIGS.
[0029]
Chunking is to group a plurality of morphemes together, and to add grammatical information and characteristics to the group of morpheme strings. That is, in this example, each morpheme of the morpheme analysis result is held in the terminal node (leaf). The row of morphological analysis results is a row of terminal nodes. Each terminal node has a morpheme surface form, vocabulary form, morpheme part-of-speech and characteristics (role within part-of-speech, etc.) as attribute values. Chunking refers to checking whether or not a specific pattern exists by referring to the attribute value held by each associated node, and, if any, grouping a node string of a specific part together. At that time, a node is generated above the node string, and this node string is related to this node. A new attribute value can be set for a newly generated node. Note that the reference to the node column targets only the highest-level node.
[0030]
The chunking unit 13 performs chunking on a person name, a title, and the like using the chunking dictionary 17. The personal name extraction unit 14 extracts a series of morpheme strings assigned the personal name characteristics in the chunking unit 13 as personal names. The person name output unit 15 outputs the extracted person name (morpheme string). The personal name may be used in a later syntax analysis, or may be used for text selection or the like. It can be used for various other purposes.
[0031]
FIG. 2 shows the configuration of the chunking unit 13. In this figure, the chunking unit 13 includes a boundary detection unit 21, a rule application unit 22, and the like. The boundary detection unit 21 detects the front-rear boundary of a person name based on the characteristics of a morpheme or a group of morphemes. For example, the front / rear boundary of a person name is detected by the title “Professor”. This will be described in detail later. The rule application unit 22 discriminates a person name by applying five rules described later.
[0032]
FIG. 3 shows the configuration of the boundary detection unit 21 of the chunking unit 13. In this figure, the boundary detection unit 21 includes a long title detection unit 23, a left boundary detection unit 24, a right boundary detection unit 25, and the like. It is configured.
[0033]
The long title detector 23 discriminates long titles according to the following long title discrimination rule.
[0034]
[Long title rule 1]:
long title = head [title-prefix: +],? * [Title-body], noun [title]; noun [title-suffix].
[0035]
However,
long title: long title head: prefix title-prefix: title prefix title-body: title body noun: noun title-suffix: title suffix
For example, an example of “former Hosei University professor Kimi Komazaku” is subjected to morphological analysis as shown in FIG. 6, and the portion of “former Hosei University professor” is distinguished as a long title.
[0037]
[Long title rule 2]:
long title =? * [Title-body: +], noun [title: +]; noun [title-suffix: +].
[0038]
For example, an example of “Satoshi Yamaguchi National Science Museum Honorary Researcher” is subjected to morphological analysis as shown in FIG. 7, and “National Science Museum Honorary Researcher” is distinguished as a long title.
[0039]
Examples of title-prefix, title-suffix, and title-body are shown in FIG.
[0040]
Note that there are rare cases such as “Defendant” that are combined with other morphemes to form a long title, and such a title is assigned the characteristics of a single title (short title) and is incorrectly chunked. It is preferable that the long title is composed.
[0041]
The left boundary detection unit 24 and the right boundary detection unit 25 detect the front and rear (left and right) boundaries of the person name using the characteristics of titles (including long titles) and the like. Titles (including long titles) act as left and right boundaries.
[0042]
Predetermined particles, punctuation marks, etc., such as “Mr.” and “Mr.”, act as right boundaries as shown in FIG.
[0043]
A particle such as “no” (eg “Yokozuna no Musashimaru”), the beginning of a sentence (BOS), and a predetermined word consisting of two or more characters (eg “management” etc.) act as a left boundary.
[0044]
Next, a rule for name discrimination of the rule application unit 22 will be described.
[Person name discrimination rule 1]:
name = | left boundary | family name (place name), name-body, given name | right boundary |.
[0045]
However, name: personal name left boundary: left boundary of personal name family name (place name): surname or place name name-body: the word that forms the body of personal name given name: right boundary of personal name
For example, “Yamada Sachiko (Seki Univ.)” Is subjected to morphological analysis as shown in FIG. 10, and then “Yamada Sachiko” is discriminated as a personal name.
[0047]
[Person name discrimination rule 2]:
name = | left boundary | family name (place name), give name, name-body [grb: +] | right bound |.
[0048]
However, name-body [grb: +]: a word that forms the body of a person having the characteristics of a candidate for the last character of the name
For example, “Miyazono Kasei Defendant” is subjected to morphological analysis as shown in FIG. 11, and then “Miyazono Kasei” is discriminated as a personal name.
[0050]
[Person name discrimination rule 3]:
name = | left boundary | family name (place name), give name | right boundary |.
[0051]
For example, “Takeharu Gonda / Professor of Senshu University familiar with mystery history” is subjected to morphological analysis as shown in FIG. 12, and then “Yuji Gonda” is distinguished as a personal name. “Detailed” has a long word characteristic of detecting the left boundary of a person name.
[0052]
[Person name discrimination rule 4]:
name = | left boundary | family name (place name), name-body + | right boundary |.
[0053]
For example, “Mr. Kenji Isayama” is subjected to morphological analysis as shown in FIG. 13, and then “Kenji Isayama” is discriminated as a personal name.
[0054]
[Person name discrimination rule 5]:
name = | left boundary | name-body [flb: +], name-body +, given name | right boundary |.
[0055]
However, name-body [flb: +]: a word that forms the main body of a person name having the characteristics of a candidate for the first character of the surname
For example, “Mr. Haruhiko Hagimoto” is subjected to morphological analysis as shown in FIG. 14, and then “Harumoto Hamamoto” is discriminated as a personal name.
[0057]
According to this embodiment, it is possible to reliably discriminate a person name even with an unusual person name. For example, “Kiki” in “Yoda Ki Master” is a relatively rare name, and if it is not in the dictionary, “Kiki” and “Ki” will be separate parts, which are usually difficult to recognize as a human name. However, it can be surely grasped as a person's name by using both name-body characteristics. At this time, there is no possibility of misrecognition by using “master” or the like as the characteristic of the personal name boundary.
[0058]
Also, if “Shimayama Haruko” is an unusual surname, “Shimayama” is divided into “Shima”, “Mountain”, and “Haruko”, which are usually difficult to recognize as personal names. Using the name-body characteristic of “mountain”, it can be surely grasped as a person name.
[0059]
A specific configuration will be further described.
[0060]
The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the invention. For example, in the above description, the name-body characteristic is given by the characteristic assigning unit 12, but it may be given after the personal name boundary is detected. It goes without saying that the processes can be switched in order or performed simultaneously as long as the functions can be realized.
[0061]
【The invention's effect】
As described above, according to the present invention, a person name can be reliably and easily distinguished from a morpheme string.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an overall morpheme string processing apparatus according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a chunking unit in FIG.
FIG. 3 is a block diagram illustrating a boundary detection unit in FIG. 2;
FIG. 4 is a diagram illustrating an example of a Japanese surname.
FIG. 5 is a diagram illustrating an example of a Japanese name.
FIG. 6 is a diagram for explaining an example of chunking of a long title.
FIG. 7 is a diagram for explaining another example of chunking of a long title.
FIG. 8 is a diagram illustrating characteristics used for long title chunking.
FIG. 9 is a diagram for explaining characteristics for discriminating the separation of personal names.
FIG. 10 is a diagram for explaining Rule 1 for discriminating person names;
FIG. 11 is a diagram for explaining rule 2 for discriminating person names;
FIG. 12 is a diagram for explaining rule 3 for discriminating person names;
FIG. 13 is a diagram for explaining a rule 4 for discriminating personal names.
FIG. 14 is a diagram for explaining a rule 5 for discriminating person names.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Japanese text input part 11 Morphological analysis part 12 Character imparting part 13 Chunking part 14 Person name extraction part 15 Person name output part 16 Morphological analysis dictionary 17 Dictionary for chunking 21 Boundary detection part 22 Rule application part 23 Long title detection part 24 Left Boundary detection unit 25 Right boundary detection unit 100 Computer system 101 Recording medium

Claims (2)

Means for inputting a morpheme string;
Means for assigning to the morpheme sequence a characteristic required for at least one of the following first and last name discrimination rules;
The characteristics of the leading boundary candidate preceding the first and last names are left bound, the characteristics of the trailing boundary candidate following the first and last names are right bound, the characteristics of the surname or place name candidate are family name (place name), and the characteristics of the first name candidate are given. name, the name of the surname part candidate is name body, the name of the surname part candidate following the common name is name body [grb: +], and the name of the surname and the surname part is preceded by name body [flb: +], And when the “+” after the characteristic represents one or more repetitions,
[Rule 1]: | left boundary | family name (place name), name body, given name | right boundary |
[Rule 2]: | left boundary | family name (place name), give name, name body [grb: +] | right boundary |
[Rule 3]: | left boundary | family name (place name), give name | right boundary |
[Rule 4]: | left boundary | family name (place name), name body + | right boundary |
[Rule 5]: | left boundary | name body [flb: +], name body +, given name | right boundary |
Applying a plurality of at least one of the first and last name discrimination rules consisting of it and a means for discriminating morphemes portion sandwiched between the portion of the characteristic of the l eft b oundary characteristics of parts and r ight b oundary as first and last name A morpheme sequence processing device characterized by the above. Means for inputting a morpheme string,
Means for assigning to the morpheme sequence a characteristic required for at least one of the following first and last name discrimination rules;
The characteristics of the leading boundary candidate preceding the first and last names are left bound, the characteristics of the trailing boundary candidate following the first and last names are right bound, the characteristics of the surname or place name candidate are family name (place name), and the characteristics of the first name candidate are given. name, the name of the surname part candidate is name body, the name of the surname part candidate following the common name is name body [grb: +], and the name of the surname and the surname part is preceded by name body [flb: +], And when the “+” after the characteristic represents one or more repetitions,
[Rule 1]: | left boundary | family name (place name), name body, given name | right boundary |
[Rule 2]: | left boundary | family name (place name), give name, name body [grb: +] | right boundary |
[Rule 3]: | left boundary | family name (place name), give name | right boundary |
[Rule 4]: | left boundary | family name (place name), name body + | right boundary |
[Rule 5]: | left boundary | name body [flb: +], name body +, given name | right boundary |
A plurality of applying at least one of the first and last name discrimination rules, means for discriminating the morpheme string part sandwiched between the portion of the characteristic parts and. Right boundary of the characteristics of the left boundary as surname name consisting of,
A computer program for morpheme sequence processing, which causes a computer to function .

Images