JP2848430B2 - Information extraction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information extraction method capable of quickly performing pattern matching by first determining the classification of information to be extracted and extracting necessary information from a sentence. .

[0002]

2. Description of the Related Art Conventionally, as a method of extracting a list of documents having different orders and writing styles, for example, Japanese Patent Application Laid-Open
There is a meaning extraction method described in JP-A-61-61. In this method, first, input records having different description formats are divided into words by morphological analysis processing (word division and part of speech),
The words divided by the section dividing process are further divided into sections using a delimiter table prepared in advance, and a plurality of order analysis routines are executed in a predetermined order for each section divided in the next semantic analysis processing. By running in
Words are extracted by finding what the words in the section represent. Also, as another method, for example,
"Yacc / lex Program Generator on UN
IX], written by Kenji Satsuki, philosophy publishing. In this method, a word (a group of words and a group of words) is extracted from a sentence using the UNIX commands “yacc” (a C language generator of a lexical analysis program) and “lex” (a C language generator of a lexical analysis program) developed by UNIX System Laboratories. Build a mechanism to do it. That is, according to the mechanism of generating a word / phrase extraction program in accordance with the syntax rules of yacc and lex, and then inputting the program to yacc and lex, a word / phrase extraction program in C language is generated.

[0003]

However, each of the above methods has the following problems. First, in the method described in JP-A-5-20361, words can be extracted by using a delimiter (for example, a comma or a colon) in a sentence. It becomes impossible to extract a simple phrase. In addition, it is necessary to previously define in detail features for semantic analysis of words, such as the position in the whole, the length of the number of characters, and the era. Next, yacc and le
In the method using x, since the syntax rules of the UNIX command are extremely complicated, there is a problem that only a person who is familiar with the command can create a program. Even if a program can be created, since yacc and lex are simple program generators,
There is also a problem that there are various restrictions on the use, and that a normal person cannot create a practical word / phrase extraction program. SUMMARY OF THE INVENTION An object of the present invention is to solve these conventional problems and to extract necessary information from a sentence even if there is no delimiter in the sentence or even a person who is not familiar with the program, and can obtain practical information. It is an object of the present invention to provide an information extraction method capable of extracting information.

[0004]

In order to achieve the above object, an information extraction method according to the present invention is provided .
When extracting words between each predetermined phrase from the input sentences arranged in the position , first, the classification type of the input sentence among the predetermined words
Storage of multiple phrases used to identify
Arrangement of a plurality of predetermined phrases for each type and between each predetermined phrase
From information used to identify one or more words
Before storing multiple fixed patterns in
If there is a match with a predetermined phrase for classifying classification stored in advance,
Or the best matching words and phrases
Identify another, and store the stored fixed-format
From the turn, the sequence of each predetermined phrase in the input sentence
Select the best matching fixed pattern, and follow the sequence and information of each predetermined phrase indicated by the selected fixed pattern.
And extract each word between each predetermined word in the input sentence.
It is characterized by putting out. Is it a stored fixed pattern ?
Et al., When selecting a constant <br/> shaped pattern that best matches the sequence of the predetermined phrase input sentence candidates of a plurality of fixed pattern
If there is, for each fixed pattern, sequentially, each fixed pattern
Each predetermined phrase to determine whether or not another of the fixed pattern of, and counts the number that does not exist, you select a fixed pattern that best matches the largest fixed pattern is the number of counts.

[0005]

According to the present invention, for example, when extracting necessary information (identification items) from the input of a family register, all sentences written therein are composed of a plurality of identification items and a set of characteristic data. First, it is determined which case type (type of birth, marriage, death, etc.) the input family register data belongs to, and the data is input as the characteristic data of the description pattern of the specified case type. The feature data of the family register data is sequentially matched, and the description pattern having the most matching feature data is regarded as the description pattern of the family register data. Then, the input family register data is developed while referring to the description pattern, and necessary information is extracted. As described above, in the present invention, information to be extracted for each fixed pattern is stored in advance in which part of the fixed pattern, and necessary information is extracted from a sentence created based on the fixed pattern. In such a case, first, it is determined to which fixed pattern the sentence belongs, based on how many feature data present for each fixed pattern exist in the sentence. Since information is extracted from the sentence by matching with the determined fixed pattern, necessary information can be accurately extracted. In addition, since fixed patterns are classified based on specific feature data,
After determining which classification the sentence from which information should be extracted belongs to, it is only necessary to perform a matching process with a fixed pattern in the corresponding classification. As a result, it is not necessary to perform the matching process with all the fixed patterns, and the information can be quickly extracted.

[0006]

Embodiments of the present invention will be described below in detail with reference to the drawings. Here, a case where the present invention is applied to information extraction from a family register is described as an example. FIG. 3 is a diagram showing an example of a family register to be processed in the present embodiment, and FIG. 4 is a diagram showing an identification matter case type code. As shown in Fig. 3, the sentences written on the family register generally describe several cases, and each of them describes the case of the person or a person related to the person after the date. Have been. These cases are, for example, sentences indicating “birth”, “marriage”, “death”, etc., as shown in FIG. Note that FIG.
Contains only "birth" and "marriage". In this way, the sentence of the family register has a certain format, and as shown below, the sentence is composed of a plurality of case sentences. Family register sentence = "incident sentence""incidentsentence""incidentsentence" ... "incident sentence" Furthermore, each incident sentence is, as shown below, a set of a plurality of identity matters and characteristic data, or an identity. It consists of a set of items. Incident text = "identification matter""characteristicdata""identificationmatter""characteristicdata" ... Here, the identification matter is a matter related to the identity such as date and place. Is data for specifying a description pattern, that is, characteristic case data such as birth, marriage, and death.

By the way, incident sentences are classified into four patterns as follows. Here, NL is an abbreviation of new line, and indicates a line feed. Pattern 1 = "identification matter""identificationmatter" Pattern 2 = "identification matter""NL" Pattern 3 = "identification matter""characteristicdata" Pattern 4 = "characteristic data" For example, "1993 The birth date in Chiyoda-ku, Tokyo on the 5th of March, and the birth on the 6th of the same month, the father was registered, and the pattern data is as follows:
"(Birth date) (birthplace) and birth (notification date) (notifier) notification registration and enrollment" is applied to the above four patterns and becomes as follows. Incident text = “(identification item) (identification item) (characteristic data)
(Identity matters) (Identity matters) (Feature data)

FIGS. 1 and 2 are operation flowcharts of an information extracting method according to an embodiment of the present invention, and particularly show a process of extracting necessary information (identification items) from an input of a family register. FIG. 5 is a development view of the internal table, and FIG. 6 is a data configuration diagram of the description pattern definition file. First, as shown in FIG. 1, after reading and registering the description pattern data from the description pattern definition file, the feature data for specifying the incident type from the entry item incident type is described.
Create data and deploy it to a table. Here, the feature data
Ta is a word such as "birth" or "marriage",
Hereinafter, it is called a keyword. Also, feature data for specifying a description pattern from registered description pattern data.
Data is extracted and developed in a table (step 400).
The concept of the expanded internal table is as shown in FIG. In Figure 5, "Birth", "Marriage registration, marriage and marriage acquisition by marriage cancellation", "Divorce", "Nationality acquisition naturalization"
Nationality selection Loss of nationality Husband nationality Wife nationality]. As shown in FIG. 6, the structure of the description pattern data stored in the description pattern definition file is divided into a plurality of patterns for each case such as birth, marriage, etc. All data is stored. The following three patterns are shown in the pattern of FIG. Birth 1 = (Birth Date) (Birthplace) Birth (Notification Date) (Notified Person) Reported Enrollment, Birth 2 = (Birth Date) (Birthplace) Birth (Notified Date) (Notified Person) Reported (Received Birth 3 = (birth date) (place of birth), birth (notification date) (notified person) (special notes)

In the flow of FIG. 1, next, family register data is input (step 401). Note that multiple family register data
If the data has been input, the family register data to be processed next is extracted. It is determined whether all family register data has been input and processed, and the process is continued until all family register data has been input (step 402). When the family register data is input, next, matching processing is performed between the description pattern data stored in the description pattern data definition file and the family register data input from the family register data file. Prior to the matching process, the description item case type is first specified (step 403). In this family register data file, the data in the identity column on the family register is stored as one record per sentence. However, since the number of all the described pattern data registered is enormous, it takes too much time to perform the matching process with all the patterns. Therefore, in order to avoid matching with all the registered pattern data, the case type is first specified before the matching process. As the specifying process, the case type is specified in advance by a keyword for specifying the described item case type and the description pattern specifying process described below.
At this time, if it is not determined uniquely (4030), matching is performed with the feature data of the written pattern for each case type (4031). In this case, the characteristic data of the description pattern of the case type different from the case type of the target sentence
Since there is no need to match the data, the description pattern can be uniquely determined.

When the description item case type is determined in this way (step 4030), the description pattern is specified next. The description pattern is specified by matching the input data and the characteristic data (data for specifying the description pattern) with respect to all the description patterns of the target case type in the order of development of the internal table (step 404). In this case, uniquely pattern - when it is determined that emissions matching (step 405), but ends the matching (step 408), when not uniquely determined, as described below, flow of FIG. 7 - described by One pattern is specified using the pattern specification processing (step 4050). If it is not possible to specify one even by using this processing (406), as will be described later, it is specified at the time of developing the identity item (step 407). If there is no pattern that matches the description pattern, an error occurs. FIG. 7 is a flowchart showing the written pattern specifying process in FIG. In the description pattern specifying process (step 4050) in FIG. 1, first, it is sequentially checked whether or not the feature data of all the target description patterns exists in the feature data of the other target description patterns. I do. Then, the number that does not exist is counted for each written pattern (step 209), and the written pattern with the largest count is set as an expansion target of the identity item. That is, in this case, as a result of matching the sentence feature data with a plurality of fixed patterns, when a fixed pattern with which the most feature data matches cannot be specified, that is, the most feature data is determined. This is when there are a plurality of matching fixed patterns. In such a case, it is determined whether or not the feature data of the last remaining fixed patterns is present for each fixed pattern in the feature data of another fixed pattern. Count the number not to do for each fixed pattern,
The fixed pattern with the largest count value is regarded as the fixed pattern of the sentence. The principle of this processing is that when a term in a sentence is a term that does not exist in the feature data of a prepared standard pattern, that is, when it is determined that it is a null character, it cannot be specified as one standard pattern. This is because it is considered that the best method is to search for a fixed pattern that has the most feature data that does not exist in the feature data of a plurality of fixed patterns, such as a null character.

For example, a process when the following three described patterns are targeted will be described. Described pattern 1 “Birth at”, “Notification”, “Registered to send” Described pattern 2 “Birth at”, “Notification with nationality hold”, “Registered to send at” Described pattern 3 ... “Birth at” , "Notification" First, the number of the feature data of the described pattern 1 ("Birth at,""Notification","Registeredfrom") that does not exist in the feature data of the other described patterns (2 and 3) is determined. Ask. "Birth at birth" is not counted up because both of the description patterns 2 and 3 exist. Since “notification” exists in both the description patterns 2 and 3, the count is not performed. Although “from sending enrollment” exists in the description pattern 2 but does not exist in the description pattern 3, it is counted up by one. From the above results, the count number of the written pattern 1 is “1”. Next, the number of the characteristic data of the description pattern 2 (“Birth at birth”, “Notification with nationality pending”, and “Registration sent from”) that do not exist in the characteristic data of the other description patterns (1 and 3) Ask for. "Birth at birth" is not counted up since both of the description patterns 1 and 3 exist. "Notification with nationality hold" is counted up by 2 since neither of the description patterns 1 and 3 exists. Although "from sending enrollment" exists in the description pattern 1 but does not exist in the description pattern 3, it is counted up by one. From the above results, the count number of the described pattern 2 is “3”. Finally, the number of the feature data of the description pattern 3 ("Birth at", "Notification") that does not exist in the feature data of the other description patterns (1 and 2) is determined. "Birth at birth" does not count up because both description patterns 1 and 2 exist. Since “notification” exists in both the description patterns 1 and 2, the count-up is not performed. From the above results, the count number of the described pattern 3 is “0”.
It is.

As a result of the check, since the count number of the description pattern 2 is the largest, the description pattern 2 is set as the subject of development of the identity item. In other words, this is a processing method in which a description pattern having a large amount of feature data that does not exist in the feature data of another description pattern is set as an expansion target of the identity item. In FIG.
The flow of the above described description pattern specifying process is shown. Until there are no more target patterns (1, 2, 3) to be compared (Step 201), the counter and the non-coincidence area are initialized (Step 202), and until there is no more characteristic data of the target pattern (Step 203). Initial setting for matching is performed (step 204). When the matching with the target pattern for comparison is completed,
The process moves to the next target pattern (steps 205 and 203).
If the target patterns are the same, the process proceeds to the next target pattern without counting (steps 206 and 205). If the target patterns are not the same, initialization is performed for matching (steps 206 and 207), and if the result of the matching is a null character (meaningless character), a count is performed (step 208). , 209). If the feature data exists, the same pattern is checked for the target pattern, and the process proceeds to the matching process (steps 210, 205,
206, 207). When the feature data to be compared in the target pattern is completed (step 203), the value of the mismatch number area is compared with the value of the counter (step 21).
2) When the mismatch area is large, the process proceeds to the comparison with the next target pattern. When the counter value is large, the counter value is written in the mismatch area and the written pattern number is stored (step 213). Move on to comparison with the pattern. When there are no more target patterns to be compared (step 201), the process ends.

Returning to FIGS. 1 and 2, after performing a description pattern specifying process as shown in FIG. 7 (step 4050), it is uniquely determined (step 406), and when the matching is completed (step 408), it is specified. The identification information is developed using the written pattern data (step 409). Hereinafter, an example of development of the identity item will be described. [Example of identification item development] Input data: "Birth day in Chiyoda-ku, Tokyo March 1, 1993 Notification of father on the same month 2 Sending enrollment from the mayor of the same month on the same month 3" Description pattern: (birth date) (place of birth) Birth at (date of notification)
(Notifier) Notification (date of receipt) From (recipient) Matching with the pattern data described in the enrollment registration, and develop the identification information. (A) The head information of the described pattern data is "birth date"
Then, it is determined that the next information is an identity item. In this case, since "birth date" is "date", it is separated by "day".
"March 1, 1993" is extracted. (A) It is determined that the next information is "place of birth" and the next is a separator. Accordingly, the feature data is separated by "departure" and "Chiyoda-ku, Tokyo" is extracted. (C) It is determined that the next information is the “notification date” and that the next is an identity matter. Therefore, since the “report date” is “date”, it is separated by “day” and “same month 2” is extracted. (D) The next information is the "reporter" and the next is the separator. Therefore, "father" is extracted by separating with the feature data "notification". (E) The next information is the "date of receipt", followed by the identity. Therefore, since the “date of receiving the transmission” is the “date”, it is separated by the “day” and “the same month 3” is extracted. (F) The next information is the "acceptor" and the next is the separator. Therefore, it is separated by the feature data
"Same ward mayor" is extracted. (G) The next information is NL (new line). Therefore, the development of the status of the input data is completed.

Returning to FIGS. 1 and 2, the processing subsequent to the development of the identity item (step 409) will be described. When the data has been expanded to NL (step 410), the identification data is set in the work area (step 411). The sentence is restored using the identification item data set in the work area and the written pattern data, and the consistency with the input data is checked (step 412). If no inconsistency occurs (step 413), the identification data of the work area is set in the save area (step 41).
4). On the other hand, if there is an inconsistency, the pattern becomes an exceptional description pattern (step 415), which is printed and output (step 4150). Of the identification item data set in the save area, omitted ones are added according to a predetermined rule to restore the identification item data (step 416). For example, “Same month” is changed to “March”, which corresponds to this. When the development of the identification information is completed by the above procedure (step 417), the contents of the identification information developed in the save area and a separate file (a file storing information other than the identification information on the family register) are stored. The consistency with the contents is checked (step 418). The check items include, for example, the name of the father or the like, the context of the date, the year of the era (correspondence with Seishi), and the like. Finally, the extracted identification information expanded in the save area is stored in the identification information file (step 419). In addition,
The storage in the identity item is performed regardless of the result of the consistency check. Also, the identification data is restored (step 41).
6) Or the process of checking consistency (step 412) is a process unique to family register data.
If the data is other than family register data, it can be omitted.

[0015]

As described above, according to the present invention,
(A) Since information is extracted from a sentence based on a predetermined fixed pattern, necessary information can be accurately extracted without a delimiter in the sentence. (B) The fixed patterns are classified based on specific feature data.
After first determining to which class a sentence from which information should be extracted belongs, a matching process is performed with a fixed pattern in the corresponding class. Therefore, it is not necessary to perform a matching process with all fixed patterns, and quick information can be obtained. An extraction can be performed.

[Brief description of the drawings]

FIG. 1 is a part of a flowchart of a process of extracting necessary information from input of a family register as one embodiment of the present invention.

FIG. 2 is another part of the flowchart of the information extraction process.

FIG. 3 is a diagram showing an example of a family register used in FIGS. 1 and 2;

FIG. 4 is a diagram showing a list of identification matter case type codes used in FIGS. 1 and 2;

FIG. 5 is a development view of an internal table used in FIGS. 1 and 2;

FIG. 6 is a data structure diagram of a description pattern data definition file used in FIGS. 1 and 2;

FIG. 7 is a detailed flowchart showing a written pattern specifying process in FIGS. 1 and 2;

[Explanation of symbols]

 01-32 Identity matter case type code

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A-56-60935 (JP, A) JP-A-2-23413 (JP, A) JP-A-5-120269 (JP, A) (58) Investigation Field (Int.Cl. ⁶ , DB name) G06F 17/30 JICST file (JOIS)

Claims (3)

(57) [Claims] 1. An input in which a plurality of predetermined words are arranged at an arbitrary position.
From the force statement, an information extracting method for extracting words in between the predetermined phrases used in a particular classification type of the input sentence of said predetermined word
Storing a plurality of predetermined words and phrases in advance, and for each of the classification types, an arrangement of the plurality of predetermined words and
Used to identify one or more word breaks between given words
For storing a plurality of fixed patterns of information in advance.
From the input sentence and the input sentence,
Extract the phrase that matches or best matches the fixed phrase
Identifying the classification type of the input sentence by
From the previously stored fixed pattern was classified type specified in-up or match the sequence of the predetermined words in the input sentence
Selecting a fixed pattern that best matches
And each predetermined word represented by the fixed pattern selected in the step.
According to the arrangement of phrases and information, between each predetermined phrase in the above input sentence
Information extraction method characterized by a step of extracting by separating certain each word. From wherein input <br/> sentence by arranging a plurality of predetermined words to an arbitrary position, an information extracting method for extracting words in between the predetermined words, arrangement and each of the plurality of predetermined phrases One or more between certain words
Storing in advance a plurality of fixed patterns consisting of information used to specify the delimiter of the phrase ; and inputting the input data from the fixed patterns stored in the step.
Matches or most matching sequence of each predetermined word sentences
Selecting a fixed pattern to be performed , and various points indicated by the fixed pattern selected in the step.
Each predetermined phrase in the above input sentence according to the list of fixed phrases and information
Information extraction method according to claim <br/> that a step of extracting separate each word in between. 3. The method according to claim 1 or 2,
The information extraction method according to either the predetermined word of the input sentence from the fixed pattern of the above-mentioned storage
Vinegar to select the most matching the fixed pattern to the list
In step, if the candidate of the fixed pattern that best match there is more <br/> number, for each fixed pattern, sequentially, each predetermined word of said fixed pattern is judged whether or not another of the fixed pattern, several of the count does not exist, the information extraction method of the largest fixed pattern the counted number and selects as the fixed pattern of the most matching.