JP6652355B2 - Information extraction device, method and program - Google Patents

Description

  An embodiment of the present invention relates to an information extraction device, a method, and a program.

With the growing momentum for utilizing big data, there is an increasing need to extract information desired by users from data. Since the information desired by the user is various, it is necessary to prepare an extraction rule for each type of information. Therefore, it is desired to create the extraction rules at low cost.
A method using machine learning can be used to create the extraction rules. For that purpose, it is necessary to create teacher information and design learning features (such as clue words). In designing learning features, it is often difficult for ordinary people who do not have knowledge of machine learning to determine what is a valid learning feature. There is a problem that it takes.

  As a method of finding a clue word at low cost, there is a technique of creating an extraction rule (feature, clue word) by combining words and evaluating the extraction rule to create an effective extraction rule.

JP 2010-262332 A

  However, when the learning feature is extracted by using the above-described technique, the evaluation of the extraction rule uses the bias of the portion where the extraction rule matches in the text. Therefore, evaluation of general-purpose features is high, and features unique to some fields are often not adopted (selected).

  The present disclosure has been made to solve the above-described problems, and has as its object to provide an information extraction device, a method, and a program that can extract an appropriate feature.

  The information extraction device according to the present embodiment includes a storage unit, an extraction unit, and an estimation unit. The storage unit stores a plurality of pieces of first teacher information each including target information to which information is to be added and a processing result of the worker with respect to the target information. The extracting unit extracts a plurality of second teacher information including the same target information and different processing results from the plurality of first teacher information. The estimating unit estimates an effective feature that is a clue to the different processing result from the difference between the plurality of pieces of second teacher information.

FIG. 1 is a block diagram illustrating an information extraction device according to a first embodiment. The figure which shows the specific example of the teacher information stored in a teacher information storage part. 9 is a flowchart illustrating an effective feature estimation process of the information extraction device. FIG. 6 is a block diagram showing an information extraction device according to a second embodiment. The figure which shows an example of object information. 9 is a flowchart illustrating a process of creating work information in a work information creation unit. The figure which shows an example of the information for work. FIG. 9 is a block diagram showing an information extraction device according to a third embodiment. The figure which shows an example of the work performed in the work part. The figure which shows the example of presentation of the effective feature in a working part.

  In the present embodiment, it is assumed that effective features are estimated based on teacher information created by crowdsourcing or the like.

  Hereinafter, the information extraction device, method, and program according to the present embodiment will be described in detail with reference to the drawings. In the following embodiments, portions denoted by the same reference numerals perform the same operation, and duplicate description will be omitted as appropriate.

(First embodiment)
The information extraction device according to the first embodiment will be described with reference to the block diagram of FIG.
The information extraction device 100 according to the first embodiment includes a teacher information storage unit 101, a teacher information extraction unit 102 (also referred to as a first extraction unit), and an effective feature estimation unit 103 (also referred to as a second extraction unit).

  The teacher information storage unit 101 receives and stores a plurality of teacher information (first teacher information) from outside. The teacher information is information including target information to which information is added by the worker and a processing result of the target information of the worker. A specific example of the teacher information will be described later with reference to FIG.

  The teacher information extraction unit 102 extracts a plurality of pieces of teacher information (second teacher information) including the same target information and different processing results from the plurality of pieces of teacher information stored in the teacher information storage unit 101.

  The effective feature estimating unit 103 receives a plurality of pieces of teacher information from the teacher information extracting unit 102. The effective feature estimating unit 103 estimates (extracts) an effective feature from a difference between a plurality of pieces of teacher information including the same target information and different processing results. The effective feature is information that is a clue to a worker's processing result and is an effective feature as a learning feature.

Next, a specific example of teacher information stored in the teacher information storage unit 101 will be described with reference to FIG.
In the table 200 shown in FIG. 2, a sentence ID 201, a sentence 202, a word string 203, a processing target ID 204, a presentation range 205, and a processing result 206 are associated with each other and stored as teacher information 210. In the present embodiment, it is assumed that the determination result of the named entity is used as teacher information.

  The sentence ID 201 is an identifier that uniquely identifies the sentence 202. The sentence 202 is the data body of the sentence. The word string 203 is a word constituting the sentence 202. Here, when the sentence 202 is divided for each word, the word string 203 indicates a set of words and an identifier for uniquely identifying the word. Note that the word string 203 may be a set of a morpheme obtained by morphologically analyzing a sentence and an identifier for uniquely identifying the obtained morpheme.

  The processing target ID 204 is a word to be processed (targeted for information addition) in the word string 203. Here, the identifier of the word string 203 is the processing target ID 204. The presentation range 205 is at least a portion including a word corresponding to the processing target ID 204 in the sentence 202, and is a range of a character string to be presented to an operator. Note that the presentation method of the presentation range 205 to the worker may be a method that can be perceived by the worker, such as displaying the character string of the presentation range 205 on the screen or reading out the character string of the presentation range 205 using synthesized voice. I just need. The processing result 206 is the result of the processing on the word corresponding to the processing target ID 204 of the worker.

  As an example of FIG. 2, as the teacher information 210, the sentence ID 201 “1”, the sentence 202 “Kawasaki is a friend, but that Kawasaki has recently been ...”, the word string 203 “1” : Kawasaki, 2: ..., 6: Friends, ..., 13: Part, 14: Kawasaki, ..., 19: Te, ... ", processing target ID 204" 14 ", presentation range 205 “6-19” and the processing result 206 “person name” are associated with each other.

  That is, in the teacher information 210, the 14th word “Kawasaki” of the sentence 202 is a word to be processed, and the presentation range 205 is a range of character strings from the 6th to the 19th. Is recently displayed to the worker, indicating that the worker has determined the proper expression that "Kawasaki" is a personal name.

Next, the effective feature estimating process of the information extracting device 100 will be described with reference to the flowchart of FIG.
In step S301, the teacher information extraction unit 102 acquires two pieces of teacher information having the same sentence ID 201 and the same processing target ID 204 from the teacher information storage unit 101 as the entry e1 and the entry e2.

  In step S302, the effective feature estimating unit 103 determines whether the processing result 206 of the entry e1 is different from the processing result 206 of the entry e2. If the processing results 206 are different, the process proceeds to step S303. If the processing results 206 are the same, the process ends.

  In step S303, the effective feature estimating unit 103 determines whether the presentation range 205 of the entry e1 and the presentation range 205 of the entry e2 are different presentation ranges. When the presentation ranges 205 are different, the process proceeds to step S304. When the presentation ranges 205 are not different, that is, when they are the same, the process proceeds to step S305.

  In step S304, the effective feature estimating unit 103 estimates an independent word as an effective feature from the difference range between the presentation range 205 of the entry e1 and the presentation range 205 of the entry e2. This is because the character string included in the difference of the presentation range 205 is considered to include information (a clue to a different processing result) that affects the processing result of the worker.

  In step S305, the effective feature estimating unit 103 determines that there is no effective feature. If step S303 is "No", that is, if the entry e1 and the entry e2 have the same presentation range 205 but different processing results 206, a simple mistake of the operator or It is determined that there is no effective feature because there is a high possibility that the writing is fluctuation.

  Thus, the estimation process of the effective feature estimation unit 103 ends. In FIG. 3, it is assumed that the teacher information extraction unit 102 extracts two pieces of teacher information having the same sentence ID 201 and the same processing target ID 204 from among a plurality of pieces of teacher information stored in the teacher information storage unit 101. Yes, but not limited to this. For example, when there are three or more pieces of teacher information having the same sentence ID 201 and the same processing target ID 204 (there are three or more entries), FIG. 3 shows each combination of two entries among the three or more entries. The process of estimating the effective feature may be performed. That is, when there are four entries and two entries are selected from the four entries, there are six combinations, and the effective feature estimation process may be performed for each combination.

Next, a specific example of the effective feature estimation processing will be described with reference to the teacher information shown in FIG.
The first teacher information (first entry) and the second teacher information (second entry) shown in FIG. 2 have the same sentence ID 201 “1” and the same processing target ID 204 “14”. On the other hand, regarding the processing result 206, the first entry is “person name” and the second entry is “place name”, which are different results. Further, while the first entry has the presentation range 205 of “6-19”, the second entry has the presentation range 205 of “8-19”, and the difference between the presentation range 205 of the first entry and the second entry is , “6-7”.

  In other words, the difference between the presentation range 205 of the first entry, "I'm at a friend, but that Kawasaki has recently been", and the presentation range 205 of the second entry, "I'm here, but that Kawasaki has recently been" The independent word included in the character string “to a friend” is estimated as an effective feature. Here, since the independent word is only "friend", "friend" is estimated as the effective feature.

  Note that the effective feature estimating unit 103 is not limited to estimating the independent word as the effective feature, and may estimate all character strings included in the difference of the presentation range 205 as effective features. Further, the effective feature estimating unit 103 may estimate a word limited by a part of speech such as “noun” or “verb” as an effective feature.

  Various methods can be considered as a method for determining an independent word and a method for determining a part of speech. For example, there is a method of referring to a word dictionary (not shown). A word dictionary including a correspondence between words and information on whether the words are independent words or a correspondence between words and parts of speech corresponding to the words is prepared in advance. By referring to the prepared word dictionary, the effective feature estimating unit 103 can determine an independent word or a part of speech.

  As another example of the part of speech determination method, the part of speech of each word in the sentence may be specified by performing morphological analysis on the sentence 202 included in the teacher information 210. In addition, the part-of-speech information of each word or information on whether or not the word is an independent word is previously included in the teacher information 210, and the effective feature estimation unit 103 includes Or information on whether the word is independent or not.

  Further, the effective feature estimating unit 103 may previously limit the candidates estimated as effective features. As a method of limiting the estimated candidates, among the words included in the difference of the presentation range between the teacher information, the position information of the word in the sentence, such as “first independent word”, or the relation between words is used. A receiving relationship may be used. Further, when nouns appear continuously in the range of the difference between the presentation ranges of the teacher information, the effective feature estimating unit 103 may collectively estimate the continuous nouns as effective features.

  According to the first embodiment described above, an appropriate feature can be estimated (extracted) by estimating the effective feature based on the difference in the presentation range of the teacher information. A learning feature can be easily designed without having to list features from scratch. Therefore, it is possible to reduce the time and cost for creating a learning feature. Further, since the effective feature is estimated from the teacher information that is the processing result of the worker, even a word unique in some fields can be used as the effective feature. As a result, effective learning features can be utilized, and highly accurate information extraction can be performed.

(Second embodiment)
The second embodiment is different from the first embodiment in that information that is a source for causing a worker to create teacher information is generated.

An information extraction device according to the second embodiment will be described with reference to the block diagram of FIG.
The information extraction device 400 according to the second embodiment includes a work information creation unit 401, an information output unit 402, a result acquisition unit 403, a teacher information storage unit 101, a teacher information extraction unit 102, and an effective feature estimation unit 103. The processing of the teacher information storage unit 101, the teacher information extraction unit 102, and the effective feature estimation unit 103 is the same as that of the first embodiment, and a description thereof will be omitted.

  The work information creating unit 401 acquires target information from outside. The work information creating unit 401 creates a plurality of pieces of work information having two or more different presentation ranges for presenting the target information to the worker. The target information and the work information will be described later with reference to FIGS. 5 and 6, respectively.

  The information output unit 402 acquires a plurality of pieces of work information from the work information creation unit 401 and outputs the plurality of pieces of work information to the outside. For example, a work is requested by transmitting (outputting) work information to a crowdsourcing system or the like.

  The result acquisition unit 403 acquires work information processed by the worker as teacher information. For example, teacher information is acquired as a processing result from the crowdsourcing system for which the work was requested.

Next, an example of the target information will be described with reference to FIG.
The target information is information in which the sentence ID 201, the sentence 202, the word string 203, and the processing target ID 204 are associated with each other. Note that, among the teacher information shown in FIG. 2, information having no items in the presentation range 205 and the processing result 206 is the target information.

Next, a process of creating work information in the work information creation unit 401 will be described with reference to the flowchart in FIG.
In step S601, the work information creating unit 401 acquires target information.

  In step S602, the work information creating unit 401 calculates, as “SENTOU”, a value obtained by subtracting N from the value of the processing target ID 204 in the target information. In addition, a value obtained by adding N to the value of the processing target ID 204 in the target information is calculated as “MATSBI”. Note that N is a predetermined positive integer, and the value of the integer corresponds to the number of words (here, the number of words).

  In step S603, the work information creating unit 401 decrements “SENTOU” by one.

  In step S604, the work information creating unit 401 determines whether “SENTOU” has reached the beginning of the sentence, in other words, whether the value of “SENTOU” is the same as the value of the processing target ID 204 corresponding to the beginning of letter. Is determined. If “SENTOU” has reached the beginning of the sentence, the process ends. If “SENTOU” has not reached the beginning of the sentence, the process proceeds to step S605.

  In step S605, the work information creation unit 401 sets the presentation range from “SENTOU” to “MATSUBI”, and associates the presentation range 205 with the target information to create work information. Thereafter, the process returns to step S603, and the same processing is repeated. This completes the work information creation process of the work information creation unit 401. Thereby, a plurality of pieces of work information having different presentation ranges can be created from one piece of target information.

Next, an example of the work information will be described with reference to FIG.
The work information is information in which the sentence ID 201, the sentence 202, the word string 203, the processing target ID 204, and the presentation range 205 are associated with each other. It should be noted that, among the teacher information, information having no item of the processing result 206 is the work information.

  The method of determining the presentation range shown in the flowchart of FIG. 6 shows a case where the presentation range is extended one word at a time toward the front of the word with reference to the word to be processed. However, the present invention is not limited to this. A method of extending the presentation range may be used. Alternatively, a method of extending the presentation range by subtracting and adding one word at a time alternately or simultaneously with the front and back may be used.

  The presentation range may be extended one character at a time instead of one word at a time, or the presentation range may be extended at a phrase unit. Alternatively, a method of extending the presentation range to the previous independent word may be used.

  Furthermore, the present invention is not limited to a method of determining a presentation range according to a distance between words based on a word to be processed, a method of determining a word to be displayed at random, and a syntax based on a sentence tree created by parsing a sentence. A method may be used in which the presentation range is expanded in order from the word closest to the word to be processed on the tree.

  According to the second embodiment described above, it is possible to determine the presentation range to be determined, create work information, and obtain teacher information based on the created work information. Further, as in the first embodiment, by estimating the effective feature based on the difference in the presentation range of the teacher information, the time and cost for creating the learning feature can be reduced, and an appropriate feature can be estimated (extracted). . Furthermore, effective learning features can be utilized, and highly accurate information extraction can be performed.

(Third embodiment)
The third embodiment is different from the above-described embodiment in that a worker process (for example, data input from a worker) for work information is received.

An information extraction device according to the third embodiment will be described with reference to the block diagram of FIG.
The information extraction device 800 according to the third embodiment includes a work information creation unit 401, an information output unit 402, a result acquisition unit 403, a teacher information storage unit 101, a teacher information extraction unit 102, an effective feature estimation unit 103, and a work unit. 801.

  Operations other than the operation unit 801 are the same as those in the second embodiment, and a description thereof will not be repeated.

  The work unit 801 receives a plurality of pieces of work information from the information output unit 402 and presents a plurality of pieces of work information to a worker. The work unit 801 receives a process for work information, for example, data input from a worker as a process result, and creates a plurality of teacher information based on the process result. Further, the working unit 801 receives the effective features estimated by the effective feature estimating unit 103 at the subsequent stage, and presents the effective features to the worker.

Next, an example of an operation performed by the operation unit 801 will be described with reference to FIG.
FIG. 9 is a display example of the work unit 801 in a case where the information adding (judgment) work is performed by the crowdsourcing system, in which the work information of the first entry in FIG. 7 is displayed. When the information adding operation is named entity extraction, it is general that the type of answer of the adding operation is predetermined.

  A question is displayed to the operator asking "Question: What is the category of the underlined word in the following sentence?", And the message "I have a friend, but recently Kawasaki" corresponding to the presentation range 205 Is displayed. Here, “person name”, “place name”, “organization name”, “others” and the like are displayed in advance as options. However, it is not always necessary to be an option type, and a free description type may be used. When the user inputs an answer to the displayed work screen, the working unit 801 generates a processing result based on the user's answer, and finally, teacher information is generated. Here, by checking the "person name" by the operator, teacher information such as the first entry shown in FIG. 2 can be obtained.

Next, an example of presenting effective features in the work unit 801 will be described with reference to FIG.
As illustrated in FIG. 10, the work unit 801 may prompt the worker to confirm the effective feature estimated by the effective feature estimation unit 103. As the timing of validity feature confirmation, for example, assuming that the worker checks the "person name" and then executes the validity feature estimation process, the confirmation window 1001 displays the message "Was the word" friend "helpful in making a decision?""?" Is displayed.

  When the operator checks “Yes” in the confirmation window 1001, the effective feature estimating unit 103 determines that the estimation of the effective feature is correct, and determines the certainty of the presented effective feature (here, “friend”). May be increased.

  By associating the effective feature with the certainty factor, a learning feature with a high certainty factor can be used when using the effective feature generated by the effective feature estimation process of the present embodiment in another application or the like. .

  According to the third embodiment described above, the work unit can present the work information to the worker to perform the work. By the working unit presenting the effective features to the worker, the accuracy of the effective features can be improved by learning, and a highly accurate learning feature can be presented. Further, similarly to the first embodiment, it is possible to reduce the learning feature creation time and creation cost, and to estimate an appropriate feature (extraction). Furthermore, effective learning features can be utilized, and highly accurate information extraction can be performed.

In the above-described embodiment, an example has been described in which the determination result of the named entity is used as the teacher information. However, the present invention is not limited to this, and information extraction relating to a plurality of words such as extraction of a parent-child relationship can be similarly performed. The extraction of the parent-child relationship is a process of determining whether a parent-child relationship exists between the personal name A and the personal name B.
For example, in the sentence "Mr. A is born as the first child of B ...", if the presentation range is "Mr. A is the first child of Ms. B", the worker determines that there is a parent-child relationship. You. On the other hand, when the presentation range is “Mr. A is B” in the same sentence, the worker determines that “the parent-child relationship is not” or “the parent-child relationship is unknown”. When these determination results are used as teacher information, the effective feature estimating unit 103 can estimate “first child” as the effective feature from the difference in the presentation range.

Further, in the above-described embodiment, the process of estimating the effective feature of a sentence has been described as an example of the information extraction device. However, the present invention is not limited to this. You may give. As the presentation range, a plurality of ranges of the face image to be displayed to the worker are prepared, such as a partial image of "eyes and nose" and a partial image of "eyes, nose and mouth". As a result, it is possible to handle a plurality of pieces of work information having different presentation ranges in the same manner.
For example, the gender determination result of the teacher information whose presentation range is a partial image of “eyes and nose” is “male”, and the gender determination result of the teacher information whose presentation range is a partial image of “eyes, nose and mouth” is “female” , It can be determined that the image of the “mouth”, which is the difference between the presentation ranges, is an effective feature in the gender determination.

The instructions shown in the processing procedure shown in the above-described embodiment can be executed based on a program that is software. A general-purpose computer system stores this program in advance, and by reading this program, it is also possible to obtain the same effects as those of the above-described information extraction device. The instructions described in the above-described embodiments are, as programs that can be executed by a computer, magnetic disks (flexible disks, hard disks, etc.) and optical disks (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD ± R, DVD ± RW, Blu-ray (registered trademark) Disc, etc.), a semiconductor memory, or a recording medium similar thereto. As long as the recording medium is readable by a computer or an embedded system, its storage format may be any. If the computer reads the program from the recording medium and causes the CPU to execute the instructions described in the program based on the program, the same operation as the information extraction device of the above-described embodiment can be realized. Of course, when the computer acquires or reads the program, the program may be acquired or read through a network.
Also, an OS (Operating System) running on the computer, database management software, MW (Middleware) such as a network, etc. realize the present embodiment based on instructions of a program installed in the computer or the embedded system from the recording medium. May be executed.
Further, the recording medium in the present embodiment is not limited to a medium independent of a computer or an embedded system, but also includes a recording medium in which a program transmitted through a LAN, the Internet, or the like is downloaded and stored or temporarily stored.
Further, the number of recording media is not limited to one, and a case where the processing in the present embodiment is executed from a plurality of media is also included in the recording medium in the present embodiment, and the configuration of the medium may be any configuration.

Note that the computer or the embedded system according to the present embodiment is for executing each process according to the present embodiment based on a program stored in a recording medium, and includes an apparatus including one such as a personal computer and a microcomputer. This device may have any configuration such as a system in which the devices are connected to a network.
Further, the computer in the present embodiment is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, and the like included in an information processing device, and is a general term for devices and devices that can realize the functions in the present embodiment by programs. ing.

  Although several embodiments of the present invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

100, 400, 800 ... information extraction device, 101 ... teacher information storage unit, 102 ... teacher information extraction unit, 103 ... effective feature estimation unit, 200 ... table, 201 ... sentence ID, 202 ... sentence, 203 ... word string, 204 ... processing target ID, 205 ... presentation range, 206 ... processing result, 210 ... teacher information, 401 ... work Information creation unit, 402: information output unit, 403: result acquisition unit, 801: working unit, 1001: confirmation window.

Claims (7)

A plurality of first information items each including target information to be given information, a processing result of the worker for the target information, and a presentation range for presenting to the worker at least a part of the target information; A storage unit for storing teacher information,
A first extraction unit configured to extract a plurality of second teacher information including the same target information and different processing results from the plurality of first teacher information;
A second extraction unit configured to extract, from a difference between the presentation ranges included in each of the plurality of pieces of second teacher information, clues that lead to the different processing results. The target information includes a sentence, a word string indicating a word constituting the sentence, and a processing target indicating a word to be processed in the word string,
2. The plurality of first teacher information and the plurality of second teacher information are at least a part of the sentence including the processing target, and further include a presentation range for presenting to the worker. Information extraction device. A creating unit that creates a plurality of different presentation ranges from one target information, and creates a plurality of work information items in which each of the presentation ranges is associated with the one target information;
An output unit that outputs the plurality of work information;
2. The acquisition device according to claim 1, further comprising: an acquisition unit configured to associate the plurality of pieces of work information with a processing result of the worker with respect to the plurality of pieces of work information and acquire the plurality of pieces of first teacher information. 3. The information extraction device according to 2.   The information extraction device according to claim 3, further comprising a work unit configured to receive a process for the plurality of pieces of work information from the worker.   The information extraction device according to any one of claims 2 to 4, wherein the presentation range is a range extended in at least one of a forward direction and a backward direction on a word or phrase basis with respect to the target information. A plurality of first information items each including target information to be given information, a processing result of the worker for the target information, and a presentation range for presenting to the worker at least a part of the target information; teacher information or et al., the first extraction unit extracts the plurality of second instruction information including the same object information and different processing results,
An information extraction method , wherein a second extraction unit extracts information that is a key to the different processing result from a difference between the presentation ranges included in each of the plurality of pieces of second teacher information. Computer
A plurality of first information items each including target information to be given information, a processing result of the worker for the target information, and a presentation range for presenting to the worker at least a part of the target information; Storage means for storing teacher information;
First extraction means for extracting, from the plurality of first teacher information, a plurality of second teacher information including the same target information and different processing results;
An information extraction program for functioning as second extraction means for extracting information serving as clues leading to the different processing results from a difference between the presentation ranges included in each of the plurality of pieces of second teacher information.