JP2018124617A - Teacher data collection apparatus, teacher data collection method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To collect high quality teacher data on machine learning at low cost.SOLUTION: A teacher data collection apparatus for collecting a data relating to a certain field used for a teacher data of machine learning, the teacher data collection apparatus that includes a feature calculation unit for calculating a first feature vector that is a feature vector of a reference data relating to the certain field registered in advance, a generation unit for generating a search condition used for collecting the data relating the certain field from the first feature vector, a collection unit for collecting the data relating the certain field based on the generated search condition, a similarity calculation unit for calculating a similarity between a second feature vector and the first feature vector after calculating the second feature vector that is a feature vector of the collected data by the feature calculation unit, and an extraction unit for extracting the collected data whose similarity is within the predetermined range as the teacher data is provided.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a natural language analysis technique, and more particularly to a teacher data collection device, a teacher data collection method, and a program that automatically acquire important teacher data in supervised learning.

  In the field of information extraction, a machine learning method is generally used in many cases. Artificial intelligence machine learning learning methods, such as deep learning, include “supervised learning” in which the system teaches the relationship between the input data and the output data, and the system itself uses only the input data. It can be broadly divided into “unsupervised learning” in which a tendency is derived or a large amount of data is classified into small amounts of data.

  With “supervised learning”, if there is a large amount of teacher data indicating an input / output relationship, the interrelationship of any information can be learned. However, there is a problem in that the creation of teacher data requires manpower and is costly.

  On the other hand, “unsupervised learning” has an advantage that the learning cost of the system can be reduced, but it can be applied only to work that can be realized without knowing the correct answer.

  As a technique for reducing the teacher data creation cost of “supervised learning”, there is a bootstrap method that is semi-supervised learning. In the bootstrap method, a large amount of teacher data is created by extracting data that conforms to the rule based on a small number of teacher data given as input first and adding it to the teacher data (for example, Patent Document 1). reference).

  On the other hand, a technique is also being studied that enables the creation of an associative concept dictionary based on high-accuracy associative words suitable for disambiguation of ambiguous words, and that collects learning data according to parameters such as the number of cases and quality policy ( For example, see Patent Document 2).

JP 2005-222532 A JP 2011-164717 A

However, in the bootstrap method as shown in Patent Document 1, even if inappropriate data is used, it is added as new teacher data if it conforms to the rules based on the first teacher data. Will contain a lot of inappropriate data.
In addition, in the learning data collection of Patent Document 2, since learning data is collected from a corpus in which sentence data is randomly collected, even if the selection accuracy is improved using an associative concept dictionary, there is a bias in the field of collected data in the first place. Cannot be considered. Therefore, a data set depending on the population is obtained.

  In view of the above, an object of the present invention is to provide a teacher data collection device, a teacher data collection method, and a program that can collect high-quality teacher data related to machine learning at a low cost.

  In order to achieve the above object, the present invention is a teacher data collection device that collects data related to a specific field for use as machine learning teacher data, and has been registered in advance as reference data related to a specific field. A feature calculation unit that calculates a first feature vector that is a feature vector of the target, a generation unit that generates a search condition used to collect data related to the specific field from the first feature vector, and the generated search Based on the condition, when the feature calculation unit calculates a second feature vector that is a feature vector of the collected data and a collection unit that collects data related to the specific field, the second feature vector, A similarity calculation unit that calculates a similarity with the first feature vector, and the collected data with the similarity within a predetermined range are extracted as the teacher data. An extraction unit which provides training data collection device comprising a.

  It is possible to automatically collect high-quality teacher data related to machine learning that is specialized for collecting information related to a specific field at a low cost.

It is a figure which shows an example of the system configuration | structure of the teacher data collection system which concerns on embodiment. It is a figure which shows an example of the hardware constitutions of the teacher data collection system which concerns on embodiment. It is a figure which shows an example of the various tables memorize | stored with the memory | storage device which concerns on embodiment. It is a flowchart which shows an example of the flow of a process of calculation of the feature vector which concerns on embodiment. It is a figure which shows the specific example of the calculation process of the feature vector which concerns on embodiment. It is a flowchart which shows an example of the flow of the data collection process regarding the specific field | area concerning embodiment. It is a figure which shows the specific example of the collection process of the data relevant to the specific field | area concerning embodiment. It is a flowchart which shows an example of the flow of a process which calculates the similarity degree of the feature vector which concerns on embodiment. It is a figure which shows the specific example of the process which calculates the similarity which concerns on embodiment. It is a flowchart which shows an example of the flow of the process which extracts the data stored as teacher data which concern on embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(System configuration)
FIG. 1 is a diagram illustrating an example of a system configuration of a teacher data collection system according to an embodiment of the present invention.

The teacher data collection system 100 extracts only data related to a specific field from a large amount of information (data).
According to FIG. 1, the teacher data collection system 100 manages a database on the storage device 300, a storage device 300 that stores reference data related to a specific field, feature vectors of the reference data, and the like in a database. It includes a management computer 200 (teacher data collection device) that enables database search and the like.

  A storage device 300 is connected to the management computer 200.

  Furthermore, the management computer 200 is connected to the terminal 400 and an external document database 410 via a network N such as the Internet. The terminal 400 can access the management computer 200. For example, the user can check the teacher data collected by the management computer 200 from the terminal 400. Further, the management computer 200 can access an external document database 410. For example, the management computer 200 can capture data stored in the external document database 410.

  The management computer 200 includes a data reception unit 210, a feature vector calculation unit 220, a search condition generation unit 230, a data collection unit 240, and a similarity calculation unit 250.

  The data reception unit 210 receives data (reference data) related to a specific field selected by the user using various input means such as a mouse, a keyboard, and a touch panel. The data receiving unit 210 stores the received reference data in the reference data storage unit 310 of the storage device 300.

  The feature vector calculation unit 220 (feature calculation unit) calculates a first feature vector, which is a feature vector related to the reference data, from the reference data stored in the reference data storage unit 310 of the storage device 300, and stores the storage device 300. Is stored in the reference data storage unit 310.

  The search condition generation unit 230 (generation unit) generates a search condition for collecting data from the first feature vector stored in the reference data storage unit 310 of the storage device 300 and outputs the search condition to the data collection unit 240. To do.

  The data collection unit 240 (collection unit) collects data (collection data) that matches the search condition from the document database 410 based on the search condition generated by the search condition generation unit 230 and collects it in the storage device 300. The data is stored in the data storage unit 320.

  The feature vector calculation unit 220 calculates a second feature vector that is a feature vector related to the collected data from the collected data stored in the collected data storage unit 320 of the storage device 300, and the collected data storage unit of the storage device 300 Stored in 320.

  The similarity calculation unit 250 compares the second feature vector stored in the collected data storage unit 320 of the storage device 300 with the first feature vector stored in the reference data storage unit of the storage device 300, The similarity of the second feature vector to the first feature vector is calculated. The similarity calculation unit 250 stores the calculated similarity in the collected data storage unit 320 of the storage device 300.

  The teacher data extraction unit 260 (extraction unit) extracts the collected data stored in the collected data storage unit 320 of the storage device 300 and having a similarity within a predetermined range as teacher data, and stores the teacher data in the storage device 300. Stored in the unit 330. In addition, the teacher data extraction unit 260 stores the reference data stored in the reference data storage unit 310 of the storage device 300 in the teacher data storage unit 330 of the storage device 300.

(Hardware configuration)
The management computer 200 according to the embodiment can be realized using, for example, a general computer 500. FIG. 2 is a diagram illustrating an example of the configuration of the computer 500.

  The computer 500 includes a CPU (Central Processing Unit) 501, a RAM (Random Access Memory) 502, a ROM (Read Only Memory) 503, a storage device 504, an external I / F (Interface) 505, an input device 506, an output device 507, a communication I / F508 and the like. These devices transmit and receive signals to and from each other via the bus B.

The CPU 501 is an arithmetic device that implements each function of the computer 500 by reading out programs and data stored in the ROM 503, the storage device 504, and the like onto the RAM 502 and executing the processing. A RAM 502 is a volatile memory used as a work area for the CPU 501. The ROM 503 is a nonvolatile memory that retains programs and data even when the power is turned off.
The storage device 504 is realized by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, and stores an OS (Operation System), application programs, various data, and the like.
The external I / F 505 is an interface with an external device. Examples of the external device include a recording medium 509. The computer 500 can read and write the recording medium 509 via the external I / F 505. The recording medium 509 includes, for example, an optical disk, a magnetic disk, a memory card, a USB (Universal Serial Bus) memory, and the like.

  The input device 506 includes, for example, a mouse, a touch panel, and a keyboard, and inputs various operations and the like to the computer 500 in response to instructions from an operator (user).

  The output device 507 is realized by a liquid crystal display, for example, and displays a processing result by the CPU 501.

  The communication I / F 508 is an interface that connects the computer 500 to a network such as the Internet (such as the network N in FIG. 1) by wired communication or wireless communication. The bus B is connected to each of the above constituent devices, and transmits and receives various control signals and the like between the control devices.

(Description of various tables)
Next, each table stored in the storage device 300 will be described with reference to FIG.

FIG. 3 is a diagram illustrating an example of various tables stored in the storage device according to the embodiment of the present invention.
The reference data management table 600 stored in the reference data storage unit 310 of the storage device 300 shown in FIG. 3A includes reference data relating to a specific field, a data identifier assigned to the reference data, and characteristics of the data It is a table stored in association with a vector (first feature vector).

  For example, in the data on the first line in FIG. 3A, the data identifier of the reference data “DA00001” relating to the specific field is “# A00001”, and the feature vector “XA00001” relating to the reference data “DA00001” is the feature vector. The first feature vector calculated by the calculation unit 220 is indicated.

  The feature vector calculation unit 220 is a first feature that is a feature vector related to reference data from the reference data “DA00001” related to a specific field that is input by the user to the terminal 400 and stored in the reference data storage unit 310 of the storage device 300. The vector “XA00001” is calculated and stored in the reference data storage unit 310 of the storage device 300. As a feature vector calculation method, for example, a TF (Term Frequency) / IDF (Inverse Document Frequency) method may be used.

  As described above, the reference data management table 600 of the reference data storage unit 310 stores feature vectors for reference data related to a specific field input by the user.

  The collected data management table 610 stored in the collected data storage unit 320 of the storage device 300 shown in FIG. 3B includes data collected about a specific field (collected data), a data identifier assigned to the data, It is a table stored in association with the feature vector of data (second feature vector) and the similarity of the data to the reference data.

  For example, the data in the first row in FIG. 3B relates to a specific field collected by the data collection unit 240 based on a search condition for collecting data regarding a specific field generated by the search condition generation unit 230. The data identifier of the collected data “DS00001” is “# S00001”, and the feature vector “XS00001” related to the collected data “DS00001” is the second feature vector calculated by the feature vector calculation unit 220 and relates to a specific field. The similarity of the feature vector (second feature vector) “XS00001” of the collected data “DS00001” with respect to the feature vector (first feature vector) of the reference data is 0.634.

  The search condition generation unit 230 generates a search condition (a combination of search terms) for collecting data from the first feature vector stored in the reference data storage unit 310 of the storage device 300, and the data collection unit 240. Output to.

  The data collection unit 240 searches the document database 410 based on the search conditions (such as combinations of search terms) generated by the search condition generation unit 230 using, for example, a search engine that is generally available on the web. Collect data that meets your requirements. Then, the data collection unit 240 stores the collected data (data suitable for the search condition) in the collected data storage unit 320 of the storage device 300 as the collected data “DS00001” regarding the specific field.

  The feature vector calculation unit 220 calculates a second feature vector “XS00001” that is a feature vector related to the collected data from the collected data “DS00001” stored in the collected data storage unit 320 of the storage device 300. Stored in the collected data storage unit 320.

The similarity calculation unit 250 uses the second feature vector “XS00001” stored in the collected data storage unit 320 of the storage device 300 as the first feature vector “X00001” stored in the reference data storage unit of the storage device 300. Compared with “XA00001”, “XA00002”, “XA00003”, etc., the similarity (here, 0.634) of the second feature vector with respect to the first feature vector is calculated.
Specifically, the similarity calculation unit 250 stores the second feature vector “XS00001” stored in the collected data storage unit 320 of the storage device 300 in the reference data storage unit of the storage device 300. Compared with the first feature vector “XA00001”, the similarity of the second feature vector “XS00001” with respect to the first feature vector “XA00001” is calculated. The same applies to “XA00002”, “XA00003”, and the like.
Next, the similarity calculation unit 250 combines the similarity of the second feature vector “XS00001” with respect to the calculated first feature vectors “XA00001”, “XA00002”, “XA00003”, etc. The similarity of the second feature vector with respect to is calculated. Examples of the combination method include an average value of similarity and a maximum value of similarity. The similarity calculation unit 250 stores the calculated similarity (here, 0.634) in the collected data storage unit 320 of the storage device 300.

  As described above, the collected data management table 610 of the collected data storage unit 320 includes collected data related to a specific field, a feature vector (second feature vector) for the collected data, and a feature vector of reference data related to the specific field. The similarity of the feature vector (second feature vector) for the collected data with respect to (first feature vector) is stored.

  The teacher data management table 620 stored in the teacher data storage unit 330 of the storage device 300 illustrated in FIG. 3C is a table that stores teacher data in association with a data identifier assigned to the data.

  For example, the data in the first row in FIG. 3C indicates that the data identifier of the collected data “DS00003” extracted as the teacher data by the teacher data extraction unit 260 is “# S00003”.

  The teacher data extraction unit 260 extracts the collected data “DS00003” stored in the collected data storage unit 320 of the storage device 300 within the predetermined range as the teacher data, and the data identifier of the collected data “DS00003” Is stored in the teacher data storage unit 330 of the storage device 300 together with “# S00003”.

  Further, as shown in FIG. 3C, the teacher data extraction unit 260 converts the reference data (data manually selected by the user) stored in the reference data storage unit 310 of the storage device 300 into the storage device 300. Is stored in the teacher data storage unit 330.

  Thus, teacher data is stored in the teacher data management table 620 of the collected data storage unit 320.

(Process flow)
Next, the feature vector calculation process will be described with reference to FIGS. 4 and 5, taking the configuration of FIG. 1 and the various tables of FIG. 3 as examples.

FIG. 4 is a flowchart illustrating an example of a flow of feature vector calculation processing according to the embodiment of the present invention.
FIG. 5 is a diagram showing a specific example of the feature vector calculation process according to the embodiment of the present invention.

  As a premise, it is assumed that the user inputs reference data “DA00001”, “DA00002”, “DA00003” (see FIG. 3A) related to a specific field determined and selected by the user himself / herself to the terminal 400. Then, the terminal 400 transmits the three reference data to the management computer 200. In the management computer 200, the data receiving unit 210 receives these reference data and stores them in the storage device 300. More specifically, the data reception unit 210 stores the reference data “DA00001” in the reference data management table 600 in association with the data identifier “# A00001”. Similarly, the data reception unit 210 stores the reference data “DA00002” and “DA00003” in the reference data management table 600 in association with each data identifier. Note that the data identifier of the reference data may be calculated by the data reception unit 210, or may be calculated by a database system or the like that operates on the storage device 300. When the reference data is stored in the storage device 300 in this way, for example, the management computer 200 starts a feature vector calculation process of the reference data according to an instruction from the user.

  Here, as an example of the present embodiment, the “specific field” is a field related to the “railway system”. Further, reference data “DA00001”, “DA00002”, “DA00003”, etc. are collected (selected) from a web or the like by a user's own judgment, and are related to documents A1, A2, A3,. ). The documents related to the railway system are, for example, “news about railway construction contracts”, “technical papers on railways” and the like.

  First, the feature vector calculation unit 220 takes in reference data from the storage device 300 (step S101). More specifically, the feature vector calculation unit 220 includes three pieces of reference data “DA00001” and “DA00002” stored in the reference data storage unit 310 of the storage device 300 (reference data management table 600 in FIG. 3A). ”And“ DA00003 ”are read and loaded.

  Next, the feature vector calculation unit 220 uses the feature vectors (first feature vectors) related to the reference data “DA00001,” “DA00002,” and “DA00003” from the fetched reference data “DA00001,” “DA00002,” “DA00003”. ) Is calculated (step S102). For example, the feature vector calculation unit 220 calculates a feature vector (first feature vector) “XA00001” for the reference data “DA00001”.

  If the reference data “DA00001” shown in FIG. 3A is the document A1 shown in FIG. 5, the feature vector (first feature vector) “XA00001” of the reference data “DA00001” is stored in the document A1. It is expressed by a combination of the included word i (word i) and its weight value. Here, the “weight value” is the degree to which each word i contributes to the characterization of the feature vector. In this embodiment, for example, the “weight value” is expressed by “number of appearances” of each word i. The word i is a group of nouns automatically extracted from the document A1 by the feature vector calculation unit 220, such as “traffic”, “train”, “railway”, “government”, and the like. In the example shown in FIG. 5, the feature vector (first feature vector) “XA00001” for the reference data “DA00001” (document A1) is “traffic = 10, train = 4, railway = 7, governance = 2,.・ ”Etc.

The feature vector calculation unit 220 outputs the calculated first feature vector to the storage device 300 (step S103). More specifically, the feature vector calculation unit 220 stores the calculated feature vector (first feature vector) in the reference data storage unit 310 (reference data management table 600) of the storage device 300. For example, the feature vector calculation unit 220 stores the feature vector (first feature vector) “XA00001” in the reference data management table 600 in association with the reference data “DA00001”.
The feature vector calculation unit 220 similarly applies to the reference data “DA00002” and “DA00003” by the feature represented by the word i included in each of the document A2 and the document A3 and its weight value (for example, “appearance count”). The vectors “XA00002” and “XA00003” are calculated.

  Next, data collection processing related to a specific field will be described with reference to FIGS. 6 and 7 taking the configuration of FIG. 1 and various tables of FIG. 3 as examples.

FIG. 6 is a flowchart showing an example of a flow of data collection processing related to a specific field according to the embodiment of the present invention.
FIG. 7 is a diagram showing a specific example of data collection processing related to a specific field according to the embodiment of the present invention.

  As a premise, a plurality of reference data “DA00001” to “DA00003” and their respective feature vectors (first feature vectors) “XA00001” are stored in the reference data storage unit 310 of the storage device 300 by the processing described in FIG. “XA00003” and the like are stored. The user inputs data collection instruction information to the terminal 400.

  Then, the search condition generation unit 230 takes in a feature vector (first feature vector) of reference data related to a specific field stored in the reference data storage unit of the storage device 300 (step S201). For example, the feature vectors (first feature vectors) “XA00001” to “XA00003” of the reference data “DA00001” to “DA00003” are captured.

  Next, the search condition generation unit 230 generates a search condition for collecting data from the feature vectors (first feature vectors) “XA00001” to “XA00003” of the captured reference data (step S202). More specifically, the search condition generation unit 230 obtains a search word, a weighting coefficient for each search word, and a combination of search words from the feature vectors (first feature vectors) “XA00001” to “XA00003” of the captured reference data. Generate search conditions such as The search condition generation unit 230 outputs the generated search condition to the data collection unit 240 (step S203).

Here, an example in which the “search condition” is “a combination of search terms” will be described with reference to FIG.
First, for example, the search condition generation unit 230 calculates a weight value for each word i using the feature vector “XA00001” for the reference data “DA00001” (document A1). In this embodiment, the “weight value” here is, for example, the product (tf × idf) of the appearance frequency (tf: Term Frequency) of the word i and the reverse document appearance frequency (idf: Inverse Document Frequency). . According to the example shown in FIG. 7, the appearance frequency related to the word “traffic” of the document A1 is calculated as 0.333, and the reverse document appearance frequency is calculated as 0.812. The search condition generation unit 230 similarly calculates a weight value (for example, tf × idf) for each word i for the other reference data “DA00002” and “DA00003” (document A2, document A3).

Next, the search condition generation unit 230 extracts a word i having a large weight value (for example, tf × idf) common to all reference data (documents A1, A2, and A3). Specifically, the search condition generation unit 230 determines whether or not the average value of the weight values calculated for each of the documents A1, A2, and A3 is equal to or greater than a predetermined determination threshold, and is equal to or greater than the determination threshold. Extract word i. Then, the search condition generation unit 230 creates a search condition using the extracted words i as search words. In this way, a word i that specifically characterizes the reference data (documents A1, A2, and A3) (that is, a word i that appears frequently in the reference data) is extracted from the plurality of words i. The combination of the extracted word i becomes the search condition.
For example, it is assumed that the search condition generation unit 230 extracts three words i “traffic”, “train”, and “railway” from the calculation result of the average value of the weight values. In this case, the search condition generation unit 230 uses a combination of three search terms “traffic”, “train”, and “railway” as a search condition.

Next, the data collection unit 240 uses, for example, a search engine that is generally available on the web, and uses the search condition (combination of search terms) generated in step S203 as a search key from the external document database 410. Data is collected (step S204). According to the above example, the data collection unit 240 collects documents including all three search terms “traffic”, “train”, and “railway” through the search engine.
Here, it is assumed that the data collection unit 240 collects, for example, three data “DS00001”, “DS00002”, and “DS00003” (see FIG. 3B) as a result of the search processing using the above search conditions. .

  The data collection unit 240 outputs the data (collected data) “DS00001”, “DS00002”, and “DS00003” collected in step S204 to the storage device 300 (step S205). More specifically, the data collection unit 240 stores the collected data “DS00001”, “DS00002”, and “DS00003” in the collected data storage unit 320 (collected data management table 610) of the storage device 300. For example, the data collection unit 240 stores the collected data “DS00001” in the collected data management table 610 in association with the data identifier “# S00001”. The same applies to the collected data “DS00002” and “DS00003”.

  Next, the calculation process of the similarity will be described with reference to FIGS. 8 and 9, taking the configuration of FIG. 1 and various tables of FIG. 3 as examples.

FIG. 8 is a flowchart showing an example of the flow of processing for calculating the similarity according to the embodiment of the present invention.
FIG. 9 is a diagram illustrating a specific example of processing for calculating similarity according to the embodiment of the present invention.

  As a premise, the collection data storage unit 320 of the storage device 300 stores a plurality of collections searched by using the three search terms “traffic”, “train”, and “railway” by the processing described in FIG. Data “DS00001” to “DS00003” are stored. The user inputs similarity calculation instruction information to the terminal 400.

  Then, the feature vector calculation unit 220 calculates each feature vector (second feature vector) of the collected data “DS00001” to “DS00003” stored in the collected data storage unit 320 of the storage device 300 (step S300). ). The feature vector calculation unit 220 stores the second feature vector in the collected data storage unit 320 of the storage device 300.

Here, as a result of the search using the search condition (combination of three search terms “traffic”, “train”, and “railway”) generated by the search condition generation unit 230, three new search conditions shown in FIG. It is assumed that document X, document Y, and document Z (for example, “News of Railway Construction Plan in the United States”) are collected. The document X, the document Y, and the document Z are the collected data “DS00001”, “DS00002”, and “DS00003” illustrated in FIG.
In this case, the feature vector (second feature vector) “XS00001” of the collected data “DS00001” is expressed by a combination of the word i (word i) included in the document X and the number of appearances thereof. The word i is a noun group automatically extracted from the document X by the feature vector calculation unit 220. In the example shown in FIG. 9, the feature vector (second feature vector) “XS00001” for the collected data “DS00001” (document X) is (traffic = 14, train = 22, railway = 67, governance = 98,.・).

  Next, the similarity calculation unit 250 takes in the first feature vector (feature vector of reference data) from the storage device 300 (step S301). More specifically, the similarity calculation unit 250 takes in the first feature vector stored in the reference data storage unit 310 (reference data management table 600 shown in FIG. 3A) of the storage device 300.

  Next, the similarity calculation unit 250 takes in the second feature vector (feature vector of collected data) from the storage device 300 (step S302). More specifically, the similarity calculation unit 250 takes in the second feature vector stored in the collected data storage unit 320 (the collected data management table 610 shown in FIG. 3B) of the storage device 300.

  Next, the similarity calculation unit 250 compares the captured first feature vector with the second feature vector, and calculates the similarity of the collected data with respect to the reference data (step S303). The similarity calculation unit 250 stores the calculated similarity in the collected data storage unit 320 of the storage device 300 in association with the data identifier of the collected data (step S304).

Specifically, the similarity calculation unit 250, for example, the feature vector (second feature vector) “XS00001” of the collected data “DS00001” and the three reference data “DA00001”, “DA00002”, “DA00003”. The cosine similarity with each of the feature vectors (first feature vectors) “XA00001”, “XA00002”, and “XA00003” is calculated. Then, the similarity calculation unit 250 applies each of the three reference data feature vectors “XA00001”, “XA00002”, and “XA00003” to the feature vector (second feature vector) “XS00001” of the collected data “DS00001”. The average value (or maximum value, etc.) of the cosine similarity is specified, and the specified similarity is stored in the collected data storage unit 320 of the storage device 300.
The similarity calculation unit 250 similarly calculates the similarity for the collected data “DS00002” and “DS00003” and stores them in the collected data storage unit 320 of the storage device 300 (see FIG. 3B).
In the following description, the similarity calculation unit 250 continues to be described as calculating the “cosine similarity” between the first feature vector and the second feature vector. However, in other embodiments, the embodiment is limited to this mode. Not. The similarity calculation unit 250 according to another embodiment may calculate the similarity based on the “Euclidean distance” between the first feature vector and the second feature vector, for example.

  Next, teacher data extraction processing will be described with reference to FIG. 10, taking the configuration of FIG. 1 and various tables of FIG. 3 as examples.

  FIG. 10 is a flowchart illustrating an example of a flow of extracting data to be stored as teacher data according to the embodiment of the present invention.

  The teacher data extraction unit 260 takes in the collected data similarity (cosine similarity) from the storage device 300 (step S401). More specifically, the teacher data extraction unit 260 takes in the cosine similarity stored in the collected data storage unit 320 (the collected data management table 610 shown in FIG. 3B) of the storage device 300.

Next, the teacher data extraction unit 260 determines whether or not the captured cosine similarity is within a predetermined range (step S402). For example, the teacher data extraction unit 260 determines whether or not the cosine similarity is a certain numerical value or more. The teacher data extraction unit 260 determines the similarity of all the collected data stored in the collected data storage unit 320 of the storage device 300 (the collected data management table 610 in FIG. 3B). Further, the teacher data extraction unit 260 extracts teacher data candidates based on the determination result, and extracts the collected data into the teacher data storage unit 330 of the storage device 300 (teacher data management table 620 shown in FIG. 3C). (Step S403).
For example, as shown in FIG. 3B, the cosine similarity cos θx for the collected data “DS00001” (document X) is 0.634, and the cosine similarity cos θy for the collected data “DS00002” (document Y) is 0. 945, and the cosine similarity cos θz for the collected data “DS00003” (document Z) is 0.803. In this case, the teacher data extraction unit 260 determines whether or not the cosine similarity for each of the collected data “DS00001”, “DS00002”, and “DS00003” is equal to or greater than a predetermined determination threshold (for example, 0.9). To do. Then, the teacher data extraction unit 260 extracts the collected data “DS00002” (document Y) that is equal to or higher than the determination threshold as a new teacher data candidate, and the extracted collected data “DS00002” is the teacher data storage unit of the storage device 300. To 330.
As described above, the teacher data management table 620 (FIG. 3C) includes a reference data feature vector (the first data) among a plurality of automatically collected data “DS00001”, “DS00002”, and “DS00003”. Only data (documents) having a feature vector (second feature vector) similar to (one feature vector) is registered. The teacher data extraction unit 260 may also register the reference data “DA00001”, “DA00002”, and “DA00003” itself selected by the user's judgment in the teacher data management table 620.

(Action / Effect)
As described above, the management computer 200 of the teacher data collection system 100 according to the present embodiment collects data (documents) relating to a specific field (for example, “railway system”) to be used as machine learning teacher data. It is a teacher data collection device.
The management computer 200 includes a feature vector calculation unit 220 that calculates a first feature vector that is a feature vector of data (reference data) related to a specific field that has been registered in advance, and a specific field from the first feature vector. Search condition generation unit 230 that generates search conditions (such as combinations of search terms) used to collect data related to data, and a data collection unit that collects data (collection data) related to a specific field based on the generated search conditions 240, when the feature vector calculation unit 220 calculates the second feature vector that is the feature vector of the collected data, a similarity calculation unit 250 that calculates the similarity between the second feature vector and the first feature vector; And a teacher data extraction unit 260 that extracts the collected data having the similarity in a predetermined range as teacher data.
According to such a configuration, first, data (documents) that the user himself / herself determines to be suitable for “teacher data” is registered in advance, and reference data (documents A1, A2) relating to a specific field. ,...) Feature vector (first feature vector) is calculated. Then, based on the feature vector (first feature vector) of the reference data, a search condition (a combination of search terms) for automatically collecting new teacher data is generated. There is a high possibility that the data (document) automatically collected based on the search condition generated from the first feature vector has a feature vector similar to the first feature vector. That is, the data collected in this way (collected data) is highly likely to have characteristics close to those of the reference data. Therefore, it is possible to automatically collect data (documents) suitable for teacher data for learning about a “specific field” with a certain high probability.
However, depending on the process of automatic collection (search), data having a feature vector that is not similar to the feature vector (first feature vector) of the reference data (that is, a document that does not belong to a specific field) happens to be the search condition. There is a possibility that it will be collected in accordance with If such data is mixed into teacher data, the reliability of machine learning for a “specific field” is reduced. Therefore, the management computer 200 further calculates a feature vector (second feature vector) for the automatically collected data (collected data) based on the search condition, and the first and second feature vectors. The similarity with the feature vector is calculated. Then, the management computer 200 takes in only the collected data whose similarity is a predetermined value or more as teacher data.
In this way, data that does not belong to a “specific field” in the collected data (data that happens to meet the search conditions) is excluded, and data that is truly suitable as teacher data is imported as teacher data. Can do.
As described above, according to the teacher data collection system 100 according to the present embodiment, it is possible to automatically collect high-quality teacher data related to machine learning, which is specialized for information collection regarding a specific field, at low cost. Become.

In addition, according to the management computer 200 according to the present embodiment, the search condition generation unit 230 generates, based on the first feature vector, a combination of words whose degree used for reference data is a predetermined value or more as a search condition. To do.
In this way, new data is collected using a combination of words (word i) that appears particularly frequently in the reference data as a search key. Therefore, the collected data has characteristics similar to the reference data. The probability of being present can be increased.

Further, according to the management computer 200 according to another embodiment, the data collection unit 240 has a weight value (for example, appearance frequency) for each word i included in data (reference data) related to a specific field registered in advance. , Tf × idf) is a combination of words with a predetermined value or more as a search condition.
The accuracy of collecting data having the same characteristics as the reference data (documents A1, A2, and A3) can be further improved, for example, by considering the document structure itself by devising the weight value calculation method.
In the above-described embodiment, the “weight value” has been described as “appearance count” and “tf × idf”, but is not limited to this aspect in other embodiments. For example, in another embodiment, the “weight value” may be “appearance frequency (tf)” or “reverse document appearance frequency (idf)”.

As described above, according to the teacher data collection device, the teacher data collection method, and the program of the present embodiment, high-quality teacher data related to machine learning specialized for information collection related to a specific field can be obtained at low cost. Can be collected automatically.
Each process in the management computer 200 described above is stored in a computer-readable recording medium in the form of a program, and the above processing is performed by the computer of the management computer 200 reading and executing this program. Is called. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.
Moreover, the management computer 200 may be comprised by one computer, and may be comprised by the some computer connected so that communication was possible.

  In addition, it is possible to appropriately replace the components in the above-described embodiments with known components without departing from the spirit of the present invention. The technical scope of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

100 teacher data collection system 200 management computer (teacher data collection device)
210 Data reception unit 220 Feature vector calculation unit (feature calculation unit)
230 Search condition generator (generator)
240 Data collection unit (collection unit)
250 Similarity calculation unit 260 Teacher data extraction unit (extraction unit)
300 storage device 310 reference data storage unit 320 collection data storage unit 330 teacher data storage unit 400 terminal 410 document database 500 general computer 600 reference data management table 610 collection data management table 620 teacher data management table

Claims (5)

A teacher data collection device that collects data related to a specific field for use as machine learning teacher data,
A feature calculation unit that calculates a first feature vector that is a feature vector of data related to a specific field registered in advance;
A generating unit that generates a search condition used for collecting data related to the specific field from the first feature vector;
A collection unit for collecting data on the specific field based on the generated search condition;
When the feature calculation unit calculates a second feature vector that is a feature vector of the collected data, a similarity calculation unit that calculates a similarity between the second feature vector and the first feature vector;
An extraction unit for extracting the collected data having the similarity in a predetermined range as the teacher data;
A teacher data collection device comprising: The generation unit generates, as the search condition, a combination of words whose degree used for the data related to the specific field registered in advance is a predetermined value or more based on the first feature vector. The teacher data collection device described in 1. The teacher data according to claim 1, wherein the generation unit uses a combination of words having a weight value for each word included in the pre-registered data related to a specific field as a search condition. Collection device. An information processing method for collecting data related to a specific field for use as machine learning teacher data,
A feature calculation step of calculating a first feature vector that is a feature vector of data related to a specific field registered in advance;
Generating a search condition used for collecting data related to the specific field from the first feature vector;
A collection step of collecting data on the specific field based on the generated search condition;
Calculating a second feature vector which is a feature vector of the collected data, and calculating a similarity between the second feature vector and the first feature vector;
An extraction step of extracting the collected data having the similarity in a predetermined range as the teacher data;
A teacher data collection method. A program for collecting data about a specific field for use as machine learning teacher data,
Computer
A feature calculation unit for calculating a first feature vector that is a feature vector of data related to a specific field registered in advance;
A generating unit that generates a search condition used to collect data related to the specific field from the first feature vector;
A collecting unit for collecting data on the specific field based on the generated search condition;
When the feature calculation unit calculates a second feature vector that is a feature vector of the collected data, a similarity calculation unit that calculates a similarity between the second feature vector and the first feature vector;
An extraction unit for extracting the collected data having the similarity within a predetermined range as the teacher data;
Program to function as.

Images