JP6875457B2 - Acquisition device, acquisition method, and acquisition program - Google Patents

Description

The present invention relates to an acquisition device, an acquisition method, and an acquisition program.

Conventionally, techniques for estimating relationships between various objects have been known. As an example of such a technique, a score indicating a temporal relationship between events indicated by a text and a score indicating a causal relationship are individually calculated, and the relationship between events is estimated based on each calculated score. The technology to do is known.

“Joint reasoning for temporal and causal relations”, Qiang Ning, Zhili Feng, Hao Wu, and Dan Roth. 2018, In Proceedings of the 56th Annual Meeting of the Association for Computational (ACL), pages 2278-2288

However, it cannot be said that the above-mentioned technique has acquired useful information for learning a model for estimating a causal relationship.

For example, in order to easily estimate the causal relationship, a method of learning a model for estimating the causal relationship between events when text is input can be considered. However, in the above-mentioned technique, it is only estimated whether or not there is a causal relationship between the events indicated by each text based on the frequency and distance at which the text indicating each event appears in a certain sentence. Therefore, it cannot be said that the information useful for learning the model for estimating the causal relationship is acquired.

The present application has been made in view of the above, and an object of the present application is to obtain information useful for learning a model for estimating a causal relationship.

The acquisition device according to the present application includes a specific unit that identifies a second target having a causal relationship with the first target, information based on the first content, and the second target from the first content explaining the first target. It is characterized by having an acquisition unit that acquires a set with information based on the second content to be described as training data of a classification model that classifies the presence or absence of a causal relationship.

According to one aspect of the embodiment, information useful for learning a model for estimating a causal relationship can be obtained.

FIG. 1 is a diagram showing an example of processing executed by the information providing device according to the embodiment. FIG. 2 is a diagram showing a configuration example of the information providing device according to the embodiment. FIG. 3 is a diagram showing an example of information registered in the content database according to the embodiment. FIG. 4 is a diagram showing an example of information registered in the target database according to the embodiment. FIG. 5 is a diagram showing an example of information registered in the learning data database according to the embodiment. FIG. 6 is a diagram showing an example of information registered in the causal relationship expression table according to the embodiment. FIG. 7 is a flowchart showing an example of a flow of processing executed by the information providing device according to the embodiment. FIG. 8 is a first diagram showing an example of experimental results of the acquisition process and the learning process according to the embodiment. FIG. 9 is a second diagram showing an example of experimental results of the acquisition process and the learning process according to the embodiment. FIG. 10 is a diagram showing an example of a hardware configuration.

Hereinafter, the acquisition device, the acquisition method, and the embodiment for implementing the acquisition program (hereinafter referred to as “the embodiment”) according to the present application will be described in detail with reference to the drawings. It should be noted that this embodiment does not limit the acquisition device, acquisition method, and acquisition program according to the present application. In addition, each embodiment can be appropriately combined as long as the processing contents do not contradict each other. Further, in each of the following embodiments, the same parts are designated by the same reference numerals, and duplicate description is omitted.

[1. Information provider]
First, with reference to FIG. 1, an example of processing executed by the information providing device 10 which is an example of the determination device will be described. FIG. 1 is a diagram showing an example of processing executed by the information providing device according to the embodiment. In FIG. 1, as a process executed by the information providing device 10, an acquisition process for acquiring learning data for learning a model for classifying whether or not the designated contents are contents indicating an object having a causal relationship with each other. , Is the content indicating a target having a causal relationship in the learning process in which the model is trained using the learning data acquired by the acquisition process and the content specified using the model trained by the learning process? An example of the classification process for classifying whether or not to use will be described.

[1-1. Overview of information providing equipment]
The information providing device 10 is an information processing device capable of communicating with the data server 100 and the terminal device 200 via a predetermined network N (see, for example, FIG. 2) such as the Internet, and is, for example, a server device or a cloud system. It is realized by such as. The information providing device 10 may be able to communicate with an arbitrary number of data servers 100 and terminal devices 200 via the network N.

The data server 100 is an information processing device that manages various types of information, and is realized by, for example, a server device or a cloud system. For example, various contents are registered in the data server 100, and a service for distributing the contents is provided in response to a request from the user.

The terminal device 200 is a terminal device used by a user, and is realized by, for example, a PC (Personal Computer), a server device, various smart devices, and the like. For example, the terminal device 200 has a function of communicating with the information providing device 10 and outputting information as a communication result when receiving input of various information from the user.

[1-2. About the contents distributed by the data server 100]
Here, the contents distributed by the data server 100 will be described. For example, the data server 100 is registered with contents that explain specific objects such as objects, substances, people, places, etc. indicated by various nouns, and events such as disease names, events, and historical events. It is a server device that distributes such contents. In the following description, specific objects and events described by the content are collectively referred to as "objects".

Further, in the data server 100, a plurality of contents explaining the same target are registered in different languages. For example, the data server 100 is registered with a plurality of contents explaining the same target in various languages such as Japanese, English, German, French, Spanish, and Czech. It should be noted that the data server 100 does not need to register the contents described in all the languages supported by the data server 100 for a certain target, and only the contents in some languages may be registered. For example, contents in Japanese, English, and German may be registered in the data server 100, respectively, and contents in English, French, and French may be registered in the data server 100, respectively.

Here, the content registered in the data server 100 may be content that is uniquely edited by a plurality of editors. For example, when one of the editors registers or edits a certain content, the data server 100 may reflect the registration or editing when the approval by the other editor is given.

In addition, links to other contents may be set in the contents registered in the data server 100. For example, among the contents registered in the data server 100, the content explaining the "apple" may be set with a link to the content explaining the target such as "fruit" or "Rosaceae". .. To give a more specific example, the data server 100 may be provided with a link to a content explaining another target with respect to a character string or an image indicating another target appearing in the description of the target. ..

Further, the data server 100 may be registered with a database indicating what kind of content is registered for a certain target. For example, in the data server 100, a unique identifier (ID: Identifier) is assigned to various targets, and a database in which the contents of each language that explains the corresponding target are linked to this identifier is registered. It may have been done. In the following description, a database in which such an identifier of a target and an identifier of a content explaining the target are associated with each other may be collectively referred to as a "target database".

The data server 100 realizes a service that provides a description of each target to the user by providing the various contents described above and the information registered in the database. To give a specific service name, the data server 100 corresponds to, for example, a service such as Wikipedia or Wikidata.

[1-3. About the use of causality]
Here, the use of the causal relationship between the two objects will be described. For example, when a first object is the cause of the second object and the second object is the result of the first object, it can be said that the first object and the second object have a causal relationship. For example, when tobacco is the first target and lung cancer is the second target, tobacco has a high probability of being the cause of lung cancer, and lung cancer has a high probability of being the result of cigarettes, and therefore has a causal relationship (has a causal relationship). There is a high possibility). In addition, it can be said that the group of protectionism and trade war, the group of wildfire and air pollution, the group of the Treaty of Versailles and World War II, etc. have a certain causal relationship.

Such a causal relationship between objects can be used for various services. For example, a knowledge database (CKB: Causality Knowledge Base) that targets an entity and shows the causal relationship between the entities is used, for example, for a QA (Question Answering) service that outputs answers to questions, reading comprehension, event prediction, and the like. be able to.

[1-4. About the outline of the acquisition process]
Here, when information about a certain object and information about another object are input, a model for classifying whether or not the two input objects have a causal relationship (hereinafter, referred to as "classification model"). It is considered that it is possible to automatically estimate whether or not each target has a causal relationship by using the content distributed by the data server 100. However, in the prior art, it is only determined whether or not the object indicated by the word appearing in a certain sentence has a causal relationship, and a method for obtaining useful information for learning a classification model has not been proposed. It was.

Therefore, the information providing device 10 acquires information useful for learning the classification model by executing the following acquisition process. First, the information providing device 10 identifies a second target having a causal relationship with the first target from the first content explaining the first target. Then, the information providing device 10 acquires a set of information based on the first content and information based on the second content explaining the second object as learning data of a classification model for classifying the presence or absence of a causal relationship.

For example, the information providing device 10 acquires the target content having a so-called causal relationship as learning data from the content provided by the data server 100. More specifically, the information providing device 10 identifies a second target that is the result when a certain first target is the cause, or a second target that is the cause when the first target is the result. , The learning data is acquired from the pair of the content explaining the first object and the content explaining the second object in the same language as the content.

For example, when a first target causes the second target, the first content explaining the first target is considered to include information suggesting that the second target is the result of the first target. , The second content explaining the second object is considered to include information suggesting that the first object is the cause of the second object. Therefore, the set of the information based on the first content and the information based on the second content is useful for learning a classification model that classifies whether or not the two input information and the corresponding two objects have a causal relationship. Information is considered to be.

Therefore, the information providing device 10 acquires a set of information based on the first content explaining the first object and information based on the second content explaining the second object having a causal relationship with the first object as learning data. To do. As a result, the information providing device 10 can acquire information useful for learning the classification model.

[1-5. About the outline of learning process and classification process]
Further, the information providing device 10 causes the model to learn the features of the learning data acquired by the above-mentioned acquisition process. For example, when the information providing device 10 inputs a set of information based on the first content explaining the first object and information based on the second content explaining the second object having a causal relationship with the first object. , The model is trained so as to output that the first object and the second object have a causal relationship.

More specifically, the information providing device 10 takes a set of information acquired by the acquisition process as a positive example, and sets two objects having no causal relationship based on the content for explaining each object. Is a negative example. Then, the information providing device 10 trains the model so that when a positive example is input, it outputs that it has a causal relationship, and when it inputs a negative example, it outputs that it does not have a causal relationship. ..

When a set of information is input, the classification model in which such learning is performed can classify whether or not the set of information and the corresponding set of objects have a causal relationship. Therefore, for example, the information providing device 10 generates a set of contents to be distributed by the data server 100, and inputs the set of the generated contents into the classification model, so that the set of the target corresponding to each content has a causal relationship. It can be automatically determined whether or not it has.

Further, when two targets are designated by the user, the information providing device 10 inputs a set of contents corresponding to the two designated targets into the classification model, and designates the set according to the classification result by the classification model. Classification processing may be performed to classify whether or not the two objects have a causal relationship. As a result of such classification processing, the information providing device 10 can accurately estimate whether or not the two designated objects have a causal relationship.

As the classification model described above, any type of model can be adopted. For example, the information providing device 10 may adopt SVM (Support Vector Machine) or DNN (Deep Neural Network) as a classification model. Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Further, the RNN may be an LSTM (Long short-term memory) or the like. That is, any type of model can be adopted as the classification model. Further, the classification model may be a model realized by combining a plurality of models, for example, a model in which a CNN and an RNN are combined.

[1-6. About an example of the processing executed by the information providing device]
Next, an example of the flow of processing executed by the information providing device 10 will be described with reference to FIG. The processing flow shown in FIG. 1 is merely an example, and the information providing device 10 does not need to execute the various processes shown in FIG. 1 in the order shown in FIG.

First, the information providing device 10 collects various contents from the data server 100 (step S1). For example, the information providing device 10 individually collects content explaining each target from a data server 100 in which a plurality of contents explaining the target in different languages are registered, such as Wikipedia. Further, the information providing device 10 collects information from the data server 100 indicating a plurality of contents for explaining the same target and contents for explaining the target in different languages. For example, the information providing device 10 collects various types of information registered in the target database.

Subsequently, the information providing device 10 identifies a pair of targets whose contents show a mutual causal relationship as a seed causal relationship by using a keyword indicating a causal relationship for each language (step S2). For example, the information providing device 10 refers to the target database and selects one of the targets as the first target. Subsequently, the information providing device 10 identifies the content of each language associated with the selected identifier of the first target, that is, the first content.

Then, the information providing device 10 identifies a second target having a causal relationship with the first target by using a preset keyword indicating a causal relationship from the specified first content. For example, when the first object and the second object have a causal relationship, it is considered that the first content includes a sentence to the effect that the first object and the second object have a causal relationship. Such sentences include, for example, a text indicating a first object, a text indicating a second object, and a text indicating a causal relationship between the first object and the second object (that is, a keyword having a causal relationship). In such a sentence, the text of the second object is likely to be included as the object of the text indicating a causal relationship. Therefore, the information providing device 10 searches for a preset keyword from the text included in the first content, and when the keyword is detected, identifies the object of the keyword by using a technique such as morphological analysis. Then, the information providing device 10 specifies the target indicated by the specified object as the second target having a causal relationship with the first target.

For example, the information providing device 10 searches for the Japanese keyword "cause" or the English keyword "Cause" from the specified first content, and when the keyword is detected, the object of the keyword is set as the first target. It may be specified as a second target text indicating the cause. Further, the information providing device 10 searches for the Japanese keyword "effect" and the English keyword "Effect" from the specified first content, and when the keyword is detected, the object of the keyword is set as the first target. It may be specified as a second target text showing the effect. Further, the information providing device 10 performs the same processing for the contents of each language.

As will be clarified in the description described later, in the process executed by the information providing device 10, the process involving human intervention is only the setting of such a keyword. Therefore, the information providing device 10 can realize the acquisition process and the learning process with the influence of human beings minimized.

Here, content such as Wikipedia is provided with a section indicating an "effect" or "cause", and the target that becomes the "effect" or "cause" may be listed in such a section. The objects listed in such a section are likely to have a causal relationship with the content and the corresponding object. Therefore, the information providing device 10 may specify an area indicating another object having a causal relationship with the first object from the first content, and specify another object indicated in the specified area as the second object. ..

For example, when the first content is described in a markup language such as HTML (Hyper Text Markup Language), the information providing device 10 analyzes each section included in the first content by analyzing the tag of the first content. And identify the sections that include keywords such as "effect" and "cause" in the section title. Then, the information providing device 10 identifies the second object described as the object having a causal relationship with the first object from the words listed in the specified section, the explanation included in the specified section, and the like.

Hereinafter, an example of the process in which the information providing device 10 specifies the seed causal relationship will be described with reference to a specific example. For example, when "cigarette" is selected as the first target, the information providing device 10 searches the content explaining "cigarette" for a keyword indicating a causal relationship or a section including the keyword in the title. For example, in the example shown in FIG. 1, the information providing device 10 can indicate a section including the keyword "effect" (that is, another object having a causal relationship with the first object) from the contents explaining "cigarette". Highly probable section) is detected.

Then, the information providing device 10 identifies another object that is likely to have a causal relationship with the "cigarette" from the text included in the detected section. For example, in the example shown in FIG. 1, the content explaining "cigarette" includes a sentence such as "cigarette can cause lung cancer." Here, the subject in this sentence is "cigarette", the object is "lung cancer", and this sentence is included in the section explaining the "effect" of "cigarette". Therefore, the information providing device 10 holds information indicating that the content of the "cigarette" is "lung cancer" is the "effect" of the "cigarette". Further, the information providing device 10 identifies another target having a causal relationship with the "cigarette" by performing the same processing on the multilingual content explaining the target "cigarette".

Further, when "lung cancer" is selected as the first target, the information providing device 10 searches the content explaining "lung cancer" for a keyword indicating a causal relationship or a section including the keyword in the title. For example, in the example shown in FIG. 1, the information providing device 10 detects a section including the keyword “cause” from the content explaining “lung cancer”.

Then, the information providing device 10 identifies another subject that is likely to have a causal relationship with "lung cancer" from the text included in the detected section. For example, in the example shown in FIG. 1, the content explaining "lung cancer" includes a sentence such as "there is an influence of cigarettes." Here, the object in this sentence is "cigarette", and this sentence is further included in the section explaining the "cause" of "lung cancer". Therefore, the information providing device 10 holds information indicating that the content of "lung cancer" is "cigarette" as the "cause" of "lung cancer". In addition, the information providing device 10 identifies another target having a causal relationship with "lung cancer" by performing the same processing on the multilingual content explaining the target "lung cancer".

After that, the information providing device 10 identifies the seed causal relationship by matching the causal relationships indicated by the contents of the same language. For example, the information providing device 10 indicates that the second target is the result of the first target in the Japanese content explaining the first target, and the first target in the Japanese content explaining the second target. When indicating that is the cause of the second target, the pair of the first target and the second target is specified as a seed causal relationship. In such a case, the information providing device 10 has information indicating that the first target is the cause of the second target and the second target is the result of the first target (for example, "first target, second target". And information such as "first target → second target") are retained as information indicating the seed causal relationship. More specifically, the information providing device 10 holds information such as "cigarette-> lung cancer" as information indicating a seed causal relationship.

The information providing device 10 indicates that the second target is the cause of the first target in the Japanese content explaining the first target, and the first target in the Japanese content explaining the second target. Is also the result of the second object, the pair of the first object and the second object is specified as a seed causal relationship. In such a case, the information providing device 10 holds information indicating that the first target is the result of the second target and the second target is the cause of the first target as information indicating the seed causal relationship. For example, in the information providing device 10, the first target is "WWII", the second target is "Versailles Treaty", and the "Versailles Treaty" is the cause of "WWII". If the keyword of is included, information such as "Treaty of Versailles-> World War II" may be retained as information indicating the seed causal relationship.

The information providing device 10 may consider the link relationship of each content when specifying the seed causal relationship. For example, the information providing device 10 may specify a target described by another content to which a link from the first content is set and having a causal relationship with the first target as the second target. To give a more specific example, the information providing device 10 indicates that the second target is the result of the first target in the first content, and that the first target is the cause of the second target in the second content. When a link to the second content is set in the first content and a link to the first content is set in the second content, a pair of the first target and the second target is set. May be a seed causal relationship. Here, the link may be set to, for example, a character string indicating the second target in the text included in the first content, or may be set to the description of the related item.

Further, the information providing device 10 executes the above-described processing for each language. As a result, the information providing device 10 specifies a seed causal relationship indicating a causal relationship between the same objects (that is, between entities) not only in Japanese but also in a plurality of languages such as English, French, and German. It will be.

Subsequently, the information providing device 10 extracts for each language a pair of a context indicating the result extracted from the target article that is the cause and a context indicating the cause extracted from the article that is the result as a positive example. (Step S3). For example, the information providing device 10 extracts a context indicating the second object from the first content, and extracts a context indicating the first object from the second content. Then, the information providing device 10 acquires the extracted set of contexts as a positive example.

For example, when the information providing device 10 specifies the target described as the cause of the first target as the second target in the first content, the text indicating the second target in the first content is used to determine the cause of the causal relationship. The text indicating the first target in the second content is extracted as the context indicating the result of the causal relationship. Further, when the information providing device 10 specifies the target described as the result of the first target as the second target in the first content, the text indicating the second target in the first content is used to display the result of the causal relationship. The text indicating the first target in the second content is extracted as the context indicating the cause of the causal relationship.

To give a more specific example, when the seed causal relationship is “cigarette → lung cancer”, the information providing device 10 identifies the text “lung cancer” from the content explaining the target “cigarette”. Then, the information providing device 10 extracts a sentence including the specified text "lung cancer" and a sentence within a predetermined range from the text "lung cancer" as a context showing the result. In addition, the information providing device 10 identifies the text "cigarette" from the content explaining the target "lung cancer". Then, the information providing device 10 extracts a sentence including the specified text "cigarette" and a sentence within a predetermined range from the text "cigarette" as a context indicating the cause.

The information providing device 10 generates learning data for each seed causal relationship of each language by executing the same processing for each specified seed causal relationship such as English and German as well as Japanese. .. That is, the information providing device 10 identifies a second target having a causal relationship with the first target from a plurality of first contents for explaining the first target in different languages, and information based on the first content for each language. And the set of the information based on the second content explaining the second target in the same language as the first content is acquired as the learning data.

When the first content and the second content are described in the markup language, it is considered that such markup itself can be an index showing a causal relationship. For example, in the content of "cigarette", a link is set to the text of "lung cancer" that is effective when "cigarette" is the cause, or a markup is set to highlight the text of "lung cancer". May have been. Therefore, when the first content or the second content is described in the markup language, the information providing device 10 extracts not only the text but also various markups added to the text as context.

Here, the classification model trained by the training data classifies whether or not the two target information indicated by the two input contents have a causal relationship. More specifically, in the classification model, does the two input contents have a causal relationship with the two objects corresponding to the two input contents based on the expression of the text contained in the two input contents and the characteristics of the markup? It is considered that the classification of whether or not is performed. Therefore, when the text indicating the object itself having a causal relationship is excluded from the training data, it is considered that the classification model can more accurately learn the expression indicating the causal relationship and the characteristics of the markup.

Therefore, the information providing device 10 converts the text indicating the target itself from the text included in the learning data into a predetermined symbol. For example, in the information providing device 10, when the seed causal relationship is "cigarette → lung cancer", the text "cigarette" is referred to as "cause" in the context extracted from the content of "cigarette" or the content of "lung cancer". Convert to a symbol and convert the text "lung cancer" to a symbol such as "effect". That is, the information providing device 10 converts the character string indicating the cause among the character strings included in each content into a predetermined first symbol indicating the cause, and converts the character string indicating the result into a predetermined second symbol. Information is used as training data. As the first symbol and the second symbol, any symbol can be adopted regardless of the language of the content.

Subsequently, the information providing device 10 generates, as a negative example, the same number of context pairs extracted from the target articles having no causal relationship as the positive example (step S4). For example, the information providing device 10 uses a set of information based on the third content explaining the third object and information based on the fourth content explaining the fourth object having no causal relationship with the third object as a negative example. Acquire and train the model using positive and negative examples.

For example, the information providing device 10 selects a third content indicating an arbitrary third object, is another object indicated by a character string included in the selected third object, and is an object of a keyword indicating a causal relationship. The target that is not specified is specified as the fourth target. Further, in the information providing device 10, the character string included in the fourth content explaining the specified fourth object includes the character string indicating the third object, and the character string indicating the third object has a causal relationship. If it is not the object of the indicated keyword, the pair of the third object and the fourth object is specified as a non-seed causal relationship.

Then, the information providing device 10 extracts a context including a character string indicating the fourth object from the third content explaining the third object specified as a non-seed causal relationship, and extracts a context including a character string indicating the fourth object from the fourth content explaining the fourth object. 3 Extract the context including the character string indicating the target. Then, the information providing device 10 takes the extracted set of contexts as a negative example.

For example, the information providing device 10 includes the character string "morphine" as a character string other than the object of the keyword indicating a causal relationship with "cancer pain" in the content explaining "cancer pain", and "morphine". If the content explaining "morphine" contains the character string "cancer pain" as a character string other than the object of the keyword indicating a causal relationship with "morphine", then "cancer pain" and "cancer pain" The pair with "morphine" is regarded as a non-seed causal relationship. Then, the information providing device 10 extracts a context including "morphine" from the content explaining "cancer pain", and extracts a context including "cancer pain" from the content explaining "morphine". Then, the information providing device 10 takes the extracted set of contexts as a negative example.

In addition, the information providing device 10 may make each target a non-seed causal relationship on condition that a mutual link is set for each content. In addition, as for what kind of relationship the target has as a non-seed causal relationship, any condition can be adopted depending on what kind of classification is performed by the classification model. For example, the information providing device 10 may specify the seeded causal relationship by the same processing when specifying the seed causal relationship by using a keyword indicating a predetermined relationship other than the causal relationship.

Then, the information providing device 10 trains the model so as to output that there is a causal relationship when a positive example is input and output that there is no causal relationship when a negative example is input (step). S5). For example, the information providing device 10 learns a model by using an arbitrary learning method such as backpropagation. The information providing device 10 may learn the model by using a technique for learning the text in consideration of the inflected form of the word such as fastText. Further, the information providing device 10 does not have to perform pre-training of the model, but may perform pre-training of the model using predetermined pre-training data. As a result of such processing, the information providing device 10 generates a classification model that classifies whether or not there is a causal relationship between the objects indicated by each context when a set of two contexts is input. Can be done.

For example, the information providing device 10 receives the designation of the content pair to be determined from the terminal device 200 used by the user (step S6). In such a case, the information providing device 10 inputs a pair of contents into the classification model and classifies whether or not the content has a causal relationship (step S7).

For example, the information providing device 10 receives a character string indicating the first object and a character string indicating the second object. In such a case, the information providing device 10 acquires the content # 1 explaining the first target and the content # 2 explaining the second target from the data server 100. Then, the information providing device 10 inputs the content # 1 and the content # 2 into the classification model, and obtains the classification result by the classification model. Then, the information providing device 10 provides the terminal device 200 with information indicating the classification result (step S8).

As described above, the information providing device 10 seeds two objects having a causal relationship from the contents explaining various objects based on the keyword indicating the causal relationship and the reciprocity of the explanations in each content. Identify as a relationship. Then, the information providing device 10 acquires information based on each content of the two objects having a seed causal relationship as learning data of the classification model. As a result of such processing, the information providing device 10 can automatically acquire useful learning data.

For example, in order to learn a classification model that classifies whether or not a causal relationship is present, a method of using an existing database showing the causal relationship can be considered. However, there are not many publicly known databases that are labeled with a causal relationship in this way, and it takes time for the latest information to be reflected. However, since the information providing device 10 can automatically acquire learning data from generally distributed contents such as Wikipedia, more useful learning data can be acquired. Further, in the content created or updated by a plurality of editors, the causal relationship is shown by various expressions, so that the information providing device 10 increases the variation of the learning data and improves the accuracy of the classification model. Can be improved.

Further, in the information providing device 10, the first content is explained as the cause or result of the second object, the first content is explained as the result or cause in the second content, and the first content and the second content are mutually exchanged. When the link is set, the first target and the second target are set as a seed causal relationship. As a result of such processing, the information providing device 10 can identify two targets having a higher possibility of having a causal relationship as a seed causal relationship, although the number of the information providing devices 10 is small, so that the information providing device 10 may have a more causal relationship. It is possible to acquire learning data based on high target content. As a result, the information providing device 10 can improve the learning accuracy of the classification model.

Further, the information providing device 10 generates learning data by using the contents explained in a plurality of languages for a certain object. As a result of such processing, the information providing device 10 specifies a seed causal relationship for each language, and for each specified seed causal relationship, for example, even when using content that avoids redundant expressions. Since learning data is generated, it is possible to emphasize the causal relationships that commonly appear in multiple languages. In other words, the information providing device 10 generates more learning data for causal relationships that appear in common in multiple languages than in causal relationships that appear in a single language. Can be realized in the same learning process as when is given to each causal relationship. Further, the information providing device 10 can increase the number of learning data, so that the coverage can be improved.

[1-7. Variations on learning processing]
In the above-mentioned example, the information providing device 10 trains the model to learn the characteristics of the context extracted from the target content as the seed causal relationship. However, the embodiment is not limited to this.

For example, Wikipedia may include a section listing the causes of a certain target. For example, in the content of the target "lung cancer", "cigarette", "radon gas", "asbestos", "second-hand smoke" and the like may be listed as causes. In such a case, the information providing device 10 may extract a pair of the target described by the content and each target shown in the section listing the causes as a seed causal relationship. For example, the information providing device 10 identifies a seed causal relationship for each target listed as a cause, such as "cigarette-> lung cancer", "radongas-> lung cancer", "asbestos-> lung cancer", and "second-hand smoke-> lung cancer". You may.

Here, when a causal target is listed in the content explaining a certain target, it is considered that the target having a higher probability of causing the cause is more likely to be described first. Therefore, the information providing device 10 may perform learning with a priority according to the order in which each object appears. For example, the information providing device 10 specifies the order in which the second target appears among the plurality of targets described as the cause of the first target in the first content, and the information based on the first content and the second target. The characteristics of the set with the information based on the content may be learned with the priority according to the order in which the second object appears in the first content.

For example, in the content of the target "lung cancer", "cigarette", "radon gas", "asbestos", and "second-hand smoke" are described in order in the section indicating the cause. In such a case, the information providing device 10 identifies the seed causal relationship “cigarette → lung cancer”, “radon gas → lung cancer”, “asbestos → lung cancer”, and “second-hand smoke → lung cancer”, respectively. Further, the information providing device 10 specifies the order in which the text indicating the causal target in each seed causal relationship appears in the content explaining the target that is effective in the seed causal relationship, and sets the specified order as the seed causal relationship. Link. For example, the information providing device 10 associates the seed causal relationship “cigarette → lung cancer” with the order “1”, “radon gas → lung cancer” with the order “2”, and “asbestos → lung cancer”. The order "3" is linked, and the order "4" is linked to "second-hand smoke → lung cancer".

Then, the information providing device 10 causes the model to learn the features of the learning data based on each seed causal relationship with the priority according to the order associated with each seed causal relationship. For example, the information providing device 10 causes the model to learn the learning data generated from the seed causal relationship “cigarette → lung cancer” as the most important learning data, and the learning data generated from “radongas → lung cancer” is the second most important learning. Train the model as data. In addition, the information providing device 10 trains the model as the training data generated from "asbestos-> lung cancer" as the third most important training data, and the training data generated from "second-hand smoke-> lung cancer" is the fourth most important training data. Train the model as training data. For learning considering such priority, various arbitrary learning methods considering weighting can be adopted.

In the same manner as the above-described processing, the information providing device 10 specifies the order in which the second target appears among the plurality of targets described as the effect of the first target in the first content, and the first content. The characteristics of the set of the information based on the information based on the second content and the information based on the second content may be learned with priority according to the order in which the second object appears in the first content. For example, the information providing device 10 specifies a section in which the effects are listed in the content of a certain first target, and the first target and the first target have priorities according to the order in which each second target is described in the specified section. The training data generated from the pair with the two objects may be trained by the model with the priority according to the order in which the second object is described.

[1-8. About content collection]
In the above example, the information providing device 10 collects each content from a service such as Wikipedia, which has a plurality of contents for explaining the target in different languages. Here, the information providing device 10 may execute each of the above-described processes using the content distributed in another service such as Wordnet. For example, the information providing device 10 may collect content from a predetermined news site, such as a news site that distributes content that explains the same content in different languages. That is, the information providing device 10 may collect the content from a highly reliable collection source, and collect the content from any collection source as long as the above-described processing is executed.

[1-9. About the language to be adopted]
Here, the information providing device 10 specifies the seed causal relationship for each language, and generates learning data based on the specified seed causal relationship. Here, the information providing device 10 does not need to specify the seed causal relationship for all languages. For example, when the information providing device 10 learns a classification model for classifying whether or not there is a causal relationship between Japanese contents, the information providing device 10 may specify the seed causal relationship only for Japanese.

Further, the information providing device 10 may execute the above-mentioned acquisition process and learning process only for a language having high coverage. For example, the information providing device 10 identifies a second target having a causal relationship with the first target from a plurality of first contents for explaining the first target in a language in which the amount of content explaining the target satisfies a predetermined condition. For each language, a set of information based on the first content and information based on the second content explaining the second target in the same language as the first content may be acquired as learning data. To give a more specific example, the information providing device 10 can acquire and process languages with wide coverage such as English, German, French, Spanish, Italian, Portuguese, Swedish, Danish, and Polish. It may be the target of learning processing. Further, the information providing device 10 may determine a language to be processed based on the ease of language analysis, for example, a language in which words are separated by spaces.

[2. Configuration of information providing device]
Hereinafter, an example of the functional configuration of the information providing device 10 described above will be described. FIG. 2 is a diagram showing a configuration example of the information providing device according to the embodiment. As shown in FIG. 2, the information providing device 10 includes a communication unit 20, a storage unit 30, and a control unit 40.

The communication unit 20 is realized by, for example, a NIC (Network Interface Card) or the like. Then, the communication unit 20 is connected to the network N by wire or wirelessly, and transmits / receives information to / from the data server 100 and the terminal device 200.

The storage unit 30 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. Further, the storage unit 30 stores the content database 31, the target database 32, the learning data database 33, and the causal relationship expression table 34.

Hereinafter, an example of information registered in the storage unit 30 will be described with reference to FIGS. 3 to 6. The content collected from the data server 100 is registered in the content database 31. For example, FIG. 3 is a diagram showing an example of information registered in the content database according to the embodiment. As shown in FIG. 3, information having items such as "content ID", "target ID", "content data", and "language" is registered in the content database 31.

Here, the "content ID" is an identifier for identifying the content. Further, the "target ID" is an identifier that identifies the target described by the content indicated by the associated "content ID". Further, the "content data" is data distributed as content, such as web content or image data described in HTML format. Further, the "language" is information indicating which language the content indicated by the associated "content ID" is.

For example, in the example shown in FIG. 3, the content database 31 is associated with information such as the content ID “content ID # 1”, the target ID “target # 1”, the content data “content # 1”, and the language “Japanese”. Is registered. Such information is such that the content indicated by the content ID "content ID # 1" is the content explaining the target indicated by the target ID "target # 1" and is created in "Japanese". Indicates that the data is the content data "content # 1".

As shown in FIG. 3, a plurality of contents associated with the same target ID and created in different languages are individually registered in the content database 31. Further, in the example shown in FIG. 3, conceptual values such as "content ID # 1", "target # 1", and "content # 1" are described, but in reality, numerical values for identifying the content or the target are described. , Character strings, various data that compose the content, etc. will be registered. In addition to the information shown in FIG. 3, various information related to each content may be registered in the content database 31.

Information about the target is registered in the target database 32. For example, FIG. 4 is a diagram showing an example of information registered in the target database according to the embodiment. As shown in FIG. 4, information having items such as "target ID", "target", and "content ID" is registered in the target database 32.

Here, the "target" is a character string indicating the name or the like of the target. For example, in the example shown in FIG. 4, the target database 32 is associated with information such as the target ID “target # 1”, the target “lung cancer”, the content ID “content ID # 1”, and the “content ID # 2”. It is registered. In such information, the target indicated by the target ID "target # 1" is "lung cancer", and as the content explaining such the target "lung cancer", the content indicated by the content ID "content ID # 1" and the content Indicates that the content indicated by the ID "content ID # 2" is registered.

In the example shown in FIG. 4, a conceptual value such as "target ID # 1" is described, but in reality, a numerical value or a character string for identifying the target is registered. In addition to the information shown in FIG. 4, various information related to each target may be registered in the target database 32.

The learning data acquired by the acquisition process is registered in the learning data database 33. For example, FIG. 5 is a diagram showing an example of information registered in the learning data database according to the embodiment. As shown in FIG. 5, information having items such as "learning data ID", "type", "first context", and "second context" is registered in the learning data database 33.

Here, the "learning data ID" is an identifier for identifying the learning data. Further, the "type" is information indicating whether the learning data indicated by the associated "learning data ID" is a "positive example" or a "negative example". Further, the "first context" and the "second context" are contexts acquired as learning data. More specifically, the "first context" and the "second context" are contexts extracted from the content describing each object identified as a seed causal relationship.

For example, in the example shown in FIG. 5, the training data database 33 has the training data ID “learning data # 1”, the type “correct example”, the first context “first context # 1”, and the second context “second context”. Information such as "context # 2" is registered in association with each other. Such information is learning data in which the learning data indicated by the learning data ID "learning data # 1" is a "normal example", and is a set of "first context # 1" and "second context # 2". Indicates that the learning data consists of.

In the example shown in FIG. 5, conceptual values such as "learning data ID # 1" and "first context # 1" are described, but in reality, numerical values and character strings for identifying the learning data, Text, markup, etc. extracted as context from the content will be registered. In addition to the information shown in FIG. 5, various information related to the learning data may be registered in the learning data database 33.

In the causal relationship expression table 34, keywords set in advance as keywords indicating the causal relationship are registered. For example, FIG. 6 is a diagram showing an example of information registered in the causal relationship expression table according to the embodiment. As shown in FIG. 6, in the causal relationship expression table 34, information such as "detection term", "type", and "language" is registered in association with each other.

Here, the "detection term" is a keyword that detects from the content as a keyword indicating a causal relationship. Further, the "type" is information indicating whether the object of the associated detection term is the cause or the result. Further, the "language" is information indicating the language of the content to be detected of the associated detection term.

For example, in the example shown in FIG. 6, information such as the detection term "cause", the type "cause", and the language "Japanese" are registered in association with each other. In such information, the detection term "cause" is registered as a keyword detected from the content of "Japanese", and the object of the "cause" is the object "cause" explained by the content. Indicates that the target is the "cause". In addition to the information shown in FIG. 6, arbitrary information may be registered in the causal relationship expression table 34.

Returning to FIG. 2, the explanation will be continued. The control unit 40 is a controller, and for example, various programs stored in a storage device inside the information providing device 10 by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) are stored in a RAM or the like. Is realized by executing as a work area. Further, the control unit 40 is a controller, and may be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 2, the control unit 40 includes a collection unit 41, a specific unit 42, an extraction unit 43, an acquisition unit 44, a learning unit 45, and a providing unit 46. The collection unit 41 collects various contents. For example, the collection unit 41 collects a plurality of contents explaining the same target in different languages such as Wikipedia, and registers the collected contents in the content database 31. In addition, the collection unit 41 collects information associated with an identifier that identifies a certain target and content that explains the target, and registers the information in the target database 32.

The identification unit 42 identifies a second target having a causal relationship with the first target from the first content explaining the first target. For example, the specific unit 42 refers to the target database 32 and selects one target to be processed as the first target. Then, the specifying unit 42 identifies the content explaining the selected first target from the target database 32, and acquires the specified content from the content database 31. That is, the specific unit 42 acquires a plurality of contents that explain the first object in different languages. Then, the specifying unit 42 identifies a second target having a causal relationship with the first target from each content. That is, the specifying unit 42 specifies a plurality of first contents explaining the first target by using the target database 32 showing the contents explaining the same target in the content database 31, and the specified plurality of first contents. The second target is specified from.

For example, the specific unit 42 uses a keyword registered in the causal relationship expression table 34, that is, a keyword indicating a preset causal relationship, and a second target having a causal relationship with the first target from each first content. To identify. For example, the specific unit 42 specifies the object of the keyword by using a technique such as morphological analysis, and sets the target indicated by the specified object as the second target. Here, the specifying unit 42 may specify a region indicating another target having a causal relationship with the first target, and may specify another target shown in the specified region as the second target. For example, the specifying unit 42 may specify a section showing the cause of the first target and a section showing the result of the first target, and each of the targets described in the specified section may be specified as the second target. That is, the specific unit 42 may specify the target described as the cause of the first target in the first content as the second target, and the target described as the result of the first target in the first content. May be specified as the second target.

Subsequently, the specific unit 42 acquires the second content explaining the second target from the content database 31, and determines whether or not the first target is explained as a target having a causal relationship with the second target in the second content. judge. For example, when the second target is the cause of the first target in the first content, the specific unit 42 determines whether or not the first target is the result of the second target in the second content. More specifically, the specific unit 42 determines whether the character string indicating the first target in the second content is the object of the keyword indicating the result. Further, for example, when the second target is the result of the first target in the first content, the specific unit 42 determines whether or not the first target is the cause of the second target in the second content. .. More specifically, the specific unit 42 determines whether the character string indicating the first target in the second content is the object of the keyword indicating the cause. That is, the specific unit 42 determines whether or not it is explained in the first content and the second content that the first object and the second object have a causal relationship.

Then, when it is explained that the first object and the second object have a causal relationship in the first content and the second content, the specific unit 42 seeds the first object and the second object in a causal relationship. And. For example, when the first target is the cause of the second target, the specific unit 42 specifies a seed causal relationship such as “first target → second target”. The specific unit 42 specifies the seed causal relationship for each language. Further, the specifying unit 42 may specify a plurality of seed causal relationships indicating a causal relationship between the common first object and the plurality of second objects.

In addition, the specific unit 42 may specify a target that is described by other content to which a link from the first content is set and has a causal relationship with the first target as the second target. For example, the specific unit 42 may use the pair of the first object and the second object as a seed causal relationship when the first content and the second content are linked to each other. Further, the specific unit 42 uses a language in which the amount of content explaining the target satisfies a predetermined condition, that is, a language capable of ensuring coverage, from a plurality of first contents explaining the first target to the first target. A second object having a causal relationship may be specified.

The extraction unit 43 extracts the context indicating the second target from the first content, and extracts the context indicating the first target from the second content. For example, the extraction unit 43 sets text, markup, etc. in a predetermined range including a character string indicating the second object as a context indicating the second object from the first content explaining the first object having a seed causal relationship. Extract. Further, the extraction unit 43 sets text, markup, etc. in a predetermined range including a character string indicating the first object as a context indicating the first object from the second content explaining the second object having a seed causal relationship. Extract. Then, the extraction unit 43 registers each extracted context in the learning data database 33 as a positive example. The extraction unit 43 may extract a context indicating only the second object from the first content and extract a context indicating only the first object from the second content.

The specific unit 42 and the extraction unit 43 generate learning data as a negative example. For example, the specific unit 42 is a set of an arbitrary third object and a fourth object, and the description of the fourth object is included in the third content as an object other than the object having a causal relationship with the third object. A pair of a third object and a fourth object whose description of the third object is included in the fourth content as an object other than the object having a causal relationship with the fourth object is specified as a non-seed causal relationship. The specific unit 42 may specify the pair of the third object and the fourth object as a non-seed causal relationship when the mutual link is set between the third content and the fourth content.

In such a case, the extraction unit 43 selects a predetermined range of text, markup, etc. including a character string indicating the fourth target from the third content explaining the third target having a non-seed causal relationship. Extract as a context that indicates. Further, the extraction unit 43 extracts text, markup, etc. in a predetermined range including a character string indicating the third object from the fourth content explaining the fourth object having a non-seed causal relationship, in a context indicating the third object. Extract as. Then, the extraction unit 43 registers the extracted context set as a negative example in the learning data database 33.

The acquisition unit 44 acquires a set of information based on the first content and information based on the second content explaining the second object as learning data of a classification model for classifying the presence or absence of a causal relationship. For example, the acquisition unit 44 acquires a set of contexts registered in the learning data database 33 as learning data.

For example, when the first target causes the second target, the acquisition unit 44 uses the second content as the content for explaining the result of the causal relationship and the first content as the content for explaining the cause of the causal relationship. Is acquired as training data. Further, when the first target is the result of the second target, the acquisition unit 44 uses the second content as the content for explaining the cause of the causal relationship and the first content as the content for explaining the result of the causal relationship. To get.

Here, the acquisition unit 44 converts the character string indicating the target causing the causal relationship from the character strings included in the first content and the second content into a predetermined first symbol indicating the cause, and the causal relationship. The information obtained by converting the character string indicating the target to be the result into a predetermined second symbol indicating the result is acquired as training data. For example, when the first target is the cause and the second target is the result, the acquisition unit 44 sets the character string indicating the first target among the character strings included in the context extracted from the second content as "". Convert to a symbol indicating the cause such as "cause", and convert the character string indicating the second target from the character strings included in the context extracted from the first content to the symbol indicating the cause such as "result". May be good. Further, when the first target is the result and the second target is the cause, the acquisition unit 44 sets the character string indicating the first target among the character strings included in the context extracted from the second content as "". Converted to a symbol indicating the result such as "result", and among the character strings included in the context extracted from the first content, the character string indicating the second target is converted to a symbol indicating the cause such as "cause". May be good.

As a result of the above-mentioned processing, the acquisition unit 44 uses the context extracted from the contents of the same language as the learning data. That is, the acquisition unit 44 acquires a set of information based on the first content and information based on the second content explaining the second target in the same language as the first content as learning data for each language. ..

In addition, the acquisition unit 44 acquires the learning data based on the seed causal relationship as a positive example, and explains the information based on the third content explaining the third object and the fourth object having no causal relationship with the third object. The pair with the information based on the fourth content is acquired as a negative example. For example, the acquisition unit 44 acquires the same number of negative examples as the positive examples from the learning data database 33.

The learning unit 45 causes the model to learn the features of the acquired learning data. For example, the learning unit 45 outputs a classification result indicating that each object has a causal relationship when a positive example is input, and a classification result indicating that each object does not have a causal relationship when a negative example is input. The model is trained so as to output.

The learning unit 45 may learn the features of each learning data with a priority according to the order in which the objects included in the seed causal relationship that is the source of the learning data appear in the content. For example, the specific unit 42 specifies the order in which the second target appears among the plurality of targets described as the cause of the first target in the first content. Then, the learning unit 45 may learn the characteristics of the set of the information based on the first content and the information based on the second content with the priority according to the order in which the second object appears in the first content. Good.

The providing unit 46 provides various information such as classification results. For example, the providing unit 46 receives from the terminal device 200 a set of objects to be classified as to whether or not they have a causal relationship. In such a case, the providing unit 46 acquires the content explaining each target from the content database 31, and inputs various contexts included in the acquired content into the trained classification model. Then, the providing unit 46 provides the output of the classification model, that is, the classification result of whether or not the target has a causal relationship, to the terminal device 200.

Even if the providing unit 46 outputs the information registered in the learning data database 33, that is, the learning data, to, for example, an external server that creates various knowledge databases, a learning server that trains the model, and the like. Good.

[3. An example of the flow of processing executed by the information providing device]
Subsequently, the flow of processing executed by the information providing device 10 will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of a flow of processing executed by the information providing device according to the embodiment.

First, the information providing device 10 detects a keyword indicating a cause or effect from various articles such as contents (step S101). In such a case, the information providing device 10 identifies the second target as a result in the article of the first target, and if the first target is the cause in the article of the second target, it is regarded as the first target. Let the second target have a seed causal relationship (step S102). Then, the information providing device 10 extracts the context of the second target from the article of the first target of the seed causal relationship, and extracts the context of the first target from the article of the second target (step S103).

Then, the information providing device 10 uses the extracted set of contexts as the learning data of a positive example (step S104). Further, the information providing device 10 generates the same number of exceptional learning data as the positive learning data (step S105). Then, the information providing device 10 learns a model for classifying whether or not the input article pair is a pair of target articles having a causal relationship by using the generated learning data (step S106). End the process.

[5. About experimental results]
Next, as an example of the effects of the acquisition process and the learning process described above, an example of the accuracy of the classification model when the acquisition process and the learning process are performed will be described.

In the experiment, a Wikipedia article was used as the content, a section showing a causal relationship was identified from the content, a target having a causal relationship was identified from the specified section, and a seed causal relationship between the specified target and the target described by the content was generated. The 1524 training data were used as a positive example. In addition, 1524 training data generated from non-seed causal relationships were taken as negative examples. Then, the classification model was trained using the acquired 3048 training data. In the following description, the classification result by the classification model in which such learning is performed is described as PROP.

In the experiment, methods such as SECTION, INFOBOS, RELATED, and ORACLE RE were prepared as comparison targets with PROP, and accuracy (Acc), precision (Prec), and recall (Prec) in each method were prepared. Rec: recall rate) and the F1 value, which is the harmonic mean of the precision rate and the recall rate, were calculated.

Here, in SECTION, the seed causal relationship extracted by the above-mentioned treatment was extracted. Then, the accuracy, precision, recall, and F1 value of whether or not each object included in the extracted seed causal relationship had a causal relationship were set as each value of SECTION.

In addition, INFOBOX has identified other objects that have a causal relationship with the object described by the article from the infobox included in the Wikipedia article. Then, the accuracy, precision, recall, and F1 value of whether or not the object explained by the article and the specified object have a causal relationship were set as each value of INFOBOX.

Further, in RELATED, the value obtained by subtracting the score sr (a, b) represented by the following formula (1) from 1 is calculated as a distance metric, and the calculated distance metric calculates a causal relationship between two objects satisfying a predetermined condition. It was the object to have. Then, the accuracy, precision, recall, and F1 value of whether or not the subject had a causal relationship were set as each value of RELATED. Here, a and b in the formula (1) indicate articles to be processed. Further, A is a set of all articles linked to the article a, and B is a set of all articles linked to the article b. In addition, W indicates all articles existing on Wikipedia.

Further, in ORACLE RE, two objects shown to have a causal relationship in one sentence are set as objects having a causal relationship, and the accuracy, precision, recall rate and F1 of whether or not the objects have a causal relationship are set. The value was taken as each value of ORACLE RE.

FIG. 8 is a first diagram showing an example of experimental results of the acquisition process and the learning process according to the embodiment. As shown in FIG. 8, the conformance rate of SECTION is 100%, and it was shown that an appropriate causal relationship can be automatically extracted by performing the above-mentioned process for specifying the seed causal relationship. Further, as shown in FIG. 8, in PROP, the best F1 value could be obtained.

Subsequently, in order to compare the changes in coverage and accuracy due to multilingualism, each combination of English, German, French, and Spanish was executed as the target of the above-mentioned acquisition process and learning process. For example, English, German, French, and PROP _en intended for all of _{Spanish, de, fr, es, PROP} en , except for the _{Spanish, de, fr,} PROP _en, except for the _{French, de, es,} PROP _en excluding the _{German, fr, es,} English and PROP _en intended for the only _{German, de,} English and PROP _en that only the target _{French, fr,} English and PROP _en intended for the only Spanish Accuracy, precision, recall, and F1 values were calculated for each of the _{, es} , and PROP _{en for English only.}

FIG. 9 is a second diagram showing an example of experimental results of the acquisition process and the learning process according to the embodiment. In the example shown in FIG. 9, the F1 value of each method was marked with a diamond or an asterisk for the results having a statistically significant difference from _{PROP en, de, fr, es} , and PROP _en. As shown in FIG. 9, it was found that when the target language was changed, the F1 value of the classification result improved as the number of articles to be processed (in other words, coverage) increased.

In addition, when 100 samples were extracted from the classification results classified by PROP and verified, the articles were easy to verify in English and the like for a large number of samples. That is, it is possible to easily identify whether the above-mentioned classification result by PROP is a correct classification result by referring to the text or the like of the corresponding content. Therefore, the information providing device 10 can ensure the verifiability of the learning data and the classification model that becomes the learning result.

In addition, the pair of subjects determined to have a causal relationship by PROP often had articles written on Wikipedia in one or more languages. Therefore, the information providing device 10 can improve the translatability of the classification result.

In addition, various contents existing on the network such as Wikipedia may be associated with various knowledge databases such as Freebase, BabelNet, ISBN-13, MeSH, DOI, and FishBase. Of the articles in the pair of subjects determined to have a causal relationship by PROP, the majority were associated with some external knowledge data. Therefore, the information providing device 10 can improve the connectivity between the learning data and the classification model and various knowledge databases.

[6. Modification example]
In the above, an example of the processing by the information providing apparatus 10 has been described. However, the embodiment is not limited to this. Hereinafter, variations of the processing executed by the information providing device 10 will be described.

[6-1. Device configuration〕
The information providing device 10 does not have to manage various information by its own device. For example, each database 31 to 33 and the causal relationship expression table 34 registered in the storage unit 30 may be stored in an external storage server. Further, the information providing device 10 may be realized by an acquisition server having a specific unit 42, an extraction unit 43, and an acquisition unit 44 shown in FIG. 2 and a learning server having a learning unit 45 and a providing unit 46.

[6-2. Others]
Further, among the processes described in the above-described embodiment, all or a part of the processes described as being automatically performed can be manually performed, and conversely, the processes described as being manually performed. It is also possible to automatically perform all or part of the above by a known method. In addition, the processing procedure, specific name, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the illustrated information.

Further, each component of each of the illustrated devices is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically distributed / physically in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

In addition, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other.

[6-3. program〕
Further, the information providing device 10 according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration as shown in FIG. FIG. 10 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to the output device 1010 and the input device 1020, and the arithmetic unit 1030, the primary storage device 1040, the secondary storage device 1050, the output IF (Interface) 1060, the input IF 1070, and the network IF 1080 are connected by the bus 1090. Has.

The arithmetic unit 1030 operates based on a program stored in the primary storage device 1040 or the secondary storage device 1050, a program read from the input device 1020, or the like, and executes various processes. The primary storage device 1040 is a memory device that temporarily stores data used by the arithmetic unit 1030 for various calculations, such as a RAM. Further, the secondary storage device 1050 is a storage device in which data used by the arithmetic unit 1030 for various calculations and various databases are registered, such as a ROM (Read Only Memory), an HDD (Hard Disk Drive), and a flash memory. Is realized by.

The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various information such as a monitor and a printer. For example, USB (Universal Serial Bus), DVI (Digital Visual Interface), and the like. It is realized by a connector of a standard such as HDMI (registered trademark) (High Definition Multimedia Interface). Further, the input IF 1070 is an interface for receiving information from various input devices 1020 such as a mouse, a keyboard, a scanner, and the like, and is realized by, for example, USB.

The input device 1020 includes, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), or a tape. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like. Further, the input device 1020 may be an external storage medium such as a USB memory.

The network IF1080 receives data from another device via the network N and sends it to the arithmetic unit 1030, and also transmits the data generated by the arithmetic unit 1030 to the other device via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040, and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 10, the arithmetic unit 1030 of the computer 1000 realizes the function of the control unit 40 by executing the program loaded on the primary storage device 1040.

[5. effect〕
As described above, the information providing device 10 identifies the second target having a causal relationship with the first target from the first content explaining the first target, and explains the information based on the first content and the second target. The pair with the information based on the second content is acquired as the training data of the classification model that classifies the presence or absence of the causal relationship. Therefore, the information providing device 10 can automatically acquire information useful for learning a classification model that classifies the presence or absence of a causal relationship.

In addition, the information providing device 10 identifies a second target having a causal relationship with the first target from the first content by using a preset keyword indicating a causal relationship. Further, the information providing device 10 specifies an area indicating another object having a causal relationship with the first object from the first content, and specifies another object indicated in the specified area as the second object. For example, the information providing device 10 sets a target having a high possibility of having a causal relationship with the first target as a second target from the section showing the cause or effect of the first content. Further, the information providing device 10 specifies a target to be described by another content to which a link from the first content is set and has a causal relationship with the first target as a second target.

As a result of each of the above-mentioned processes, the information providing device 10 can identify a pair of objects having a high possibility of having a causal relationship as a pair that is a source of learning data.

Further, the information providing device 10 extracts a context indicating the second target from the first content, and extracts a context indicating the first target from the second content. Then, the information providing device 10 acquires the extracted set of contexts as learning data. Therefore, the information providing device 10 can realize learning of a classification model capable of accurately classifying whether or not a pair of targets has a causal relationship from the content explaining the target.

Further, in the first content, the information providing device 10 identifies the target described as the cause of the first target as the second target, sets the second content as the content explaining the cause of the causal relationship, and sets the first content as the content. Acquire a set of contents that explain the result of the causal relationship as training data. Further, in the first content, the information providing device 10 specifies the target described as the result of the first target as the second target, sets the second content as the content explaining the result of the causal relationship, and sets the first content as the content. Acquire a set of contents that explain the cause of the causal relationship as training data. Therefore, the information providing device 10 can realize the learning of the classification model capable of accurately classifying whether or not the target pair has a causal relationship.

Further, the information providing device 10 converts the character string indicating the target causing the causal relationship from the character strings included in the first content and the second content into a predetermined first symbol indicating the cause, and the causal relationship is established. The information obtained by converting the character string indicating the target to be the result into a predetermined second symbol indicating the result is acquired as training data. As a result, the information providing device 10 can realize learning of a classification model that classifies whether or not each object has a causal relationship based on the characteristics of the representation of the context that explains each object.

Further, the information providing device 10 identifies a second target having a causal relationship with the first target from a plurality of first contents for explaining the first target in different languages, and information based on the first content for each language. And the set of the information based on the second content explaining the second target in the same language as the first content is acquired as learning data. Further, the information providing device 10 identifies a second target having a causal relationship with the first target from a plurality of first contents for explaining the first target in a language in which the amount of content explaining the target satisfies a predetermined condition. For each language, a set of information based on the first content and information based on the second content explaining the second target in the same language as the first content is acquired as learning data. Further, the information providing device 10 uses a second database showing contents for explaining the same target in a first database in which a plurality of contents for explaining the target in different languages are registered to explain the first target. The first content of the above is specified, and the second target is specified from the specified plurality of first contents.

By the various processes described above, the information providing device 10 can improve the coverage of the learning data, so that the accuracy of the classification model in which the learning using the learning data is performed can be improved.

Further, the information providing device 10 causes the model to learn the features of the learning data. For example, the information providing device 10 acquires a set of information based on the first content and information based on the second content as a positive example, and also obtains information based on the third content for explaining the third target and a third target. A set of information based on the fourth content explaining the fourth object having no causal relationship is acquired as a negative example, and the model is trained using the positive example and the negative example. Therefore, the information providing device 10 can realize learning of a classification model capable of classifying whether or not an object has a causal relationship with high accuracy.

Further, the information providing device 10 specifies the order in which the second target appears among the plurality of targets described as the cause of the first target in the first content, and the information based on the first content and the second target. The characteristics of the set with the information based on the content are learned with the priority according to the order in which the second object appears in the first content. Therefore, the information providing device 10 can improve the classification accuracy of the classification model.

Although some of the embodiments of the present application have been described in detail with reference to the drawings, these are examples, and various modifications are made based on the knowledge of those skilled in the art, including the embodiments described in the disclosure column of the invention. It is possible to practice the present invention in other improved forms.

Further, the above-mentioned "section, module, unit" can be read as "means" or "circuit". For example, the imparting unit can be read as a specific means or a specific circuit.

10 Information provider 20 Communication unit 30 Storage unit 31 Content database 32 Target database 33 Learning data database 34 Causal relationship expression table 40 Control unit 41 Collection unit 42 Specific unit 43 Extraction unit 44 Acquisition unit 45 Learning unit 46 Providing unit 100 Data server 200 Terminal device

Claims (16)

From the first content explaining the first target, a specific part that identifies the second target having a causal relationship with the first target, and
It is characterized by having an acquisition unit that acquires a set of information based on the first content and information based on the second content explaining the second object as training data of a classification model for classifying the presence or absence of a causal relationship. Acquisition device. The first aspect of claim 1, wherein the specific unit identifies a second object having a causal relationship with the first object from the first content by using a preset keyword indicating a causal relationship. Acquisition device. The specific unit is characterized in that a region indicating another target having a causal relationship with the first target is specified from the first content, and another target shown in the specified region is specified as the second target. The acquisition device according to claim 1 or 2. The specific unit is characterized in that an object described by another content to which a link from the first content is set and having a causal relationship with the first object is specified as the second object. The acquisition device according to any one of claims 1 to 3. It has an extraction unit that extracts a context indicating the second object from the first content and extracts a context indicating the first object from the second content.
The acquisition device according to any one of claims 1 to 4, wherein the acquisition unit acquires a set of contexts extracted by the extraction unit as the learning data. In the first content, the specific unit identifies the target described as the cause of the first target as the second target.
The acquisition unit is characterized in that the acquisition unit acquires the set in which the second content is the content explaining the cause of the causal relationship and the first content is the content explaining the result of the causal relationship as the learning data. Item 4. The acquisition device according to any one of Items 1 to 5. In the first content, the specific unit identifies the target described as the result of the first target as the second target.
The acquisition unit is characterized in that the acquisition unit acquires the set in which the second content is the content explaining the result of the causal relationship and the first content is the content explaining the cause of the causal relationship as the learning data. Item 4. The acquisition device according to any one of Items 1 to 5. Among the character strings included in the first content and the second content, the acquisition unit converts the character string indicating the target causing the causal relationship into a predetermined first symbol indicating the cause, and the result of the causal relationship. The information obtained by converting the character string indicating the target to be a predetermined second symbol indicating the result into the training data is acquired.
The acquisition device according to claim 6 or 7. The specific unit identifies a second object having a causal relationship with the first object from a plurality of first contents for explaining the first object in different languages.
The acquisition unit acquires a set of information based on the first content and information based on the second content explaining the second target in the same language as the first content as the learning data for each language. The acquisition device according to any one of claims 1 to 8, wherein the acquisition device is characterized by the above. The specific unit identifies a second object having a causal relationship with the first object from a plurality of first contents explaining the first object in a language in which the amount of content explaining the object satisfies a predetermined condition.
The acquisition unit acquires a set of information based on the first content and information based on the second content explaining the second target in the same language as the first content as the learning data for each language. The acquisition device according to claim 9, wherein the acquisition device is characterized by the above. The specific unit describes the first object by using a second database showing the contents explaining the same object in the first database in which a plurality of contents explaining the object in different languages are registered. The acquisition device according to claim 9 or 10, wherein one content is specified and the second target is specified from a plurality of specified first contents. The acquisition device according to any one of claims 1 to 11, wherein the acquisition unit has a learning unit that allows a model to learn the characteristics of the learning data acquired by the acquisition unit. The acquisition unit acquires a set of information based on the first content and information based on the second content as a positive example, and also obtains information based on the third content explaining the third target and the third target. And the set with the information based on the 4th content explaining the 4th object having no causal relationship is acquired as a negative example.
The acquisition device according to claim 12, wherein the learning unit learns the model by using the positive example and the negative example. In the first content, the specific unit specifies the order in which the second target appears among the plurality of targets described as the cause of the first target.
The learning unit learns the characteristics of the set of the information based on the first content and the information based on the second content with priority according to the order in which the second object appears in the first content. The acquisition device according to claim 12 or 13. This is the acquisition method executed by the acquisition device.
A specific process for identifying a second target having a causal relationship with the first target from the first content explaining the first target, and
It is characterized by including an acquisition step of acquiring a set of information based on the first content and information based on the second content explaining the second object as training data of a classification model for classifying the presence or absence of a causal relationship. How to get it. A specific procedure for identifying a second target having a causal relationship with the first target from the first content explaining the first target, and
To make a computer execute a procedure of acquiring a set of information based on the first content and information based on the second content explaining the second object as training data of a classification model for classifying the presence or absence of a causal relationship. Acquisition program.

Images